You're reading from Distributed Data Systems with Azure Databricks Create, deploy, and manage enterprise data pipelines

Product type Paperback

Published in May 2021

Publisher Packt

ISBN-13 9781838647216

Length 414 pages

Edition 1st Edition

Languages

Python

Tools

Azure

Concepts

Data Science

Author (1):

Alan Bernardo Palacio

View More author details

Table of Contents (17) Chapters

Preface

1. Section 1: Introducing Databricks

2. Chapter 1: Introduction to Azure Databricks FREE CHAPTER

3. Chapter 2: Creating an Azure Databricks Workspace

4. Section 2: Data Pipelines with Databricks

5. Chapter 3: Creating ETL Operations with Azure Databricks

6. Chapter 4: Delta Lake with Azure Databricks

7. Chapter 5: Introducing Delta Engine

8. Chapter 6: Introducing Structured Streaming

9. Section 3: Machine and Deep Learning with Databricks

10. Chapter 7: Using Python Libraries in Azure Databricks

11. Chapter 8: Databricks Runtime for Machine Learning

12. Chapter 9: Databricks Runtime for Deep Learning

13. Chapter 10: Model Tracking and Tuning in Azure Databricks

14. Chapter 11: Managing and Serving Models with MLflow and MLeap

15. Chapter 12: Distributed Deep Learning in Azure Databricks

16. Other Books You May Enjoy

Setting up the Azure Databricks CLI

Azure Databricks comes with a CLI tool that allows us to manage our resources. It's built on top of the Azure Databricks API and allows you to access the workspace, jobs, clusters, libraries, and more. This is an open source project hosted on GitHub.

The Azure Databricks CLI is based on Python 3 and is installed through the following pip command:

pip3 install databricks-cli

You can confirm that the installation was successful by checking the version. If the installation was successful, you will see as a result the current version of the Azure Databricks CLI:

databricks --version

It's good to bear in mind that using the Databricks CLI with firewall-enabled storage containers is not possible and, in that case, it is recommended to use Databricks Connect or AZ storage.

To be able to install the Azure CLI, you will need to have Python 3 already installed and added to the path of the environment you will be working on.