Chapter 5: Integrating with Azure Key Vault, App Configuration, and Log Analytics
These days, most data processing or data ingestion pipelines read or write data from or to various external sources such as Azure Blob/ADLS Gen-2, SQL, Cosmos, and Synapse. To access these sources, you will need credentials. Storing these credentials, such as a storage account key or SQL password, in a notebook is not an option from a security standpoint and not a recommended approach when deploying these data ingestion or processing pipelines to production. To overcome these problems, Microsoft provides Azure Key Vault, where you can store credentials such as username and password, and access them securely from your notebooks. Apart from Key Vault, we can also use App Configuration to store these passwords and access them from Databricks notebooks.
By the end of this chapter, you will have learned how to create an Azure Key Vault from the Azure portal and the Azure CLI, as well as how to access Key...
