DBX

DBX is a central tool meant for CI workloads when working with Databricks. You can use it to create a project template and deploy and launch your workflows. Since DBX uses Databricks APIs, it is able to use Databricks workflows. A workflow is a grouping of dbt notebooks or jobs meant to flow together.

These are some of the most important files:

• .dbx/project.json: Organized by environments; used to manage configuration across your project.
• project_folder: Used to store your Python code that isn’t included in notebooks or tests.
• conf/deployment.yml: A YAML-based configuration file that allows you to define the details of Databricks workflows. You can define tasks for dbt notebooks and jobs at the moment.
• notebooks: Used to hold Databricks notebooks.
• tests: Should be used for integration and unit tests, with each in its own subfolder structure.

Important commands

To create your shell project (not required but useful), run the following command...