dbx documentation
dbx is an open source tool which is designed to extend Databricks CLI and provide functionality for rapid development lifecycle and CI/CD on Databricks platform.
dbx aims to improve development experience for Data Teams on Databricks, by providing following features:
Flexible deployment configuration
Built-in version management
CLI-based access for CI/CD pipelines
dbx is language-agnostic. You can use it with projects written in any language supported by Databricks Platform.
Follow the Quickstart to install the package and create the first sample project.
- dbx by Databricks Labs
- Quickstart
- CLI Reference
- Integration with orchestration tools
- Job permissions management
- Handling jobless deployments with Run Submit API
- Fixed properties propagation from cluster policies
- Multitask jobs support
- Jinja2 Support: Environment variables, logic and loops
- Core dependency management and build options
- Path adjustment logic during deployment
- Passing environment variables to the deployment configuration
- Named properties support
- Just enough theory for data pipelines CI/CD automation
- Templates
- Examples