Skip to content

Local development

Install prerequisites

Push code to Databricks using the CLI

databricks workspace import-dir <local source folder> <Databricks folder path> --overwrite

Pull code from Databricks using the CLI

databricks workspace export-dir <Databricks folder path> <local dest folder>

Set up your python environment

We use python 3 and virtual environment. We alos recommend using virtual environments. For more details on the Python version and packages that are pre-installed, please check out the runtime doc.

Install packages

We pre-install certain packages to make it easier for developers. You need to install the same ones locally or make sure to add the necessary packages and their versions to your requirements.txt file.

Set environment variables

We set certain environment variables on every app to make it easier to work with Databricks or a few of the most used frameworks.

Port

Likely the most important thing to remember is that the port your app will bind to is specified using DATABRICKS_APP_PORT. It is recommended to also set this environment variable locally and use it to reduce the chance of hard-coding a port.

Request headers

In Databricks, your app is running behind a proxy, and we inject certain headers for you convenience. See the full list in our runtime doc. Locally, if you want to test how you read these headers, one of the easiest ways is to install a browser extension to intercept requests and inject headers. In Chrome, there are several extensions that do something like this.