Path adjustment logic during deployment

During deployment, dbx supports uploading local files and properly referencing them in the job definition. By default, any files referenced in the deployment file (file path shall be relative to the project root) will be uploaded to the dbfs and properly referenced in the job definition.

However, this logic might be too wide defined in some particular cases. To make it more compliant and reliable, since version 0.2.0 dbx deploy supports strict path adjustment policy. To enable this policy, add the following property to the environment:

JSONYAML

{
    "default": {
        "strict_path_adjustment_policy": true,
        "jobs": [
            {
                "name": "your-job-name",
                "new_cluster": {
                    "spark_version": "7.3.x-cpu-ml-scala2.12",
                    "node_type_id": "some-node-type",
                    "aws_attributes": {
                        "first_on_demand": 0,
                        "availability": "SPOT"
                    },
                    "num_workers": 2
                },
                "libraries": [],
                "max_retries": 0,
                "spark_python_task": {
                    "python_file": "file://placeholder_1.py",
                    "parameters": [
                        "file:fuse://placeholder_1.py",
                        "./placeholder_1.py"
                    ]
                }
            }
        ]
    }
}

Since 0.2.0 dbx also supports FUSE-based path replacement (instead of dbfs:/... a FUSE-based path will be provided /dbfs/).