Fixed properties propagation from cluster policies

When working with cluster policies, sometimes it’s quirky to specify all the fixed fields in the deployment configuration.

To make this simpler, a new functionality is introduced - this functionality is called “property propagation”.

How this works?

• Specify policy_name property in the new_cluster section

• dbx will automatically fetch the policy definition using Cluster Policies API

• Job cluster definition will be updated accordingly with all fixed properties provided from the policy definition during the dbx deploy step

• There is no need to specify policy_id manually, it will be automatically added to the cluster definition

Caveats

• policy_name shall exist in your Cluster Policies

• If there are any conflicting configurations provided (i.e. deployment configuration has value provided, and this value is not the same as in the policy) - deployment will fail

• Job properties adjustment happens only during the dbx deploy step, and it’s applied only when there is a new_cluster with policy_name property specified

• Only the fixed properties will be automatically propagated