Page Comparison

...

Defines whether the Dataproc autoscaling with predefined Autoscaling policy by CDAP will be enabled.

When you enable predefined autoscaling:

• The properties Number of primary workers, Number of secondary workers, and Autoscaling policy are not considered.

• The worker machine type/configuration is the same as the chosen profile's.

• Turning off the Use predefined Autoscaling toggle disables predefined autoscaling, and then runs the original behavior of the profile.

You can also use a Runtime argument to enable autoscaling for a pipeline run.

system.profile.properties.enablePredefinedAutoScaling = true.

Number of Primary Workers

...