Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Introduce an autoscaling compute profile

Description

Our default compute profile is very small and leads to confusion when customers are trying our product.
Other products usually provide a good out of box experience and require additional settings only for advanced use.
We should:

  • Provide a default autoscaling compute profile that goes to, say, 2..40 nodes instead or fixed 2 nodes by default

  • Provide an easier way to configure basic autoscaling (e.g. an additional input where one can specify a scaling factor. Scaling factor of 20 would mean to use scale up to 20x nodes)

Release Notes

None

Activity

Show:

Sanket SahuFebruary 1, 2022 at 2:11 PM


Yes there will be doc changes for this.
1 new field needs to added to this doc. We can sync up on this once the names / labels are finalised.

Sanket SahuJanuary 31, 2022 at 4:34 PM

1st PR :

Robin RielleyJanuary 28, 2022 at 7:04 PM

I’m not sure what the doc impact is. I don’t see any new properties for the Dataproc compute profile: Is there doc impact or is this just an internal fix?

Sanket SahuJanuary 22, 2022 at 7:26 AM


This is only applicable for the profile with provisioner as “dataproc” ( not even existing data proc ) .
In case of native it’s not applicable.

Robin RielleyJanuary 21, 2022 at 7:29 PM
Edited

Does this apply to the CDAP Sandbox “native” Dataproc compute profile, which you can’t edit or view? Or does this apply to a Dataproc compute profile that you create in CDAP?

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Labels

Fix versions

Priority

Created December 2, 2021 at 4:57 PM
Updated February 28, 2022 at 7:22 AM
Resolved February 2, 2022 at 9:27 PM