...
Cluster reuse is implemented in a model similar to connection pooling or thread pooling: any cluster is kept up and running for a specified time after the run is finished. When a new run is started, it will try to find a an idle cluster available that matches the configuration of the compute profile. If one is present, it will be used, otherwise a new cluster will be started.
...
Clusters are not shared. Similar to the regular ephemeral cluster provisioning model, a cluster runs a single pipeline run at a time. A cluster is reused only if it is idle.
What if I have many parallel pipeline runs?
...