Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

With adaptive execution tuning, you generally need only to specify the range of partitions to use, not the exact partition number. The exact partition number, even if set in pipeline configuration, is ignored when this feature is enabled.

...

On each worker node, the Node Manager reserves a portion of the available machine memory and CPUs for running YARN containers. For example, on a Dataproc cluster, if your worker nodes are n1-standard-4 VMs (4 cpuCPU, 15gb 15 GB memory), each Node Manager will reserve 4cpus 4 CPUs and 12gb 12 GB memory for running YARN containers. The remaining 3gb 3 GB of memory is left for the other Hadoop services running on the node.

...