Use YarnClient instead of making an http request to Yarn webapp, for cluster metrics
Description
In DistributedProgramRuntimeService, we make an http request to YARN web app to fetch the cluster metrics, such as "resources.total.memory" and "resources.available.memory". We may be able to do this using YarnClient, by getting the NodeReports, and summing up their metrics.
Release Notes
Use YarnClient instead of the YARN Http API to fetch node reports
In DistributedProgramRuntimeService, we make an http request to YARN web app to fetch the cluster metrics, such as "resources.total.memory" and "resources.available.memory".
We may be able to do this using YarnClient, by getting the NodeReports, and summing up their metrics.