Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagejava
titleHBase Statistics Output
collapsetrue
{
    "nodes": {
      "totalRegionServers": 37,
      "liveRegionServers": 34,
      "deadRegionServers": 3,
      "masters": 3
    },
    "tables": 56,
    "namespaces": 43
}

TODO: Add output for Kafka, Zookeeper, Sentry, KMS

TODO: CDAP Master Uptime?

Caching

It is not possible to hit HBase/YARN/HDFS for every request from the UI. As a result, the result of the statistics API will have to be cached, with a configurable timeout. Details TBD.time to live. The cache will be keyed by the service provider name, with the value as the statistics for the service provider. In addition to a time-to-live, the cache will also need to be invalidated when:

  1. In the ProgramController and ProgramRuntimeService class hierarchy, so YARN statistics can be updated
  2. In the Entity (app, dataset, stream) lifecycle (creation and deletion) when storage statistics will need to be updated 
  3. P3: In the authorization class hierarchy, when the authorization statistics may need to be updated. However, we may choose to not do this, because we eventually want to get out of the business of managing authorization policies in CDAP
  4. TODO: Invalidation for Kafka and Zookeeper