API Requirements

Graph

Information Provided:

1. List of namespaces
2. Start Time
3. End Time
4. Time Resolution

Information Needed:

1. Memory Usage over time per namespace, cluster, and max available
2. Core Usage over time per namespace,  cluster, and max available
3. Bucketed over time resolution aggregate. (The aggregate, we should be able to identify pipeline vs custom app):
   1. Manual start
   2. Scheduled start
   3. Status (RUNNING, COMPLETED, FAILED)
   4. Delay between STARTING and RUNNING
4. If start time and end time is for future date, show the scheduled apps

Details when Graph Time Range is Clicked

Information Provided:

1. List of Namespace
2. Start Time
3. End Time

Information Needed:

1. Entity Details:
   1. Namespace
   2. App Name
   3. Program Type
   4. Program Name
   5. Parent Artifact
   6. Duration
   7. User
   8. Start Method (time schedule, trigger, manual)
   9. Status

Reports View

Information Provided:

1. List of namespaces
2. List of statuses
3. Start Time
4. End Time

Information Needed:

1. Entity Details:
   1. Namespace
   2. App Name
   3. Program Type
   4. Program Name
   5. Parent Artifact
   6. Duration
   7. User
   8. Start Method
   9. Status
   10. Runtime Arguments
   11. Memory Usage
   12. Number of CPU
   13. Number of Containers
   14. Number of Log Warnings
   15. Number of Log Errors
   16. Number of records out
2. Summary Counts:
   1. Runs per namespace
   2. Time range
   3. Pipelines (Realtime vs Batch), custom apps
   4. Durations: min, max, average
   5. Last Started: Oldest and Newest
   6. List of users & count per user
   7. List of start method & count per methods

Action Items:

1. What happens to older version of CDAP that gets upgraded to 5.0.0 that doesn’t have some information (ie. program start methods)
2. Future timeline (design should get updated, grey out the statuses and manually started in graph)
3. Feasibility of features (core & memory usage, start methods for programs)
4. How should the runs list be displayed, Batch vs Realtime vs Custom Apps (collapsed by workflow? What about if the programs started outside workflow?)
5. Permalink in Reports view? Unsure about time ranges, can we get exact time range? authorization: if the user getting the permalink do not have access to some of the namespaces, etc.
6. Need to figure out how to generate the summary section in Reports view. Pre-aggregate from backend? Pagination issue with the list view