Note: Reports Microservices is currently a beta feature of CDAP 6.2.0, and is subject to change without notice.
Reports provides administrators and developers a way to generate comprehensive report of program runs, specify filters based on various options such as namespaces, program status and type of program, within a given time range such as 24 hours or one week or a custom time range. You can also specify the columns that you would like to include in the generated report.
Reports are backed by the ReportGenerationApp system application, which is a Spark service (named ReportGenerationSpark) with endpoints to generate, list, view, download, share, save or delete reports.
All methods or endpoints described in this API have a base URL (typically http://<host>:11015
or https://<host>:10443
) that precedes the resource identifier, as described in the Microservices Conventions. These methods return a status code, as listed in the Microservices Status Codes.
Generating Reports
POST /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports
With Request Json provided as part of the request body.
Request JSON with explanation of JSON fields:
{ "name":<Name of the report>, "start":<start-time-in-seconds>, "end":<end-time-in-seconds>, "fields":[ <include-this-field-1-in-report>, <...>, <include-this-field-N-in-report> ], "filters":[ { // value filter "fieldName":<name-of-the-field-to-filter-on>, "whitelist":[ <acceptable-value-1>, <..> <acceptable-value-N> ] }, { // value filter "fieldName":<name-of-the-field-to-filter-on>, "blacklist":[ <non-acceptable-value-1>, <...> <non-acceptable-value-N> ] }, { // range filter "fieldName":"<name-of-the-field-to-filter-on>", "range":{ "min":<minimum-allowed-value> "max":<maximum-allowed-value> } } ], "sort":[ { "order":"ASCENDING (or) DESCENDING", "fieldName":"<name-of-the-field-to-sort-by>" } ] }
Example request JSON:
{ "name":"Failed, Stopped, Running, Succeeded runs - Jul 23, 2018 00:00am to Jul 24, 2018 21:00pm", "start":1532329200, "end":1532491200, "fields":[ "artifactName", "applicationName", "program", "programType", "namespace", "status", "start", "end" ], "filters":[ { "fieldName":"status", "whitelist":[ "FAILED", "STOPPED", "RUNNING", "COMPLETED", "KILLED" ] }, { "fieldName":"namespace", "whitelist":[ "default" ] } ] }
On submitting this request, the backend asynchronously launches a spark job to process and generate the report. The immediate response contains the report id.
{ "id": <report-id> }
You can use this report-id to check the status of the report, if it is running, completed or failed using the info endpoint and download completed reports to get information using download endpoint.
Note: Currently sort cannot be performed on more than one field.
Listing Reports
GET /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports
Query Param | Description |
---|---|
offset | offset to read reports from |
limit | limit on the number of reports to return |
Response JSON with explanation for fields:
{ "offset":<offset>, "limit":<limit>, "total":<total-reports-size-returned>, "reports":[ { "id":<report-id>, "name":<report-name>, "created":<creation-time-seconds>, "expiry":<report-expiry-time-seconds>, "status":<report-status> }, ... ] }
Example Response JSON:
{ "offset":0, "limit":20, "total":7, "reports":[ { "id":"18a782b5-8fd4-11e8-ae71-acde48001122", "name":"Failed, Succeeded, Running, Stopped runs - Jul 24, 2018 22:29pm to Jul 24, 2018 23:29pm", "created":1532500174, "expiry":1532672986, "status":"COMPLETED" }, ... { "id":"09ec25f4-8fd4-11e8-9d5d-acde48001122", "name":"Failed, Succeeded, Running, Stopped runs - Jul 24, 2018 22:29pm to Jul 24, 2018 23:29pm", "created":1532500149, "expiry":1532672965, "status":"COMPLETED" } ] }
Sharing Reports
On Authentication enabled clusters, reports are isolated by logged-in users who generated the reports, If you wish, you can generate an encrypted share-id for your report, that can be used by others to access the report.
POST /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/<report-id>/share
Response JSON:
{ "shareId":<share-id> }
Viewing Report Status
The info endpoint can be used to get the report status and summary for a report. It can be queried either by providing the report-id or by providing a share-id through query param.
GET using Report-Id : /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/info?report-id=<report-id> GET using Share-Id : /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/info?share-id=<share-id> { "request":{ <report-generation-request> }, "summary":{ "namespaces":[ { "namespace":"<namespace>", "runs":<runs-in-report-for-the-namespace> }, ... ], "start":<start-time-seconds>, "end":<end-time-seconds>, "artifacts":[ { "name":<artifactName>, "version":<artifactVersion>, "scope":<artifactScope, "runs":<runs-in-report-belonging-to-the-artifact> }, ... ], "durations":{ "min":<minimum-duration-of-runs-in-seconds>, "max":<max-duration-of-runs-in-seconds>, "average":<average-duration-of-runs-in-seconds> }, "starts":{ "oldest":<earliest-started-program-time>, "newest":<latest-started-program-time> }, "owners":[ { "user":<userName>, "runs":<runs-started-by-this-user> }, ... ], "startMethods":[ { "method":"MANUAL/SCHEDULED/TRIGGERED", "runs":<runs-started-by-this-start-method> }, ... ], "recordCount":<total-records-in-report>, "creationTimeMillis":<time-when-report-was-created>, "expirationTimeMillis":<time-when-report-will-be-expired> }, "name":<name-of-the-report>, "created":<report-creation-time-seconds>, "expiry":<report-expiry-time-seconds>, "status":<report-status> }
For reports that are currently running, the response does not contain a report summary
{ "request":{ <report-generation-request> }, "name":<name-of-the-report>, "created":<creation-time>, "status":"RUNNING" }
For failed reports, the response contains the error information.
{ "request":{ <report-generation-request> }, "error" : <error-message>, "name":<name-of-the-report>, "created":<creation-time>, "status":"FAILED" }
Downloading Report Contents
The download endpoint can be used to download the report contents. You can either provide a report id or a share id through a query parameter.
GET using report-id : /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/download?report-id=<report-id> GET using share-id : /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/download?share-id=<share-id> { "offset":<offset>, "limit":<limit-of-records-to-return>, "total":<total-number-of-records-in-report>, "details":[ { "field1":<value>, ... "fieldN":<value> }, ... ] }
Example JSON response:
{ "offset":0, "limit":20, "total":25, "details":[ { "duration":21, "program":"DataPipelineWorkflow", "programType":"Workflow", "applicationName":"testPipeline", "artifactName":"cdap-data-pipeline", "status":"COMPLETED", "end":1532499023, "start":1532499002, "namespace":"default" }, ... ] }
Saving Reports
By default reports expire after 2 weeks, however they can be saved by using the following Microservices endpoint:
POST /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/<report-id>/save
with request body containing the name and description for saving the report:
{ "name" : <report-name>, "description" : <description-for-report> }
Deleting Reports
Unneeded reports (whether generated or failed) can be deleted by making a call to the following endpoint:
DELETE /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/<report-id>