Note: Reports Microservices is currently a beta feature of CDAP 6.2.0, and is subject to change without notice.
Note: Reports Microservices is currently a beta feature of CDAP 6.2.0, and is subject to change without notice.
Reports provides administrators and developers a way to generate comprehensive report of program runs, specify filters based on various options such as namespaces, program status and type of program, within a given time range such as 24 hours or one week or a custom time range. You can also specify the columns that you would like to include in the generated report.
Reports are backed by the ReportGenerationApp system application, which is a Spark service (named ReportGenerationSpark) with endpoints to generate, list, view, download, share, save or delete reports.
All methods or endpoints described in this API have a base URL (typically http://<host>:11015 or https://<host>:10443) that precedes the resource identifier, as described in the Microservices Conventions. These methods return a status code, as listed in the Microservices Status Codes.
POST /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports |
With Request Json provided as part of the request body.
{
"name":<Name of the report>,
"start":<start-time-in-seconds>,
"end":<end-time-in-seconds>,
"fields":[
<include-this-field-1-in-report>,
<...>,
<include-this-field-N-in-report>
],
"filters":[
{
// value filter
"fieldName":<name-of-the-field-to-filter-on>,
"whitelist":[
<acceptable-value-1>,
<..>
<acceptable-value-N>
]
},
{
// value filter
"fieldName":<name-of-the-field-to-filter-on>,
"blacklist":[
<non-acceptable-value-1>,
<...>
<non-acceptable-value-N>
]
},
{
// range filter
"fieldName":"<name-of-the-field-to-filter-on>",
"range":{
"min":<minimum-allowed-value>
"max":<maximum-allowed-value>
}
}
],
"sort":[
{
"order":"ASCENDING (or) DESCENDING",
"fieldName":"<name-of-the-field-to-sort-by>"
}
]
} |
{
"name":"Failed, Stopped, Running, Succeeded runs - Jul 23, 2018 00:00am to Jul 24, 2018 21:00pm",
"start":1532329200,
"end":1532491200,
"fields":[
"artifactName",
"applicationName",
"program",
"programType",
"namespace",
"status",
"start",
"end"
],
"filters":[
{
"fieldName":"status",
"whitelist":[
"FAILED",
"STOPPED",
"RUNNING",
"COMPLETED",
"KILLED"
]
},
{
"fieldName":"namespace",
"whitelist":[
"default"
]
}
]
} |
On submitting this request, the backend asynchronously launches a spark job to process and generate the report. The immediate response contains the report id.
{
"id": <report-id>
} |
You can use this report-id to check the status of the report, if it is running, completed or failed using the info endpoint and download completed reports to get information using download endpoint.
Note: Currently sort cannot be performed on more than one field.
Note: Currently sort cannot be performed on more than one field.
GET /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports |
Query Param | Description |
|---|---|
offset | offset to read reports from |
limit | limit on the number of reports to return |
{
"offset":<offset>,
"limit":<limit>,
"total":<total-reports-size-returned>,
"reports":[
{
"id":<report-id>,
"name":<report-name>,
"created":<creation-time-seconds>,
"expiry":<report-expiry-time-seconds>,
"status":<report-status>
},
...
]
} |
{
"offset":0,
"limit":20,
"total":7,
"reports":[
{
"id":"18a782b5-8fd4-11e8-ae71-acde48001122",
"name":"Failed, Succeeded, Running, Stopped runs - Jul 24, 2018 22:29pm to Jul 24, 2018 23:29pm",
"created":1532500174,
"expiry":1532672986,
"status":"COMPLETED"
},
...
{
"id":"09ec25f4-8fd4-11e8-9d5d-acde48001122",
"name":"Failed, Succeeded, Running, Stopped runs - Jul 24, 2018 22:29pm to Jul 24, 2018 23:29pm",
"created":1532500149,
"expiry":1532672965,
"status":"COMPLETED"
}
]
} |
On Authentication enabled clusters, reports are isolated by logged-in users who generated the reports, If you wish, you can generate an encrypted share-id for your report, that can be used by others to access the report.
POST /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/<report-id>/share |
Response JSON:
{
"shareId":<share-id>
} |
The info endpoint can be used to get the report status and summary for a report. It can be queried either by providing the report-id or by providing a share-id through query param.
GET using Report-Id : /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/info?report-id=<report-id>
GET using Share-Id : /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/info?share-id=<share-id>
{
"request":{
<report-generation-request>
},
"summary":{
"namespaces":[
{
"namespace":"<namespace>",
"runs":<runs-in-report-for-the-namespace>
},
...
],
"start":<start-time-seconds>,
"end":<end-time-seconds>,
"artifacts":[
{
"name":<artifactName>,
"version":<artifactVersion>,
"scope":<artifactScope,
"runs":<runs-in-report-belonging-to-the-artifact>
},
...
],
"durations":{
"min":<minimum-duration-of-runs-in-seconds>,
"max":<max-duration-of-runs-in-seconds>,
"average":<average-duration-of-runs-in-seconds>
},
"starts":{
"oldest":<earliest-started-program-time>,
"newest":<latest-started-program-time>
},
"owners":[
{
"user":<userName>,
"runs":<runs-started-by-this-user>
},
...
],
"startMethods":[
{
"method":"MANUAL/SCHEDULED/TRIGGERED",
"runs":<runs-started-by-this-start-method>
},
...
],
"recordCount":<total-records-in-report>,
"creationTimeMillis":<time-when-report-was-created>,
"expirationTimeMillis":<time-when-report-will-be-expired>
},
"name":<name-of-the-report>,
"created":<report-creation-time-seconds>,
"expiry":<report-expiry-time-seconds>,
"status":<report-status>
} |
For reports that are currently running, the response does not contain a report summary
{
"request":{
<report-generation-request>
},
"name":<name-of-the-report>,
"created":<creation-time>,
"status":"RUNNING"
} |
For failed reports, the response contains the error information.
{
"request":{
<report-generation-request>
},
"error" : <error-message>,
"name":<name-of-the-report>,
"created":<creation-time>,
"status":"FAILED"
} |
The download endpoint can be used to download the report contents. You can either provide a report id or a share id through a query parameter.
GET using report-id : /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/download?report-id=<report-id>
GET using share-id : /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/download?share-id=<share-id>
{
"offset":<offset>,
"limit":<limit-of-records-to-return>,
"total":<total-number-of-records-in-report>,
"details":[
{
"field1":<value>,
...
"fieldN":<value>
},
...
]
} |
{
"offset":0,
"limit":20,
"total":25,
"details":[
{
"duration":21,
"program":"DataPipelineWorkflow",
"programType":"Workflow",
"applicationName":"testPipeline",
"artifactName":"cdap-data-pipeline",
"status":"COMPLETED",
"end":1532499023,
"start":1532499002,
"namespace":"default"
},
...
]
} |
By default reports expire after 2 weeks, however they can be saved by using the following Microservices endpoint:
POST /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/<report-id>/save |
with request body containing the name and description for saving the report:
{
"name" : <report-name>,
"description" : <description-for-report>
} |
Unneeded reports (whether generated or failed) can be deleted by making a call to the following endpoint:
DELETE /v3/namespaces/system/apps/ReportGenerationApp/spark/ReportGenerationSpark/methods/reports/<report-id> |