Use the CDAP Workflow HTTP RESTful API Microservices to access workflow tokens; list and delete local datasets used in a workflow; retrieve the state of nodes that were executed; and retrieve statistics across successful runs of a workflow.
...
All methods or endpoints described in this API have a base URL (typically http://<host>:11015 or https://<host>:10443) that precedes the resource identifier, as described in the RESTful API Conventions. These methods return a status code, as listed in the RESTful API Status Codes.
Workflow Tokens
This interface supports accessing the value of workflow tokens set during runs of a workflow. Returned values can be specified for a particular scope, node or key.
Obtaining a Token's Values
To retrieve the values that were put into the workflow token for a particular run, use an HTTP GET request to the query's URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| UUID of the workflow run |
| Name of a node in the workflow (optional) |
Extending the Request
The request can be extended with query parameters:
Parameter | Description |
|---|---|
| user (default) returns the token for |
| Returns the value for a specified key; if unspecified, returns all keys (optional) |
Comments
If the query was successful, the body will contain the workflow token for a particular workflow run, such as:
| Code Block |
|---|
{
"key1": [
{
"nodeName": "node1",
"value": "value1"
}, {
"nodeName": "node2",
"value": "value2"
}
],
"key2": [
{
"nodeName": "node2",
"value": "v2"
}
]
}
|
HTTP Responses
Status Codes | Description |
|---|---|
| The query was successfully received and the workflow token was returned in the body |
| The app, workflow, run-id, scope or key was not found |
| Internal server error |
Examples
HTTP Request |
|
|---|---|
HTTP Response |
|
Description | Retrieves the token for a specific run of PurchaseHistoryWorkflow |
HTTP Request |
|
HTTP Response |
|
Description | Retrieves the token in the scope System for a specific run of PurchaseHistoryWorkflow |
HTTP Request |
|
HTTP Response |
|
Description | Retrieves the values for the key "key1" from the token in the scope User for a specific run of MyWorkflow |
HTTP Request |
|
HTTP Response |
|
Description | Retrieves the value for the key key1 at node MyExitNode from the token in the scope User for a specific run of MyWorkflow |
Workflow Local Datasets
This interface supports listing and deleting the local datasets associated with a particular workflow run.
Listing Local Datasets
To retrieve the local datasets associated with a particular run, use an HTTP GET request to the query's URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| UUID of the workflow run |
Deleting Local Datasets
To delete the local datasets associated with a particular run, use an HTTP DELETE request to the query's URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| UUID of the workflow run |
Workflow State
The workflow state interface allows you to retrieve the state of all nodes that were executed as a part of a given workflow run.
...
| Code Block |
|---|
{
"PurchaseHistoryBuilder": {
"nodeId": "PurchaseHistoryBuilder",
"nodeStatus": "COMPLETED",
"runId": "53504da2-0ce3-11e6-92bb-da6a50dd7318"
},
"PurchaseReader": {
"nodeId": "PurchaseReader",
"nodeStatus": "COMPLETED"
}
}
|
Workflow Statistics
These requests provide statistics across successful runs of a workflow, in a time interval, for a succession of runs, or between two specific runs. These requests will help in determining which jobs are responsible for delays or problems.
Statistics of Successful Runs
This request returns general statistics about all successful workflow runs in a particular time interval, with an analysis based on a series of (optionally) provided percentiles:
...
If the query was successful, the body will contain a JSON structure of statistics.
Note: When specifying start and end times, in addition to giving an absolute timestamp in seconds, you can specify a relative time, using now minus an increment with units. Examples: now-<n>s, now-<n>m, now-<n>h, or now-<n>d.
HTTP Responses
Status Codes | Description |
|---|---|
| The query was successfully received and the statistics were returned in the body in a JSON format |
| The app, workflow, run-id, scope or key was not found |
| Internal server error |
Example
The query:
| Code Block |
|---|
GET /v3/namespaces/default/apps/Purchase/workflows/PurchaseHistoryWorkflow/statistics? start=1441918778&end=1442005182&percentile=80&percentile=90&percentile=95&percentile=99 |
...
| Code Block |
|---|
{
"startTime": 0,
"endTime": 1442008469,
"runs": 4,
"avgRunTime": 7.5,
"percentileInformationList": [
{
"percentile": 80.0,
"percentileTimeInSeconds": 9,
"runIdsOverPercentile": [
"e0cc5b98-58cc-11e5-84a1-8cae4cfd0e64"
]
},
{
"percentile": 90.0,
"percentileTimeInSeconds": 9,
"runIdsOverPercentile": [
"e0cc5b98-58cc-11e5-84a1-8cae4cfd0e64"
]
},
{
"percentile": 95.0,
"percentileTimeInSeconds": 9,
"runIdsOverPercentile": [
"e0cc5b98-58cc-11e5-84a1-8cae4cfd0e64"
]
},
{
"percentile": 99.0,
"percentileTimeInSeconds": 9,
"runIdsOverPercentile": [
"e0cc5b98-58cc-11e5-84a1-8cae4cfd0e64"
]
}
],
"nodes": {
"PurchaseHistoryBuilder": {
"avgRunTime": "7.0",
"99.0": "8",
"80.0": "8",
"95.0": "8",
"runs": "4",
"90.0": "8",
"type": "MapReduce"
}
}
}
|
Comparing a Run to Runs Before and After
This request returns a list of workflow metrics, based on a workflow run and a surrounding number of successful runs of the workflow that are spaced apart by a time interval from each other:
| Code Block |
|---|
GET /v3/namespaces/<namespace-id>/apps/<app-id>/workflows/<workflow-id>/runs/<run-id>/statistics?
limit=<limit>&interval=<interval>
|
where
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| UUID of the workflow run |
| The number of the records to compare against (before and after) the run; if not provided, defaults to |
| The time interval with which to space out the runs before and after, with units; if not provided, defaults to |
If the query was successful, the body will contain a JSON structure of statistics.
HTTP Responses
Status Codes | Description |
|---|---|
| The query was successfully received and the statistics were returned in the body in a JSON format |
| The app, workflow, run-id, scope or key was not found |
| Internal server error |
Example
The query:
| Code Block |
|---|
GET /v3/namespaces/default/apps/Purchase/workflows/PurchaseHistoryWorkflow/runs/ 1873ade0-58d9-11e5-b79d-8cae4cfd0e64/statistics?limit=10&interval=10s' |
...
| Code Block |
|---|
{
"startTimes": {
"1dd36962-58d9-11e5-82ac-8cae4cfd0e64": 1442012523,
"2523aa44-58d9-11e5-90fd-8cae4cfd0e64": 1442012535,
"1873ade0-58d9-11e5-b79d-8cae4cfd0e64": 1442012514
},
"programNodesList": [
{
"programName": "PurchaseHistoryBuilder",
"workflowProgramDetailsList": [
{
"workflowRunId": "1dd36962-58d9-11e5-82ac-8cae4cfd0e64",
"programRunId": "1e1c3233-58d9-11e5-a7ff-8cae4cfd0e64",
"programRunStart": 1442012524,
"metrics": {
"MAP_INPUT_RECORDS": 19,
"REDUCE_OUTPUT_RECORDS": 3,
"timeTaken": 9,
"MAP_OUTPUT_BYTES": 964,
"MAP_OUTPUT_RECORDS": 19,
"REDUCE_INPUT_RECORDS": 19
}
},
{
"workflowRunId": "1873ade0-58d9-11e5-b79d-8cae4cfd0e64",
"programRunId": "188a9141-58d9-11e5-88d1-8cae4cfd0e64",
"programRunStart": 1442012514,
"metrics": {
"MAP_INPUT_RECORDS": 19,
"REDUCE_OUTPUT_RECORDS": 3,
"timeTaken": 7,
"MAP_OUTPUT_BYTES": 964,
"MAP_OUTPUT_RECORDS": 19,
"REDUCE_INPUT_RECORDS": 19
}
}
],
"programType": "Mapreduce"
}
]
}
|
Comparing Two Runs
This request compares the metrics of two runs of a workflow:
...
| Code Block |
|---|
'The other run-id provided was not found : dbd59091-58cb-11e5-a7c6-8cae4cfd0e64' was not found |
HTTP Responses
Status Codes | Description |
|---|---|
| The query was successfully received and the statistics were returned in the body in a JSON format |
| The app, workflow, run-id, scope, or key was not found |
| Internal server error |
Example
Comparing two runs (14b8710a-58cd-11e5-98ca-8cae4cfd0e64 and e0cc5b98-58cc-11e5-84a1-8cae4cfd0e64):
...