Workflow Microservices
- Robin Rielley
Use the CDAP Workflow Microservices to access workflow tokens; list and delete local datasets used in a workflow; retrieve the state of nodes that were executed; and retrieve statistics across successful runs of a workflow.
Additional details and examples are found in Workflows.
All methods or endpoints described in this API have a base URL (typically http://<host>:11015 or https://<host>:10443) that precedes the resource identifier, as described in Microservices Conventions. These methods return a status code, as listed in the Microservices Status Codes.
Workflow Tokens
This interface supports accessing the value of workflow tokens set during runs of a workflow. Returned values can be specified for a particular scope, node or key.
Obtaining a Token's Values
To retrieve the values that were put into the workflow token for a particular run, use an HTTP GET request to the query's URL:
GET /v3/namespaces/<namespace-id>/apps/{app-id}/workflows/{workflow-id}/runs/{run-id}[/nodes/{node-id}]/tokenThe request can (optionally) contain a node-id to limit the request to a particular node in workflow.
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| UUID of the workflow run |
| Name of a node in the workflow (optional) |
Extending the Request
The request can be extended with query parameters:
Parameter | Description |
|---|---|
| user (default) returns the token for |
| Returns the value for a specified key; if unspecified, returns all keys (optional) |
Comments
If the query was successful, the body will contain the workflow token for a particular workflow run, such as:
{
"key1": [
{
"nodeName": "node1",
"value": "value1"
}, {
"nodeName": "node2",
"value": "value2"
}
],
"key2": [
{
"nodeName": "node2",
"value": "v2"
}
]
}HTTP Responses
Status Codes | Description |
|---|---|
| The query was successfully received and the workflow token was returned in the body |
| The app, workflow, run-id, scope or key was not found |
| Internal server error |
Examples
HTTP Request |
|
|---|---|
HTTP Response |
|
Description | Retrieves the token for a specific run of PurchaseHistoryWorkflow |
HTTP Request |
|
HTTP Response |
|
Description | Retrieves the token in the scope System for a specific run of PurchaseHistoryWorkflow |
HTTP Request |
|
HTTP Response |
|
Description | Retrieves the values for the key "key1" from the token in the scope User for a specific run of MyWorkflow |
HTTP Request |
|
HTTP Response |
|
Description | Retrieves the value for the key key1 at node MyExitNode from the token in the scope User for a specific run of MyWorkflow |
Workflow Local Datasets
This interface supports listing and deleting the local datasets associated with a particular workflow run.
Listing Local Datasets
To retrieve the local datasets associated with a particular run, use an HTTP GET request to the query's URL:
GET /v3/namespaces/<namespace-id>/apps/<app-id>/workflows/<workflow-id>/runs/<run-id>/localdatasetswhere
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| UUID of the workflow run |
Deleting Local Datasets
To delete the local datasets associated with a particular run, use an HTTP DELETE request to the query's URL:
where
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| UUID of the workflow run |
Workflow State
The workflow state interface allows you to retrieve the state of all nodes that were executed as a part of a given workflow run.
To retrieve the state of a particular workflow run, use an HTTP GET request to the query's URL:
where
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| UUID of the workflow run |
Example
The query:
would return results similar to these, pretty-printed for display:
Workflow Statistics
These requests provide statistics across successful runs of a workflow, in a time interval, for a succession of runs, or between two specific runs. These requests will help in determining which jobs are responsible for delays or problems.
Statistics of Successful Runs
This request returns general statistics about all successful workflow runs in a particular time interval, with an analysis based on a series of (optionally) provided percentiles:
where
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| Start time of runs (in seconds); if not provided, defaults to |
| End time of runs (in seconds); if not provided, defaults to |
| List of percentiles (each greater than zero and less than 100) to be used for generating statistics; if not provided, defaults to 90 (optional) |
If the query was successful, the body will contain a JSON structure of statistics.
Note: When specifying start and end times, in addition to giving an absolute timestamp in seconds, you can specify a relative time, using now minus an increment with units. Examples: now-<n>s, now-<n>m, now-<n>h, or now-<n>d.
HTTP Responses
Status Codes | Description |
|---|---|
| The query was successfully received and the statistics were returned in the body in a JSON format |
| The app, workflow, run-id, scope or key was not found |
| Internal server error |
Example
The query:
would return results similar to these, pretty-printed for display:
Comparing a Run to Runs Before and After
This request returns a list of workflow metrics, based on a workflow run and a surrounding number of successful runs of the workflow that are spaced apart by a time interval from each other:
where
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| UUID of the workflow run |
| The number of the records to compare against (before and after) the run; if not provided, defaults to |
| The time interval with which to space out the runs before and after, with units; if not provided, defaults to |
If the query was successful, the body will contain a JSON structure of statistics.
HTTP Responses
Status Codes | Description |
|---|---|
| The query was successfully received and the statistics were returned in the body in a JSON format |
| The app, workflow, run-id, scope or key was not found |
| Internal server error |
Example
The query:
would return results similar to these, pretty-printed for display:
Comparing Two Runs
This request compares the metrics of two runs of a workflow:
where
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| UUID of the workflow run |
| UUID of the other workflow run to be used in the comparison |
If the query was successful, the body will contain a JSON structure of statistics. Note that if either of the run-ids is from an unsuccessful run, an error message will be returned:
HTTP Responses
Status Codes | Description |
|---|---|
| The query was successfully received and the statistics were returned in the body in a JSON format |
| The app, workflow, run-id, scope, or key was not found |
| Internal server error |
Example
Comparing two runs (14b8710a-58cd-11e5-98ca-8cae4cfd0e64 and e0cc5b98-58cc-11e5-84a1-8cae4cfd0e64):
would return results similar to these, pretty-printed for display:
Created in 2020 by Google Inc.