...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/apps/<app-id>/versions/<version-id>/create |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| (DEPRECATED) Version of the application, typically following semantic versioning; The |
The request body is a JSON object specifying the artifact to use to create the application, and an optional application configuration. For example:
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/apps/<app-id>/update |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
The request body is a JSON object specifying the updated artifact version and the updated application config. For example, a request body of:
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps[? [artifactName=<artifact-names>] [&artifactVersion=<artifact-version>] [&pageSize=<page-size>] [&pageToken=<page-token>] [&orderBy=<order-by>] [&nameFilter=<name-filter>] [&nameFilterType=<name-filter-type>] [&sortCreationTime=<sort-creation-time>] [&latestOnly=<latest-only>]] |
Parameter | Version Introduced | Description |
---|---|---|
| Namespace ID. | |
| Optional filter to list all applications that use the specified artifact name. Valid values are | |
| Optional filter. This is the version of the artifact given in the | |
| 6.6.0 | Optional filter. The number of pipelines to be returned in the response. If |
| 6.6.0 | Optional filter. If |
| 6.6.0 | Optional filter. Specifies the sorting order. The sorting is by Application Name and then Application Version. Values can be |
| 6.8.0 | Optional filter. Filters the application name based on |
| 6.8.0 | Optional filter. Values can be
|
| 6.8.0 | Optional Boolean. Values can be Default is |
| 6.8.0 | Optional filter. Values can be |
Note: When upgrading the instance from versions < 6.8 to versions >= 6.8, it’s important to follow the “upgrade applications” process in order to have the UI properly render all existing applications.
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps/<app-id> |
Parameter | Description |
---|---|
| Namespace ID. |
| Name of the application. |
Note: To get the creation time of an application and other types of metadata, see Metadata Microservices.
...
Code Block |
---|
{ "name": "POS_Sales_per_Region", "appVersion": "-SNAPSHOT", "description": "Data Pipeline Application", "change": { "author": "joe", "creationTimeMillis": 1668540944833, "latest": true }, "configuration": "{\"resources\":{\"memoryMB\":2048.0,\"virtualCores\":1.0},\"driverResources\":{\"memoryMB\":2048.0,\"virtualCores\":1.0},\"connections\":[{\"from\":\"GCS - POS Sales\",\"to\":\"Wrangler\"},{\"from\":\"Wrangler\",\"to\":\"GCS2\"}],\"comments\":[],\"postActions\":[],\"properties\":{},\"processTimingEnabled\":true,\"stageLoggingEnabled\":false,\"stages\":[{\"name\":\"GCS - POS Sales\",\"plugin\":{\"name\":\"GCSFile\",\"type\":\"batchsource\",\"label\":\"GCS - POS Sales\",\"artifact\":{\"name\":\"google-cloud\",\"version\":\"0.15.3\",\"scope\":\"SYSTEM\"},\"properties\":{\"project\":\"auto-detect\",\"format\":\"text\",\"skipHeader\":\"false\",\"serviceFilePath\":\"auto-detect\",\"filenameOnly\":\"false\",\"recursive\":\"false\",\"encrypted\":\"false\",\"schema\":\"{\\\"type\\\":\\\"record\\\",\\\"name\\\":\\\"etlSchemaBody\\\",\\\"fields\\\":[{\\\"name\\\":\\\"offset\\\",\\\"type\\\":\\\"long\\\"},{\\\"name\\\":\\\"body\\\",\\\"type\\\":\\\"string\\\"}]}\",\"referenceName\":\"pos-sales\",\"path\":\"gs://flat-files-1/POS-r01.txt\"}},\"outputSchema\":\"{\\\"type\\\":\\\"record\\\",\\\"name\\\":\\\"etlSchemaBody\\\",\\\"fields\\\":[{\\\"name\\\":\\\"offset\\\",\\\"type\\\":\\\"long\\\"},{\\\"name\\\":\\\"body\\\",\\\"type\\\":\\\"string\\\"}]}\",\"id\":\"GCS---POS-Sales\"},{\"name\":\"Wrangler\",\"plugin\":{\"name\":\"Wrangler\",\"type\":\"transform\",\"label\":\"Wrangler\",\"artifact\":{\"name\":\"wrangler-transform\",\"version\":\"4.2.3\",\"scope\":\"SYSTEM\"},\"properties\":{\"field\":\"*\",\"precondition\":\"false\",\"threshold\":\"1\",\"schema\":\"{\\\"type\\\":\\\"record\\\",\\\"name\\\":\\\"etlSchemaBody\\\",\\\"fields\\\":[{\\\"name\\\":\\\"Store_Nbr\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Item_Nbr\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"WM_Week\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Daily\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Whse_Nbr\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Whse_Name\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"POS_Sales\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"POS_Qty\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"POS_Cost\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Net_Ship_Qty\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Sales_Type\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Sales_Description\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Max_Shelf_Qty\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Store_Specific_Retail\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Store_Specific_Cost\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Current_HO_Retail\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]}]}\",\"workspaceId\":\"af95f757-a2d8-4efb-90b0-fad0ff2a543b\",\"directives\":\"parse-as-csv :body \\u0027,\\u0027 true\\ndrop body\"}},\"outputSchema\":\"{\\\"type\\\":\\\"record\\\",\\\"name\\\":\\\"etlSchemaBody\\\",\\\"fields\\\":[{\\\"name\\\":\\\"Store_Nbr\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Item_Nbr\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"WM_Week\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Daily\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Whse_Nbr\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Whse_Name\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"POS_Sales\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"POS_Qty\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"POS_Cost\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Net_Ship_Qty\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Sales_Type\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Sales_Description\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Max_Shelf_Qty\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Store_Specific_Retail\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Store_Specific_Cost\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Current_HO_Retail\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]}]}\",\"inputSchema\":[{\"name\":\"GCS - POS Sales\",\"schema\":\"{\\\"type\\\":\\\"record\\\",\\\"name\\\":\\\"etlSchemaBody\\\",\\\"fields\\\":[{\\\"name\\\":\\\"offset\\\",\\\"type\\\":\\\"long\\\"},{\\\"name\\\":\\\"body\\\",\\\"type\\\":\\\"string\\\"}]}\"}],\"id\":\"Wrangler\"},{\"name\":\"GCS2\",\"plugin\":{\"name\":\"GCS\",\"type\":\"batchsink\",\"label\":\"GCS2\",\"artifact\":{\"name\":\"google-cloud\",\"version\":\"0.15.3\",\"scope\":\"SYSTEM\"},\"properties\":{\"project\":\"auto-detect\",\"suffix\":\"yyyy-MM-dd-HH-mm\",\"format\":\"csv\",\"serviceFilePath\":\"auto-detect\",\"location\":\"us\",\"referenceName\":\"pos-sales-per-region\",\"path\":\"gs://flat-files-1\"}},\"inputSchema\":[{\"name\":\"Wrangler\",\"schema\":\"{\\\"type\\\":\\\"record\\\",\\\"name\\\":\\\"etlSchemaBody\\\",\\\"fields\\\":[{\\\"name\\\":\\\"Store_Nbr\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Item_Nbr\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"WM_Week\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Daily\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Whse_Nbr\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Whse_Name\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"POS_Sales\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"POS_Qty\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"POS_Cost\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Net_Ship_Qty\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Sales_Type\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Sales_Description\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Max_Shelf_Qty\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Store_Specific_Retail\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Store_Specific_Cost\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]},{\\\"name\\\":\\\"Current_HO_Retail\\\",\\\"type\\\":[\\\"string\\\",\\\"null\\\"]}]}\"}],\"id\":\"GCS2\"}],\"schedule\":\"0 * * * *\",\"engine\":\"spark\",\"numOfRecordsPreview\":100.0,\"description\":\"Data Pipeline Application\",\"maxConcurrentRuns\":1.0}", "datasets": [], "programs": [ { "type": "Spark", "app": "POS_Sales_per_Region", "name": "phase-1", "description": "Sources 'GCS - POS Sales' to sinks 'GCS2'." }, { "type": "Workflow", "app": "POS_Sales_per_Region", "name": "DataPipelineWorkflow", "description": "Data Pipeline Workflow" } ], "plugins": [ { "id": "GCS - POS Sales", "name": "GCSFile", "type": "batchsource" }, { "id": "GCS2:csv", "name": "csv", "type": "validatingOutputFormat" }, { "id": "GCS - POS Sales:text", "name": "text", "type": "validatingInputFormat" }, { "id": "GCS2", "name": "GCS", "type": "batchsink" }, { "id": "Wrangler", "name": "Wrangler", "type": "transform" } ], "artifact": { "name": "cdap-data-pipeline", "version": "6.8.0", "scope": "SYSTEM" } } |
HTTP Responses
Status Codes | Description |
---|---|
| The event successfully called the method, and the body contains the results |
Upgrade an Application
Notes:
For all realtime pipelines, except pipelines created in CDAP 6.8.0 with a Kafka streaming source, upgrading realtime pipelines to use the latest version of application artifacts is not supported.
Back up all applications before performing the upgrade.
To get the name of the application you want to upgrade, use the GET request listed in “List Applications”.
If upgrading the instance from versions < 6.8 to versions >= 6.8, it’s important to set
latestOnly=false
for the above GET request.
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/apps/<app-id>/upgrade |
Parameter | Description |
---|---|
| Namespace ID. |
| Name of the application. |
| Optional scope filter. If not specified, artifacts in the |
| Optional filter to allow SNAPSHOT version of artifacts for upgrade. Set to TRUE to allow SNAPSHOT version of artifacts for upgrade. Set to FALSE to ignore SNAPSHOT version of artifacts for upgrade. Default is false. |
The response will contain a list of application details containing name, application version, namespace, and entity. For example:
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/upgrade |
Parameter | Description |
---|---|
| Namespace ID. |
| Optional filter to allow artifacts of scope USER and SYSTEM for upgrade. Leave blank to allow artifacts of scope USER and SYSTEM for upgrade. Optional scope filter. If not specified, artifacts in the |
| Optional filter to allow SNAPSHOT version of artifacts for upgrade. Set to TRUE to allow SNAPSHOT version of artifacts for upgrade. Set to FALSE to ignore SNAPSHOT version of artifacts for upgrade. Default is false. |
The request body is a JSON object specifying the updated artifact version and the updated application config.
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps/<app-id>/versions |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application being called |
The response will be a JSON array containing details about the application. The details returned depend on the application.
...
Code Block |
---|
DELETE /v3/namespaces/<namespace-id>/apps/<app-id>/versions/<version-id> |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application to be deleted |
| (DEPRECATED) Version of the application to be deleted |
Note: The app-id
in this URL is the name of the application as configured by the application specification, and not necessarily the same as the name of the JAR file that was used to deploy the application.
...
DELETE namespaces/<namespace-name>/apps/<app-name>/state
Parameter | Description |
| Namespace name. |
| Name of the application with the state to be deleted. |
You might use this endpoint after you upgrade a CDAP instance or stop a streaming pipeline to delete the state for the last processed record.
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps/<app-id>/workflows/<workflow-id>/schedules |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application being called |
| One of |
| Name of the MapReduce, custom service, Spark, worker, or workflow being called |
| Name of the workflow being called, when retrieving schedules |
The response will be a JSON array containing details about the program. The details returned depend on the program type.
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/mapreduce |
Parameter | Description |
---|---|
| Namespace ID |
The response will be a JSON array containing details about the MapReduce program:
Parameter | Description |
---|---|
| One of |
| Name of the application being called |
| Name of the MapReduce, custom service, Spark, worker, or workflow being called |
| Description of the program |
Spark Jobs Associated with a Namespace
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/spark |
Parameter | Description |
---|---|
| Namespace ID |
The response will be a JSON array containing details about the Spark program:
Parameter | Description |
---|---|
| One of |
| Name of the application being called |
| Name of the MapReduce, custom service, Spark, worker, or workflow being called |
| Description of the program |
Workflows Associated with a Namespace
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/workflows |
Parameter | Description |
---|---|
| Namespace ID |
The response will be a JSON array containing details about the workflows:
Parameter | Description |
---|---|
| One of |
| Name of the application being called |
| Name of the MapReduce, custom service, Spark, worker, or workflow being called |
| Description of the program |
Services Associated with a Namespace
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/services |
Parameter | Description |
---|---|
| Namespace ID |
The response will be a JSON array containing details about the services:
Parameter | Description |
---|---|
| One of |
| Name of the application being called |
| Name of the MapReduce, custom service, Spark, worker, or workflow being called |
| Description of the program |
Workers Associated with a Namespace
...
The response will be a JSON array containing details about the workers:
Parameter | Description |
---|---|
| One of |
| Name of the application being called |
| Name of the MapReduce, custom service, Spark, worker, or workflow being called |
| Description of the program |
Spark program status for an application
...
GET /v3/namespaces/<namespace-id>/apps/{app-name}/spark/{program-name}/available
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application. |
| Name of the program. |
Start a Program
After an application is deployed, you can start its MapReduce and Spark programs, custom services, workers, or workflows by submitting an HTTP POST request:
...
When starting a program, you can optionally specify runtime arguments as a JSON map in the request body. CDAP will use these runtime arguments only for this single invocation of the program.
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application being called |
| Version of the application being called |
| One of |
| Name of the MapReduce, custom service, Spark, worker, or workflow being called |
Service, Spark, and Worker programs do not allow concurrent program runs. Programs of these types cannot be started unless the program is in the STOPPED state. MapReduce and Workflow programs support concurrent runs. If you start one of these programs, a new run will be started even if other runs of the program have not finished yet.
...
with a JSON array in the request body consisting of multiple JSON objects with these parameters:
Parameter | Description |
---|---|
| Name of the application being called |
| One of |
| Name of the MapReduce, custom service, Spark, worker, or workflow being started |
| Optional JSON object containing a string to string mapping of runtime arguments to start the program with |
The response will be a JSON array containing a JSON object for each object in the input. Each JSON object will contain these parameters:
Parameter | Description |
---|---|
| Name of the application being called |
| One of |
| Name of the MapReduce, custom service, Spark, worker, or workflow being started |
| The status code from starting an individual JSON object |
| If an error, a description of why the program could not be started (for example, the specified program was not found) |
runId | A UUID that uniquely identifies a run with CDAP |
For example:
Code Block |
---|
POST /v3/namespaces/default/start [ {"appId": "App1", "programType": "Service", "programId": "Service1"}, {"appId": "App1", "programType": "Spark", "programId": "Spark2"}, {"appId": "App2", "programType": "Spark", "programId": "Spark1", "runtimeargs": { "arg1":"val1" }} ]' |
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/apps/<app-id>/versions/<version-id>/<program-type>/<program-id>/stop |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application being called |
| Version of the application being called |
| One of |
| Name of the MapReduce, custom service, Spark, worker, or workflow being stopped |
A program that is in the STOPPED state cannot be stopped. If there are multiple runs of the program in the RUNNING state, this call will stop one of the runs, but not all of the runs.
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/apps/<app-id>/<program-type>/<program-id>/runs/<run-id>/stop |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application being called |
| One of |
| Name of the MapReduce, custom service, Spark, worker, or workflow being called |
| Run id of the run being called |
For example:
Code Block |
---|
POST /v3/namespaces/default/apps/PurchaseHistory/mapreduce/PurchaseHistoryBuilder/runs/631bc459-a9dd-4218-9ea0-d46fb1991f82/stop |
...
with a JSON array in the request body consisting of multiple JSON objects with these parameters:
Parameter | Description |
---|---|
| Name of the application being called |
| One of |
| Name of the MapReduce, custom service, Spark, worker, or workflow being stopped |
The response will be a JSON array containing a JSON object corresponding to each object in the input. Each JSON object will contain these parameters:
Parameter | Description |
---|---|
| Name of the application being called |
| One of |
| Name of the MapReduce, custom service, Spark, worker, or workflow being stopped |
| The status code from stopping an individual JSON object |
| If an error, a description of why the program could not be stopped (for example, the specified program was not found) |
For example:
Code Block |
---|
POST /v3/namespaces/default/stop [ {"appId": "App1", "programType": "Service", "programId": "Service1"}, {"appId": "App1", "programType": "Mapreduce", "programId": "MapReduce2"}, {"appId": "App2", "programType": "Spark", "programId": "Spark2"} ]' |
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps/<app-id>/<program-type>/<program-id>/status |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application being called |
| One of |
| Name of the MapReduce, schedule, custom service, Spark, worker, or workflow being called |
The response will be a JSON array with status of the program. For example, retrieving the status of the UploadService of the program SportResults:
...
with a JSON array in the request body consisting of multiple JSON objects with these parameters:
Parameter | Description |
---|---|
| Name of the application being called |
| One of |
| Name of the MapReduce, schedule, custom service, Spark, worker, or workflow being called |
The response will be the same JSON array as submitted with additional parameters for each of the underlying JSON objects:
Parameter | Description |
---|---|
| Maps to the status of an individual JSON object's queried program if the query is valid and the program was found |
| The status code from retrieving the status of an individual JSON object |
| If an error, a description of why the status was not retrieved (for example, the specified program was not found) |
The status
and error
fields are mutually exclusive meaning if there is an error, then there will never be a status and vice versa.
...
Code Block |
---|
PUT /v3/namespaces/<namespace-id>/apps/<app-id>/versions/<version-id>/schedules/<schedule-id> |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| Name of the schedule; it is unique to the application and, if specified, the application version |
| Version of the application, typically following semantic versioning |
The request body is a JSON object specifying the details of the schedule to be created:
...
Note: For any schedule, the program must be for a workflow and the programType
must be set to WORKLFLOW
.
HTTP Responses
Status Codes | Description |
---|---|
| Schedule with the same name already exists |
Update a Schedule
To update a schedule, submit an HTTP POST request:
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/apps/<app-id>/versions/<version-id>/schedules/<schedule-id>/update |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| Name of the schedule; it is unique to the application and, if specified, the application version. |
| Version of the application, typically following semantic versioning. |
The request body is a JSON object specifying the details of the schedule to be updated, and follows the same form as documented in Add a Schedule.
Only changes to the schedule configurations are supported; changes to the schedule name, or to the program associated with it are not allowed. If any properties are provided, they will overwrite all existing properties with what is provided. You must include all properties, even ones you are are not altering.
HTTP Responses
Status Codes | Description |
---|---|
| If the new schedule type does not match the existing schedule type or there are other client errors |
Retrieving a Schedule
To retrieve a schedule in an application, submit an HTTP GET request:
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps/<app-id>/schedules/<schedule-name> |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| Name of the schedule |
The response will contain the schedule in the same form described in this topic in “Add a Schedule”.
...
The response will contain a list of schedules in the same form as described in “Add a Schedule”.
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
Next Scheduled Run Time
To list the next time that the workflow will be be scheduled by a time trigger, use the parameter nextruntime
:
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps/<app-id>/workflows/<workflow-id>/nextruntime |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
Example: Retrieving The Next Runtime
HTTP Method |
|
---|---|
HTTP Response |
|
Description | Retrieves the next runtime of the workflow PurchaseHistoryWorkflow of the application PurchaseHistory |
Next Scheduled Run Time in Batch
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/nextruntime |
Parameter | Description |
---|---|
| Namespace ID |
The request body must be a JSON array of objects with the following parameters:
Parameter | Description |
---|---|
| Name of the application being called |
| Currently, only the Workflow type is supported. |
| Name of the program being called |
The response will be an array of JSON Objects, each of which will contain the three input parameters as well as two of three possible extra fields: "schedules" or “error” if an error occurs.
Parameter | Description |
---|---|
| The next scheduled runtimes for the program defined by the individual JSON object's parameters |
| The status code from retrieving the program runs |
| If an error, a description of why the status was not retrieved (for example, the specified program was not found, or the requested JSON object was missing a parameter) |
Example
HTTP Method |
|
---|---|
HTTP Body |
|
HTTP Response |
|
Description | Attempt to retrieve the next scheduled run of the service Service1 in the application App1, the workflow testWorkflow in the application App1 and the workflow DataPipelineWorkflow in the application App2, all in the namespace default |
Previous Run Time of All Schedules
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/previousruntime |
Parameter | Description |
---|---|
| Namespace ID |
The request body must be a JSON array of objects with the following parameters:
Parameter | Description |
---|---|
| Name of the application being called |
| Currently, only the Workflow type is supported. |
| Name of the program being called |
The response will be an array of JSON Objects, each of which will contain the three input parameters as well as two of three possible extra fields: "schedules" or “error” if an error occurs.
Parameter | Description |
---|---|
| The previous scheduled runtimes for the program defined by the individual JSON object's parameters |
| The status code from retrieving the program runs |
| If an error, a description of why the status was not retrieved (for example, the specified program was not found, or the requested JSON object was missing a parameter) |
Example
HTTP Method |
|
---|---|
HTTP Body |
|
HTTP Response |
|
Description | Attempt to retrieve the previous scheduled run of the service Service1 in the application App1, the workflow testWorkflow in the application App1 and the workflow DataPipelineWorkflow in the application App2, all in the namespace default |
Previous Run Time of a Schedule
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps/<app-id>/workflows/<workflow-id>/previousruntime |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application to be deleted |
| Name of the Workflow |
Example
HTTP Method |
|
---|---|
HTTP Response |
|
Description | Retrieves the previous runtime of the workflow PurchaseHistoryWorkflow of the application PurchaseHistory |
Delete a Schedule
To delete a schedule, submit an HTTP DELETE:
...
Code Block |
---|
DELETE /v3/namespaces/<namespace-id>/apps/<app-id>/versions/<version-id>/schedules/<schedule-id> |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application to be deleted |
| Name of the schedule; it is unique to the application and, if specified, the application version |
| Version of the application to be deleted |
HTTP Responses
Status Codes | Description |
---|---|
| If the schedule given by |
Schedule: Disable and and Enable
...
Note: You can also use suspend and resume instead of disable and enable.
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| Name of the schedule |
Example: Disabling a Schedule
HTTP Method |
|
---|---|
HTTP Response |
|
Description | Disables the schedule DailySchedule of the application PurchaseHistory |
Container Information
To find out the address of an program's container host and the container’s debug port, you can query CDAP for a service’s live info via an HTTP GET method:
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps/<app-id>/<program-type>/<program-id>/live-info |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application being called |
| One of |
| Name of the program (service or worker) |
Example:
Code Block |
---|
GET /v3/namespaces/default/apps/WordCount/flows/WordCounter/live-info |
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/instances |
Parameter | Description |
---|---|
| Namespace ID |
with a JSON array in the request body consisting of multiple JSON objects with these parameters:
Parameter | Description |
---|---|
| Name of the application being called |
| One of |
| Name of the program (service or worker) being called |
The response will be the same JSON array as submitted with additional parameters for each of the underlying JSON objects:
Parameter | Description |
---|---|
| Number of instances the user requested for the program defined by the individual JSON object's parameters |
| Number of instances that are actually running for the program defined by the individual JSON object's parameters. |
| The status code from retrieving the instance count of an individual JSON object |
| If an error, a description of why the status was not retrieved (for example, the specified program was not found, or the requested JSON object was missing a parameter) |
Note: The requested
and provisioned
fields are mutually exclusive of the error
field.
Example
HTTP Method |
|
---|---|
HTTP Body |
|
HTTP Response |
|
Description | Attempt to retrieve the instances of programType Worker in the application MyApp1, and the service handler MyHandler1 in the user service MySvc1 in the application MyApp3, all in the namespace default |
Scaling Services
You can query or change the number of instances of a service by using the instances
parameter with HTTP GET or PUT methods:
...
Code Block |
---|
{ "instances" : <quantity> } |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| Name of the service |
| Number of instances to be used |
Note: You can scale system services using the Monitor HTTP RESTful API Scaling System Services.
...
Code Block |
---|
{ "instances" : <quantity> } |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| Name of the worker |
| Number of instances to be used |
Example
Retrieve the number of instances of the worker DataWorker in the application DemoApp in the namespace default:
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps/<app-id>/<program-type>/<program-id>/runs |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| One of |
| Name of the MapReduce, custom service, Spark, or workflow being called |
You can filter the runs by the status of a program, the start and end times, and can limit the number of returned records:
Query Parameter | Description |
---|---|
|
|
| start timestamp |
| end timestamp |
| maximum number of returned records |
The response will be a JSON array containing a JSON object for each object in the input. Each JSON object will contain these parameters:
Parameter | Description |
---|---|
| A UUID that uniquely identifies a run within CDAP, with the start and end times in seconds since the start of the Epoch (midnight 1/1/1970). Use that |
| The timestamp at which the program was requested to start by the user. |
| The timestamp at which the program actually started. |
| The timestamp at which this run was suspended (if it was suspended). |
| The timestamp at which this run was resumed (if it was resumed after being suspended). |
| The timestamp at which the request to stop this run was made. |
| The timestamp after which the run will be forcefully killed if it does not stop gracefully. |
| The status of the run in question. |
| A map of the properties of the run. Has subfields. |
| The runtime arguments provided to the run serialized as a JSON string. |
| provides information about the cluster on which the run was executed. Has subfields. |
| The current status of the cluster. |
| The number of nodes in the cluster. |
| The compute profile used for the run. |
| The name of the compute profile. |
| The namespace of the compute profile. |
| The profile’s entity type. |
Example: Retrieving Run Records
HTTP Method |
|
---|---|
HTTP Response |
|
Description | Retrieve the run records of the MapReduce ScoreCounter of the application SportResults. |
Retrieving Specific Run Information
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps/<app-id>/<program-type>/<program-id>/runs/<run-id> |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| One of |
| Name of the MapReduce, custom service, Spark, or workflow being called |
| Run id of the run |
The response will be a JSON array containing a JSON object for each object in the input. Each JSON object will contain these parameters:
Parameter | Description |
---|---|
| A UUID that uniquely identifies a run within CDAP, with the start and end times in seconds since the start of the Epoch (midnight 1/1/1970). Use that |
| The timestamp at which the program was requested to start by the user. |
| The timestamp at which the program actually started. |
| The timestamp at which this run was suspended (if it was suspended). |
| The timestamp at which this run was resumed (if it was resumed after being suspended). |
| The timestamp at which the request to stop this run was made. |
| The timestamp after which the run will be forcefully killed if it does not stop gracefully. |
| The status of the run in question. |
| A map of the properties of the run. Has subfields. |
| The runtime arguments provided to the run serialized as a JSON string. |
| provides information about the cluster on which the run was executed. Has subfields. |
| The current status of the cluster. |
| The number of nodes in the cluster. |
| The compute profile used for the run. |
| The name of the compute profile. |
| The namespace of the compute profile. |
| The profile’s entity type. |
Example: Retrieving a Particular Run Record
HTTP Method |
|
---|---|
HTTP Response |
|
Description | Retrieve the run record of the MapReduce ScoreCounter of the application SportResults run b78d0091-da42-11e4-878c-2217c18f435d |
For services, you can retrieve:
...
Example: Retrieving The Most Recent Run
HTTP Method |
|
---|---|
HTTP Response |
|
Description | Retrieve the most recent successful completed run of the service CatalogLookup of the application PurchaseHistory |
Retrieving Run Records in Batch
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/runs |
Parameter | Description |
---|---|
| Namespace ID |
The request body must be a JSON array of objects with the following parameters:
Parameter | Description |
---|---|
| A UUID that uniquely identifies a run within CDAP, with the start and end times in seconds since the start of the Epoch (midnight 1/1/1970). Use that |
| The timestamp at which the program was requested to start by the user. |
| The timestamp at which the program actually started. |
| The timestamp at which this run was suspended (if it was suspended). |
| The timestamp at which this run was resumed (if it was resumed after being suspended). |
| The timestamp at which the request to stop this run was made. |
| The timestamp after which the run will be forcefully killed if it does not stop gracefully. |
| The status of the run in question. |
| A map of the properties of the run. Has subfields. |
| The runtime arguments provided to the run serialized as a JSON string. |
| provides information about the cluster on which the run was executed. Has subfields. |
| The current status of the cluster. |
| The number of nodes in the cluster. |
| The compute profile used for the run. |
| The name of the compute profile. |
| The namespace of the compute profile. |
| The profile’s entity type. |
The response will be an array of JSON Objects, each of which will contain the three input parameters as well as two of three possible extra fields: runs
, which is a list of the latest run record for that program, statusCode
, which maps to the status code for retrieving the runs for that program, and error
if there was an error retrieving runs for that program. The "statusCode" property will always be included, but runs
and error
are mutually exclusive.
Parameter | Description |
---|---|
| The latest run records for the program defined by the individual JSON object's parameters |
| The status code from retrieving the program runs |
| If an error, a description of why the status was not retrieved (for example, the specified program was not found, or the requested JSON object was missing a parameter) |
Example
HTTP Method |
|
---|---|
HTTP Body |
|
HTTP Response |
|
Description | Attempt to retrieve the latest run records of the service Service1 in the application App1, the workflow testWorkflow in the application App1 and the workflow DataPipelineWorkflow in the application App2, all in the namespace default |
Retrieving Run Counts in Batch
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/runcount |
Parameter | Description |
---|---|
| Namespace ID |
The request body must be a JSON array of objects with the following parameters:
Parameter | Description |
---|---|
| Name of the application being called |
| One of |
| Name of the program (mapreduce, spark, workflow, service, or worker) being called |
The response will be an array of Json Objects, each of which will contain the three input parameters as well as two of three possible extra fields -- "runCount", which is count for the program run, "statusCode", which maps to the status code for retrieving the run count for that program, and "error" if there was an error retrieving runs for that program. The "statusCode" property will always be included, but "runCount" and "error" are mutually exclusive.
Parameter | Description |
---|---|
| The number of program runs for the program defined by the individual JSON object's parameters over the entire lifetime |
| The status code from retrieving the program run count |
| If an error, a description of why the status was not retrieved (for example, the specified program was not found, or the requested JSON object was missing a parameter) |
Example
HTTP Method |
|
---|---|
HTTP Body |
|
HTTP Response |
|
Description | Attempt to retrieve the run count of the service Service1 in the application App1, the workflow testWorkflow in the application App1 and the workflow DataPipelineWorkflow in the application App2, all in the namespace default |
Retrieving Specific Run Count
...
Code Block |
---|
GET /v3/namespaces/<namespace-id>/apps/{app-id}/{program-type}/{program-id}/runcount |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| One of |
| Name of the program (mapreduce, spark, workflow, service, or worker) being called |
Example
HTTP Method |
|
---|---|
HTTP Response |
|
Description | Retrieve the run count of the workflow DataPipelineWorkflow of the application myApp |
Workflow Runs: Suspend and Resume
...
Code Block |
---|
POST /v3/namespaces/<namespace-id>/apps/<app-id>/workflows/<workflow-id>/runs/<run-id>/suspend POST /v3/namespaces/<namespace-id>/apps/<app-id>/workflows/<workflow-id>/runs/<run-id>/resume |
Parameter | Description |
---|---|
| Namespace ID |
| Name of the application |
| Name of the workflow |
| UUID of the workflow run |
Example: Suspending a Workflow
HTTP Method |
|
---|---|
HTTP Response |
|
Description | Suspends the run |