Lifecycle Microservices

Owned by Robin Rielley
Last updated: Jan 24, 2024 by Ankit Jain
30 min read

 

Use the CDAP Lifecycle Microservices to deploy or delete applications and manage the lifecycle of MapReduce (DEPRECATED) and Spark programs, custom services, workers, and workflows.

For more information about CDAP components, see CDAP Components.

All methods or endpoints described in this API have a base URL (typically http://<host>:11015 or https://<host>:10443) that precedes the resource identifier, as described in the Microservices Conventions. These methods return a status code, as listed in the Microservices Status Codes.

Application Lifecycle

Create an Application

To create an application, submit an HTTP PUT request:

PUT /v3/namespaces/<namespace-id>/apps/<app-id>

(DEPRECATED) To create an application with a non-default version, submit an HTTP POST request with the version specified:

POST /v3/namespaces/<namespace-id>/apps/<app-id>/versions/<version-id>/create

Parameter

Description

Parameter

Description

namespace-id

Namespace ID

app-id

Name of the application

version-id

(DEPRECATED) Version of the application, typically following semantic versioning;

The version-id defaults to -SNAPSHOT when you don’t specify a version-id.

The request body is a JSON object specifying the artifact to use to create the application, and an optional application configuration. For example:

PUT /v3/namespaces/default/apps/purchaseWordCount { "artifact": { "name": "WordCount", "version": "6.8.0", "scope": "USER" }, "config": { “datasetName”: “purchases” }, "principal":"user/example.net@EXAMPLEKDC.NET", "app.deploy.update.schedules":"true" }

will create an application named purchaseWordCount from the example WordCount artifact. The application will receive the specified config, which will configure the application to create a dataset named purchases instead of using the default dataset name.

Optionally, you can specify a Kerberos principal with which the application should be deployed. If a Kerberos principal is specified, then all the datasets created by the application will be created with the application's Kerberos principal.

Optionally, you can set or reset the flag app.deploy.update.schedules. If true, redeploying an application will modify any schedules that currently exist for the application; if false, redeploying an application does not create any new schedules and existing schedules are neither deleted nor updated.

Update an Application (DEPRECATED)

To update an application, submit an HTTP POST request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID

app-id

Name of the application

The request body is a JSON object specifying the updated artifact version and the updated application config. For example, a request body of:

will update the purchaseWordCount application to use version 6.3.0 of the WordCount artifact, and update the name of the dataset to logs. If no artifact is given, the current artifact will be used.

Only changes to artifact version are supported; changes to the artifact name are not allowed. If no config is given, the current config will be used. If the config key is present, the current config will be overwritten by the config specified in the request. As the principal of an application cannot be updated, during an update the principal should either be the same or absent.

Deploy an Artifact and Application

To deploy an application from your local file system into the namespace namespace-id, submit an HTTP POST request:

with the name of the JAR file as a header:

and Kerberos principal with which the application is to be deployed (if required):

and enable or disable updating schedules of the existing workflows using the header:

This will add the JAR file as an artifact and then create an application from that artifact. The archive name must be in the form <artifact-name>-<artifact-version>.jar. An optional header can supply a configuration object as a serialized JSON string:

The application's content is the body of the request:

Invoke the same command to update an application to a newer version. However, be sure to stop all of its Spark and MapReduce programs before updating the application.

For an application that has a configuration class such as:

We can deploy it with this call:

Note: The X-App-Config header contains the JSON serialization string of the MyAppConfig object.

List Applications

To list all of the applications in the namespace namespace-id, issue an HTTP GET request:

Parameter

Version Introduced

Description

Parameter

Version Introduced

Description

namespace-id

 

Namespace ID.

artifactName

 

Optional filter to list all applications that use the specified artifact name. Valid values are cdap-data-pipeline, cdap-data-streams, and delta-app.

artifactVersion

 

Optional filter. This is the version of the artifact given in the artifactName parameter. It will be different depending on the artifact given. To get the version list, use Artifact Microservices.

pageSize

6.6.0

Optional filter. The number of pipelines to be returned in the response. If pageSize is not specified, all pipelines will be returned. Basically, pageSize enables pagination of the results. Using this parameter changes the result format. For details, see the examples that follow.

pageToken

6.6.0

Optional filter. If pageSize is specified, use nextPageToken as the value from the response. If pageToken is not specified, returns the first page.

orderBy

6.6.0

Optional filter. Specifies the sorting order. The sorting is by Application Name and then Application Version.

Values can be ASCor DESC. If unspecified, the value will be ASC. ASC stands for ascending order whereas DESC stands for descending order.

nameFilter

6.8.0

Optional filter. Filters the application name based on nameFilterType. If nameFilterType is not set, it will use default CONTAINS type.

nameFilterType

6.8.0

Optional filter. Values can be EQUALS, CONTAINS or EQUALS_IGNORE_CASE. Specifies the way to filter application names based on nameFilter.

EQUALS: Application name must be equal to the nameFilter, case sensitive.

CONTAINS: Application name must contain the nameFilter, case insensitive.

EQUALS_IGNORE_CASE: Application name must be equal to the nameFilter, case insensitive.

sortCreationTime

6.8.0

Optional Boolean. Values can be true or false. If set to true, the sorting order will be based on the creation time of the applications, instead of by Application Name and then Application Version.

Default is false.

latestOnly

6.8.0

Optional filter. Values can be trueor false. If set to false, it returns all versions of the apps. If not specified, it will use a default true value, that only returns the latest version of the applications.

Note: When upgrading the instance from versions < 6.8 to versions >= 6.8, it’s important to follow the “upgrade applications” process in order to have the UI properly render all existing applications.

If pageSize is not specified, this returns an array of JSON Objects that lists each application with its name, description, and artifact. The list can optionally be filtered by one or more artifact names. It can also be filtered by artifact version. For example:

will return all applications that use either the cdap-data-pipeline,cdap-data-streams,or delta-app artifacts.

The following is an example response when pageSize is not specified:

If pageSize is specified, the result is a JSON object that returns the applications as a JSON array under the applications key. The page token identifier for the next page of results is specified under nextPageToken key. The absence of nextPageToken in the response denotes that it was the last page in the results.

Details of an Application

For detailed information on an application in a namespace namespace-id, use:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

app-id

Name of the application.

Note: To get the creation time of an application and other types of metadata, see Metadata Microservices.

The information will be returned in the body of the response. It includes the name and description of the application; the artifact and datasets that it uses, all of its programs; and the Kerberos principal, if that was provided during the deployment. For example:

HTTP Responses

Status Codes

Description

Status Codes

<