Artifact Microservices

Use the CDAP Artifact Microservices to deploy artifacts, list available artifacts, and retrieve information about plugins available to artifacts. Artifacts, their use, and examples of using them, are described in Artifacts.

All methods or endpoints described in this API have a base URL (typically http://<host>:11015 or https://<host>:10443) that precedes the resource identifier, as described in the Microservices Conventions. These methods return a status code, as listed in the Microservices Status Codes.

Add an Artifact

An artifact can be added (loaded) with an HTTP POST method to the URL:

POST /v3/namespaces/<namespace-id>/artifacts/<artifact-name>

The request body must contain the binary contents of the artifact.

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact to be created.

Several optional headers may also be specified:

Header

Description

Example

Header

Description

Example

Artifact-Version

The version of the artifact to add. If not specified, the Bundle-Version attribute in the JAR file's Manifest will be used.

1.0.0

Artifact-Extends

If the artifact contains plugins, describes which parent artifacts should have access to those plugins. Multiple parents can be given by separating them with a /

cdap-data-pipeline[3.2.0,5.0.0)/cdap-data-streams[3.2.0,5.0.0)

Artifact-Plugins

JSON Array of plugins contained in the artifact that are not annotated as a plugin. This should be used for third-party JARs that need to be plugins, such as JDBC drivers. Each element in the array is a JSON object containing name, type, and className of the plugin.

[ { "name": "mysql", "type": "jdbc", "className": "com.mysql.jdbc.Driver" } ]

For more information about custom plugins, see Plugins.

List Available Artifacts

To retrieve a list of available artifacts, submit an HTTP GET request:

GET /v3/namespaces/<namespace-id>/artifacts[?scope=<scope>]

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

scope

Optional scope filter. If not specified, artifacts in the USER and SYSTEM scopes are returned. Otherwise, only artifacts in the specified scope are returned.

This will return a JSON array that lists each artifact with its name, version, and scope. Example output (pretty-printed):

$ GET /v3/namespaces/default/artifacts [ { "name": "cdap-data-pipeline", "version": "6.3.0", "scope": "SYSTEM" }, { "name": "cdap-data-streams", "version": "6.3.0", "scope": "SYSTEM" }, { "name": "Purchase", "version": "6.3.0", "scope": "USER" } ]

List Artifact Versions

To list all versions or a specific range of versions of an artifact, submit an HTTP GET request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact.

scope

Optional scope filter. If not specified, defaults to USER.

artifact-version

Optional version range filter. All versions within this range will be returned. It can be an exact version such as 1.0.0 or a version range such as [1.0.0, 2.0.0). If not specified, defaults to returning all versions.

limit

Optional limit filter. Limits the number of the returned results. If not specified, defaults to returning all artifacts which match the requirements.

order

Optional order filter. Accepted values are one of DESCASC, or UNORDEREDDESC will sort the artifacts by version from highest to lowest; for ASC, sorted by version from lowest to highest. If not specified, defaults to UNORDERED.

This will return a JSON array that lists each version of the specified artifact with its name, version, and scope. Example output for the cdap-data-pipeline artifact (pretty-printed):

Retrieve Artifact Details

To retrieve details about a specific version of an artifact, submit an HTTP GET request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact.

artifact-version

Version of the artifact.

scope

Optional scope filter. If not specified, defaults to USER.

This will return a JSON object that contains information about: classes in the artifact; the schema of the config object supported by the Application class; and the artifact name, version, and scope. Example output for version 6.3.0 of the WordCount artifact (pretty-printed and reformatted to fit):

Set Artifact Properties

To set properties for a specific version of an artifact, submit an HTTP PUT request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact.

artifact-version

Version of the artifact.

The request body must be a JSON object that contains the properties for the artifact. The keys and values in the object must be strings. If any properties are already defined, they will be overwritten.

Set an Artifact Property

To set a specific property for a specific version of an artifact, submit an HTTP PUT request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact.

artifact-version

Version of the artifact.

property

Property to set.

The request body must contain the value to set for the property. If the property already exists, the previous value will be overwritten.

Retrieve Artifact Properties

To retrieve properties for a specific version of an artifact, submit an HTTP GET request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact.

artifact-version

Version of the artifact.

scope

Optional scope filter. If not specified, defaults to USER.

keys

Optional comma-separated list of property keys to return. If not specified, all keys are returned.

This will return a JSON object that contains the properties of the artifact.

Retrieve an Artifact Property

To retrieve a specific property for a specific version of an artifact, submit an HTTP GET request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact.

artifact-version

Version of the artifact.

property

Property to retrieve.

scope

Optional scope filter. If not specified, defaults to USER.

Delete Artifact Properties

To delete all properties for a specific version of an artifact, submit an HTTP DELETE request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact.

artifact-version

Version of the artifact.

Delete an Artifact Property

To delete a specific property for a specific version of an artifact, submit an HTTP DELETE request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact.

artifact-version

Version of the artifact.

property

Property key to delete.

List Extensions (Plugin Types) available to an Artifact

To list the extensions (plugin types) available to an artifact, submit an HTTP GET request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact.

artifact-version

Version of the artifact.

scope

Optional scope filter. If not specified, defaults to USER.

This will return a JSON array that lists the extensions (plugin types) available to the artifact. Example output for version 6.2.0 of the cdap-data-pipeline artifact:

List Plugins available to an Artifact

To list plugins of a specific type available to an artifact, submit an HTTP GET request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact.

artifact-version

Version of the artifact.

plugin-type

Type of plugins to list.

scope

Optional scope filter. If not specified, defaults to USER.

This will return a JSON array that lists the plugins of the specified type available to the artifact. Each element in the array is a JSON object containing the artifact that the plugin originated from, and the plugin's class name, description, name, and type. Note that the details provided are a summary compared to those provided by the endpoint Retrieve Plugin Details.

Example output for plugins of type transform available to version 6.3.0 of the cdap-data-pipeline artifact (pretty-printed and reformatted to fit):

Retrieve Plugin Details

To retrieve details about a specific plugin available to an artifact, submit an HTTP GET request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the parent artifact.

artifact-version

Version of the parent artifact.

plugin-type

Type of the plugin.

plugin-name

Name of the plugin.

scope

Optional scope filter of the parent artifact. If not specified, defaults to USER.

artifact-name

Optional plugin artifact name filter. If not specified, defaults to returning all plugins available to the parent artifact.

artifact-version

Optional plugin version filter. All plugin artifact versions within this range will be returned. It can be an exact version such as 1.0.0 or a version range such as [1.0.0, 2.0.0). If not specified, defaults to returning all versions of the plugins available to the parent artifact.

artifact-scope

Optional plugin scope filter. Scope of the plugin artifact scope. If not specified, defaults to returning plugins in both the USER and SYSTEM scopes.

limit

Optional limit filter. Limits the number of the returned results. If not specified, defaults to returning all plugins which match the requirements.

order

Optional order filter. Accepted values are one of DESCASC, or UNORDEREDDESC will sort the plugin artifacts first by scope (first by USER and then by SYSTEM), then by name descending alphabetically, and then by version from highest to lowest; for ASC, sorted first by scope, then by name ascending alphabetically, and then by version from lowest to highest. If not specified, defaults to UNORDERED.

This will return a JSON array that lists the plugins of the specified type and name available to the artifact. As can been seen compared with the endpoint List Plugins available to an Artifact, this provides all details on the specified plugin. Each element in the array is a JSON object containing the artifact that the plugin originated from, and the plugin's class name, description, name, type, and properties.

Example output for the JavaScript plugin available to version 6.3.0 of the cdap-data-pipeline artifact (pretty-printed and reformatted to fit):

Delete an Artifact

To delete an artifact, submit an HTTP DELETE request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID.

artifact-name

Name of the artifact.

artifact-version

Version of the artifact.

Deleting an artifact is an advanced feature. If there are programs that use the artifact, those programs will not be able to start unless the artifact is added again, or the program application is updated to use a different artifact.

Load System Artifacts

To load all system artifacts on the CDAP Master node(s), submit an HTTP POST request:

This call will make the CDAP master scan the artifacts directly and add any new versions of artifacts that it finds. Any snapshot artifacts will be re-loaded.

Delete a System Artifact

To delete a system artifact, submit an HTTP DELETE request:

Parameter

Description

Parameter

Description

artifact-name

Name of the artifact.

artifact-version

Version of the artifact.

Deleting an artifact is an advanced feature. If there are programs that use the artifact, those programs will not be able to start unless the artifact is added again, or the program application is updated to use a different artifact.

List Application Classes

To list application classes, submit an HTTP GET request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID

scope

Optional scope filter. If not specified, classes from artifacts in the USER and SYSTEM scopes are returned. Otherwise, only classes from artifacts in the specified scope are returned.

This will return a JSON array that lists all application classes contained in artifacts. Each element in the array is a JSON object that describes the artifact the class originates in as well as the class name. Example output (pretty-printed and reformatted to fit):

Retrieve Application Class Details

To retrieve details about a specific application class, submit an HTTP GET request:

Parameter

Description

Parameter

Description

namespace-id

Namespace ID

class-name

Application class name

scope

Optional scope filter. If not specified, defaults to USER.

This will return a JSON array that lists each application class with that class name. Each element in the array is a JSON object that contains details about the application class, including the artifact the class is from, the class name, and the schema of the config supported by the application class. Example output for the WordCount application (pretty-printed and reformatted to fit):

 

Created in 2020 by Google Inc.