Metadata Microservices

Use the CDAP Metadata Microservices to set, retrieve, and delete the metadata annotations of applications, datasets, and other entities in CDAP.

Metadata consists of properties (a list of key-value pairs) or tags (a list of keys). Metadata and their use are described in the Metadata and Lineage section.

The Microservices is divided into these sections:

• Metadata properties

• Metadata tags

• Searching metadata

• Viewing lineage

• Field level lineage

• Metadata for a run of a program

Metadata keys, values, and tags must conform to the CDAP alphanumeric extra extended character set, and are limited to 50 characters in length. The entire metadata object associated with a single entity is limited to 10K bytes in size.

There is one reserved word for property keys and values: tags, either as tags or TAGS. Tags themselves have no reserved words.

All methods or endpoints described in this API have a base URL (typically http://<host>:11015 or https://<host>:10443) that precedes the resource identifier, as described in the Microservices Conventions. These methods return a status code, as listed in the Microservices Status Codes.

Metadata Properties

Annotating Properties

To annotate user metadata properties for an application, dataset, or other entities including custom entities, submit an HTTP POST request:

or, for a particular program of a specific application:

or, for a particular version of an artifact:

or, for a custom entity like field of a dataset:

with the metadata properties as a JSON string map of string-string pairs, passed in the request body:

New property keys will be added and existing keys will be updated. Existing keys not in the properties map will not be deleted.

HTTP Responses

Retrieving Properties

To retrieve user metadata properties for an application, dataset, or other entities including custom entities, submit an HTTP GET request:

or, for a specific application:

or, for a particular program of a specific application:

or, for a particular version of an artifact:

or, for a custom entity like field of a dataset:

with the metadata properties returned as a JSON string map of string-string pairs, passed in the response body (pretty-printed):

Example

To get the creation time for a deployed data pipeline called POS_SALES, issue the following GET request:

The result is:

HTTP Responses

Deleting Properties

To delete all user metadata properties for an application, dataset, or other entities including custom entities, submit an HTTP DELETE request:

or, for all user metadata properties of a particular program of a specific application:

or, for a particular version of an artifact:

To delete a specific property for an application, dataset, or submit an HTTP DELETE request with the property key:

or, for a particular property of a program of a specific application:

or, for a particular version of an artifact:

or, for a custom entity like field of a dataset:

HTTP Responses

Metadata Tags

Adding Tags

To add user metadata tags for an application, dataset, or other entities including custom entities, submit an HTTP POST request:

or, for a particular program of a specific application:

or, for a particular version of an artifact:

or, for a custom entity like field of a dataset:

with the metadata tags, as a list of strings, passed in the JSON request body:

HTTP Responses

Retrieving Tags

To retrieve user metadata tags for an application, dataset, or other entities including custom entities, submit an HTTP GET request:

or, for a particular program of a specific application:

or, for a particular version of an artifact:

or, for a custom entity like field of a dataset:

with the metadata tags returned as a JSON string in the return body: