Note: Datasets are deprecated and will be removed in CDAP 7.0.0.
The CDAP Dataset Microservices allows you to interact with datasets through HTTP. You can list, create, delete, and truncate datasets.
All methods or endpoints described in this API have a base URL (typically http://<host>:11015 or https://<host>:10443) that precedes the resource identifier, as described in the Microservices Conventions. These methods return a status code, as listed in the Microservices Status Codes.
Listing all Datasets
You can list all datasets in CDAP by issuing an HTTP GET request to the URL:
...
| Code Block |
|---|
{
"name": "cdap.user.purchases",
"type": "io.cdap.cdap.api.dataset.lib.ObjectStore",
"description": "Purchases Dataset",
"properties": {
"schema":"...",
"type":"..."
},
"datasetSpecs": {
...
}
}
|
Creating a Dataset
You can create a dataset by issuing an HTTP PUT request to the URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the new dataset |
| Type of the new dataset |
| Dataset properties, map of String to String |
| Dataset description |
| Kerberos principal with which the dataset should be created; once a dataset has been created, this cannot be changed (optional) |
HTTP Responses
Status Codes | Description |
|---|---|
| Requested dataset was successfully created |
| The dataset already exist with a different Kerberos principal |
| Requested dataset type was not found |
| Dataset with the same name already exists |
...
HTTP Request |
|
|---|---|
Body |
|
Description | Creates a dataset named mydataset of the type |
Properties of an Existing Dataset
You can retrieve the properties with which a dataset was created or last updated by issuing an HTTP GET request to the URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the existing dataset |
HTTP Responses
Status Codes | Description |
|---|---|
| Requested dataset was successfully updated |
| Requested dataset instance was not found |
...
Note that this returns the original properties that were submitted when the dataset was created or updated. You can use these properties to create a clone of the dataset, or as a basis for updating some properties of this dataset without modifying the remaining properties.
Metadata of an Existing Dataset
You can retrieve the metadata with which a dataset was created by issuing an HTTP GET request to the URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the existing dataset |
HTTP Responses
Status Codes | Description |
|---|---|
| Metadata for the requested dataset instance was successfully returned |
| Requested dataset instance was not found |
...
| Code Block |
|---|
{
"spec": {
"name": "ownedDataset",
"type": "datasetType1",
"originalProperties": {},
"properties": {},
"datasetSpecs": {}
},
"type": {
"name": "datasetType1",
"modules": [
{
"name": "module1",
"className": "io.cdap.cdap.data2.datafabric.dataset.service.TestModule1",
"jarLocationPath": "/path/data/module1/archive/module1.jar",
"types": [
"datasetType1"
],
"usesModules": [],
"usedByModules": []
}
]
},
"principal": "user/example.net@EXAMPLEKDC.NET"
}
|
Updating an Existing Dataset
You can update an existing dataset's table and properties by issuing an HTTP PUT request to the URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Name of the existing dataset |
HTTP Responses
Status Codes | Description |
|---|---|
| Requested dataset was successfully updated |
| Requested dataset instance was not found |
...
HTTP Request |
|
|---|---|
Body |
|
Description | For the mydataset of type |
Deleting a Dataset
You can delete a dataset by issuing an HTTP DELETE request to the URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Dataset name |
HTTP Responses
Status Codes | Description |
|---|---|
| Dataset was successfully deleted |
| Dataset named dataset-name could not be found |
...
HTTP Request |
|
|---|---|
Description | Deletes the dataset mydataset in the namespace default |
Deleting all Datasets
If the property enable.unrecoverable.reset in cdap-site.xml is set to true, you can delete all Datasets (in a namespace) by issuing an HTTP DELETE request to the URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
HTTP Responses
Status Codes | Description |
|---|---|
| All Datasets were successfully deleted |
| Property to enable unrecoverable methods is not enabled |
| Programs are currently running in the namespace |
...
This method must be exercised with extreme caution, as there is no recovery from it.
Truncating a Dataset
You can truncate a dataset by issuing an HTTP POST request to the URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Dataset name |
HTTP Responses
Status Codes | Description |
|---|---|
| Dataset was successfully truncated |
Datasets used by an Application
You can retrieve a list of datasets used by an application by issuing a HTTP GET request to the URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Application ID |
HTTP Responses
Status Codes | Description |
|---|---|
| Request was successful |
Datasets used by a Program
You can retrieve a list of datasets used by a program by issuing a HTTP GET request to the URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Application ID |
| Program type, one of |
| Program ID |
HTTP Responses
Status Codes | Description |
|---|---|
| Request was successful |
Programs using a Dataset
You can retrieve a list of programs that are using a dataset by issuing a HTTP GET request to the URL:
...
Parameter | Description |
|---|---|
| Namespace ID |
| Dataset ID |
HTTP Responses
Status Codes | Description |
|---|---|
| Request was successful |
...