Use the CDAP Replication Microservices to manage replication jobs.
All methods or endpoints described in this API have a base URL (typically http://<host>:11015
or https://<host>:10443
) that precedes the resource identifier, as described in the Microservices Conventions. These methods return a status code, as listed in the Microservices Status Codes.
To assess a replication job in draft mode based on the source and target tables, submit an HTTP POST request:
POST /v3/namespaces/system/apps/delta/services/assessor/methods/v1/contexts/<namespace>/drafts/<draft-id>/assessPipeline |
Parameter | Description |
---|---|
| The namespace for the draft replication jobs. |
| The Draft ID of the replication job. The Draft ID can be any alphanumeric string. It must be unique in the namespace. |
To assess a specific table in a replication job in draft mode, submit an HTTP POST request:
POST /v3/namespaces/system/apps/delta/services/assessor/methods/v1/contexts/<namespace>/drafts/<draft-id>/assessPipeline |
Parameter | Description |
---|---|
| The namespace for the draft replication jobs. |
| The Draft ID of the replication job. The Draft ID can be any alphanumeric string. It must be unique in the namespace. |
The request body is a JSON object specifying the database and table name in the draft replication job.
For example:
{ "database": "db1", "table": "table1" } |
To add a new replication job in draft mode or overwrite an existing one, submit an HTTP PUT request:
PUT /v3/namespaces/system/apps/delta/services/assessor/methods/v1/contexts/<namespace>/drafts/<draft-id> |
Parameter | Description |
---|---|
| The namespace for the draft replication jobs. |
| The Draft ID of the replication job. The Draft ID can be any alphanumeric string. It must be unique in the namespace. |
The request body is a JSON object specifying the draft replication job configuration.
For example:
{ "name": "<app_name>", "artifact": { "name": "delta-app", "version": "0.6.1", "scope": "SYSTEM" }, "config": { "tables": [ { "database": "testreplication", "table": "persons", "schema": "dbo" } ], "stages": [ { "name": "Microsoft SQLServer", "plugin": { "name": "sqlserver", "type": "cdcSource", "properties": { "port": "1433", "serverTimezone": "UTC", "replicateExistingData": "true", "user": "<>", "password": "<>", "host": "1.2.3.4", "jdbcPluginName": "sqlserver42", "database": "testreplication" }, "artifact": { "name": "sqlserver-delta-plugins", "version": "0.6.1", "scope": "SYSTEM" } } }, { "name": "BigQuery", "plugin": { "name": "bigquery", "type": "cdcTarget", "properties": { "project": "auto-detect", "serviceAccountKey": "auto-detect", "stagingBucketLocation": "us", "loadInterval": "90", "stagingTablePrefix": "staging", "requireManualDrops": "false", "datasetName": "testreplication1" }, "artifact": { "name": "bigquery-delta-plugins", "version": "0.6.1", "scope": "SYSTEM" } } } ], "tableTransformations": [], "parallelism": { "numInstances": 1 }, "description": "<>" } } |
To delete an existing replication job in draft mode, submit an HTTP DELETE request:
DELETE /v3/namespaces/system/apps/delta/services/assessor/methods/v1/contexts/<namespace>/drafts/<draft-id> |
Parameter | Description |
---|---|
| The namespace for the draft replication jobs. |
| The Draft ID of the replication job. The Draft ID can be any alphanumeric string. It must be unique in the namespace. |
To list the details of a source table in replication job draft, submit an HTTP POST request:
POST /v3/namespaces/system/apps/delta/services/assessor/methods/v1/contexts/<namespace>/drafts/<draft-id>/describeTable |
Parameter | Description |
---|---|
| The namespace for the draft replication jobs. |
| The Draft ID of the replication job. The Draft ID can be any alphanumeric string. It must be unique in the namespace. |
The request body is a JSON object specifying the database and table name in the draft replication job.
For example:
{ "database": "db1", "table": "table1" } |
To list the details of a replication job draft, submit an HTTP GET request:
GET /v3/namespaces/system/apps/delta/services/assessor/methods/v1/contexts/<namespace>/drafts/<draft-id> |
Parameter | Description |
---|---|
| The namespace for the draft replication jobs. |
| The Draft ID of the replication job. The Draft ID can be any alphanumeric string. It must be unique in the namespace. |
To list all replication jobs in Draft mode in a namespace, submit an HTTP GET request:
GET /v3/namespaces/system/apps/delta/services/assessor/methods/v1/contexts/<namespace>/drafts |
Parameter | Description |
---|---|
| The namespace for the draft replication jobs. |
The information will be returned in the body of the response. It list the draft objects, where a draft contains the full job configuration. For example:
[ { "name": "Draft1", "created": timestamp "config": { "stages": [ { "name": "source", "plugin": { "name": "oracle", "type": "cdcSource", "properties": { "host": "[hostname]", "port": "1433", "user": "repluser", "password": "${secure(kms-key)}" } } }, ... ] }, } ] |
To get the generation number for a replication job, submit an HTTP GET request:
GET /v3/namespaces/system/apps/delta/services/assessor/methods/v1/contexts/{namespace}/apps/{app_name}/maxgeneration |
Parameter | Description |
---|---|
| The namespace for your application. |
| The name of the replication job. |
The offset is maintained at for every task instance. When you configure a replication job, there is an option to set the number of tasks and each task has its own offset information.
To get the offset information for each task instance for a replication job, submit an HTTP GET request:
GET /v3/namespaces/system/apps/delta/services/assessor/methods/v1/contexts/{namespace}/apps/{app_name}/generations/{generation_num}/instances/{instance_id}/offset |
Parameter | Description |
---|---|
| The namespace for the deployed replication job. |
| The name of the deployed replication job. |
| The generation number for the deployed replication job. It’s generally a LONG type value. |
| The ID of the task. |
Each Task/instance handles a particular set of tables. It is responsible to snapshot and replication process for those tables.
To list the replication tables mapped to an Instance_id
for a deployed replication job, submit an HTTP GET request:
GET v3/namespaces/system/apps/{app_name}/workers/DeltaWorker |
Parameter | Description |
---|---|
| The name of the deployed replication job. |
In the Response body, look for properties
. table.assignments
.
To list the source tables in a replication job in draft mode, submit an HTTP POST request:
POST /v3/namespaces/system/apps/delta/services/assessor/methods/v1/contexts/<namespace>/drafts/<draft-id>/listTables |
Parameter | Description |
---|---|
| The namespace for the draft replication jobs. |
| The Draft ID of the replication job. The Draft ID can be any alphanumeric string. It must be unique in the namespace. |