Page Comparison

...

Code Block
PUT /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/drafts/<draft>

Parameter	Description
`context`	The namespace for the Draft pipelines.
`draft`	The Draft ID of the pipeline. The Draft ID can be any alphanumeric string. It must be unique in the namespace.

The request body is a JSON object specifying the name, description, and artifact for the pipeline. There are two supported artifacts for connections: cdap-data-pipeline and cdap-data-streams.

...

Code Block
DELETE /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/drafts/<draft>

Parameter	Description
`context`	The namespace for the Draft pipeline.
`draft`	The Draft ID of the pipeline.

Details of a Draft Pipeline

...

Code Block
GET /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/drafts/<draft>

Parameter	Description
`context`	The namespace for the Draft pipelines.
`draft`	The Draft ID of the pipeline.

The information will be returned in the body of the response. It includes the Draft ID, name, description, and revision number of each pipeline in Draft mode; the artifact that the pipelines uses; and details of the pipeline configuration (pipeline json). For example:

...

Code Block
GET /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/drafts

Path Parameter	Description
`context`	The namespace for the Draft pipelines.

Query Parameter	Description
`sortBy`	Name of the fields to sort by. These field names must match the Draft pipeline json.
`filter`	Takes in any text and does prefix matching on the draft name field.
`includeConfig`	Includes the pipeline json for each draft in the response. Default value is false.

The information will be returned in the body of the response. It includes the Draft ID, create time, update time, name, description, and revision number of each pipeline in Draft mode; and the artifact that each pipelines uses. For example:

...

Code Block
PUT /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/connections/<connection-id>

Parameter	Description
`context`	The namespace for the connection.
`draft`	The Connection ID of the connection. The Connection ID can be any alphanumeric string. It must be unique in the namespace.

The request body is a JSON object specifying the name, description, and artifact for the connection. These are the supported artifacts for connections: google-cloud, database-plugin, mysql-plugin, oracle-plugin, postgresql-plugin, sqlserver-plugin.

...

Code Block
GET /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/connections

Path Parameter	Description
`context`	The namespace for the connections.

The information will be returned in the body of the response. It includes the Connection ID, create time, update time, name, description, and plugin category, plugin type, and plugin properties for each in connection; and the artifact that each connection uses. For example:

...

Code Block
GET /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/connections/<connection-id>

Parameter	Description
`context`	The namespace for the connection.
`connection-id`	The Connection ID of the connection.

The information will be returned in the body of the response. It includes the Connection ID, create time, update time, name, description, and plugin category, plugin type, and plugin properties for each in connection; and the artifact that each connection uses. For example:

...

Code Block
DELETE /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/connections/<connection-id>

Parameter	Description
`context`	The namespace for the Connection.
`connection-id`	The Connection ID of the pipeline.

Test a Connection

To test a connection in a namespace, submit an HTTP POST request:

Code Block
POST /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/connections/test

Parameter	Description
`context`	The namespace for the Connection.

The request body is a JSON object specifying the name, description, and artifact for the connection. These are the supported artifacts for connections: google-cloud, database-plugin, mysql-plugin, oracle-plugin, postgresql-plugin, sqlserver-plugin.

...

Code Block
POST /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/connections/<connection-id>/browse

Parameter	Description
`context`	The namespace for the Connection.
`connection-id`	The Connection ID of the pipeline.

The request body is a JSON object specifying the name, description, and artifact for the connection. These are the supported artifacts for connections: google-cloud, database-plugin, mysql-plugin, oracle-plugin, postgresql-plugin, sqlserver-plugin.

...

Code Block
POST /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/connections/<connection-id>/sample

Parameter	Description
`context`	The namespace for the Connection.
`connection-id`	The Connection ID of the pipeline.

The request body is a JSON object specifying the name, description, and artifact for the connection. These are the supported artifacts for connections: google-cloud, database-plugin, mysql-plugin, oracle-plugin, postgresql-plugin, sqlserver-plugin.

...

Code Block
POST /v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/<context>/connections/<connection-id>/specification

Parameter	Description
`context`	The namespace for the Connection.
`connection-id`	The Connection ID of the pipeline.

The request body is a JSON object specifying the name, description, and artifact for the connection. These are the supported artifacts for connections: google-cloud, database-plugin, mysql-plugin, oracle-plugin, postgresql-plugin, sqlserver-plugin.

...

For information about setting the path for each type of connection, see the Connection Reference.

Call GetSchema from REST API

This topic describes the process for getting the output schema for plugins using the REST API.

The GetSchema button sends a POST request to this endpoint:/v3/namespaces/system/apps/pipeline/services/studio/methods/v1/contexts/{namespace}/validations/stage

This endpoint expects a payload with the stage to validate and the inputSchemas to this stage. The stage object can be directly copied from the pipeline json and the inputSchemas can be left empty since it is a source. Example payload:

Code Block

{
    "stage":{
        "name": "BigQuery",
        "plugin": {
            "name": "BigQueryTable",
            "type": "batchsource",
            "label": "BigQuery",
            "artifact": {
                "name": "google-cloud",
                "version": "0.14.6",
                "scope": "SYSTEM"
            },
            "properties": {
                "project": "auto-detect",
                "serviceFilePath": "auto-detect",
                "datasetProject": "meseifan-test",
                "dataset": "GCPQuickStart",
                "table": "ADT_Out",
                "referenceName": "test"
            }
        },
        "outputSchema": [
            {
                "name": "etlSchemaBody",
                "schema": ""
            }
        ]
    },
    "inputSchemas":[]
}

The response will be a fully populated stage object, one of the fields is "outputSchema" which will contain the schema of the input you are attempting to use. Example response:

Code Block

{
  "spec": {
    "name": "BigQuery",
    "plugin": {
      "type": "batchsource",
      "name": "BigQueryTable",
      "properties": {
        "serviceFilePath": "auto-detect",
        "project": "auto-detect",
        "datasetProject": "meseifan-test",
        "dataset": "GCPQuickStart",
        "table": "ADT_Out",
        "referenceName": "test"
      },
      "artifact": {
        "name": "google-cloud",
        "version": {
          "version": "0.14.6",
          "major": 0,
          "minor": 14,
          "fix": 6
        },
        "scope": "SYSTEM"
      }
    },
    "outputSchema": {
      "type": "record",
      "name": "output",
      "fields": [
        {
          "name": "SRC_NTW_ID_",
          "type": [ "string", "null" ]
        },
        {
          "name": "NTW_CUST_NUM_",
          "type": [ "long", "null" ]
        },
        {
          "name": "NTW_CS_NO",
          "type": [ "string", "null" ]
        },
        {
          "name": "SRC_NTW_ID_masked",
          "type": [ "string", "null" ]
        },
        {
          "name": "NTW_CS_NO_masked",
          "type": [ "string", "null" ]
        }
      ]
    },
    "inputSchemas": {},
    "outputPorts": {},
    "portSchemas": {},
    "stageLoggingEnabled": true,
    "processTimingEnabled": true,
    "maxPreviewRecords": 100,
    "inputStages": [],
    "failures": [],
  }
}

Versions Compared

Old Version 39

New Version Current

Key

Details of a Draft Pipeline

Test a Connection

Call GetSchema from REST API