Managing Connections

A connection stores information, such as user credentials and host information, needed to connect to data sources and sinks. You can create and manage connections that ETL developers, data analysts, and data engineers can reuse in Wrangler and data pipelines.

From the Namespace Admin page, you can add, edit, duplicate, import, export, and delete connections for each namespace.

Note: In versions before CDAP 6.5.0, you create connections when you create a pipeline or before wrangling data. Data scientists and ETL developers provide credentials (such as the connection IP, port, and password) each time they create a pipeline. In versions 6.5.0 and later, Admins can instead create and manage the connections for data pipelines, and developers can use these connections without needing to provide credentials.

In Wrangler, you can add, edit, export, duplicate, and delete connections for each namespace. In Pipeline Studio, you can add and import connections for supported plugins for each namespace.

Note: Connections are required to browse data and tables for plugins that support connections.

For more information about supported connections, see the Connection Reference.

Adding a Connection

Note: Before you add a connection for a database source, you must upload the JDBC driver to CDAP. You can upload the driver from the Hub or the Namespace Admin page.

To add a connection, follow these steps:

From the Namespace Admin page, click Connections.
Click Add Connection.
From the Add a connection page, click the type of connection you want to create.
Configure the connection properties.
Note: Connection names must be unique in a namespace and can only include letters, numbers, underscores, and hyphens.
As a best practice, click Test Connection to ensure the connection works.
Click Create.
The connection appears under the connection type and is available for all users in the namespace. The connection is available for use in the Wrangler and in the corresponding source plugin in the Pipeline Studio.

Editing a Connection

To edit connection properties, follow these steps:

From the Namespace Admin page, click Connections.
Locate the connection you want to edit.
Click the three dots and click Edit.
You can edit the connection properties and test the connection.
Click Save.

Duplicating a Connection

To duplicate a connection, follow these steps:

From the Namespace Admin page, click Connections.
Locate the connection you want to duplicate.
Click the three dots and click Duplicate.
CDAP creates a new connection with the properties from the original connection.
Enter a Name for the connection.
Note: Connection names must be unique in the namespace.
(Optional) Edit the connection properties.
Click Test Connection and resolve any errors.
Click Create.
The connection is available for use in the Wrangler and in the corresponding source plugin in the Pipeline Studio.

Importing a Connection

To import a connection, follow these steps:

From the Namespace Admin page, click Connections.
Click Import and navigate to the JSON file you want to import into CDAP.
Edit the name of the connection and any other properties.
Note: Connection names must be unique in the namespace.

Exporting a Connection

You might want to export connections to share them with other team members, add them to version control, or deploy them from a development environment to a test or production environment.

To export a connection, follow these steps:

From the Namespace Admin page, click Connections.
Locate the connection you want to export.
From the Namespace Admin page, click the three dots and click Export.
CDAP exports the connection properties to a JSON file.

Deleting a Connection

To delete a connection, follow these steps:

From the Namespace Admin page, click Connections.
Locate the connection you want to delete.
From the Namespace Admin page, click the three dots and click Delete.
CDAP deletes the connection and it is no longer available for use.