Using a connection in a plugin

A connection stores information, such as user credentials and host information, needed to connect to data sources. When you use a connection in a plugin, the plugin inherits all of the properties configured in the connection. When you export a pipeline that has a plugin that uses a connection, the pipeline configuration JSON includes all of these properties.

Starting in CDAP 6.5.0, you can create a connection and use it in a supported batch source or sink plugin. For a list of the support connections, see the Connection Reference.

You can use connections to browse supported data sources. For example, if the CDAP Administrator creates a connection for Amazon S3, you can browse buckets to find the file you want to add as a source in a data pipeline.

You can create a connection from the Pipeline Studio, Wrangler, and Namespace Admin page. For more information, see Managing Connections.

To use a connection in one of the supported plugins, follow these steps:

1. In the Pipeline Studio, open a pipeline.

2. Locate the batch source plugin for which you want to use a connection.

3. Click Properties.

4. Set Use Connection to Yes.

5. Click Browse Connections. You can select an existing connection, create a new connection, or import the JSON file of a connection.
   Note: The Connection field supports macros so you can switch between Dev, Test, Prod environments. You can also use a macro if you have multiple connections to the same source, such as multiple GCS connections for different folders in the same Google Cloud Platform instance.
   After you select the connection, all properties associated with the connection no longer appear in the plugin properties. The properties are stored in the connection. However, the pipeline configuration JSON includes all of the connection information.

6. Finish configuring the plugin properties and validate the plugin.

If you encounter any issues using connections, see the Troubleshooting Guide.