SQL Server Query Post-run Action

Plugin version: 0.10.0

Runs a SQL Server query at the end of the pipeline run. Can be configured to run only on success, only on failure, or always at the end of the run.

Use this plugin when you need to run a query at the end of a pipeline run. For example, you might have a pipeline that imports data from a database table to HDFS files. At the end of the run, you might want to run a query that deletes the data that was read from the table.

Configuration

Property

Macro Enabled?

Description

Property

Macro Enabled?

Description

Run Condition

Yes

Optional. When to run the action. Must be 'completion', 'success', or 'failure'. Defaults to 'success'. If set to 'completion', the action will be executed regardless of whether the pipeline run succeeded or failed. If set to 'success', the action will only be executed if the pipeline run succeeded. If set to 'failure', the action will only be executed if the pipeline run failed.

Default is success.

Driver Name

No

Required.  Name of the JDBC driver to use.

Default is sqlserver42.

Host

Yes

Required. Host that SQL Server is running on.

Default is localhost.

Port

Yes

Required. Port that SQL Server is listening to. If the port number is specified in the connection string, no request to SQLbrowser is made. When the Port and Instance Name are both specified, the connection is made to the specified port. However, the Instance Name is validated and an error is thrown if it does not match the port.

Default is 1433.

Database

Yes

Required. SQL Server database name.

Query

Yes

Required. Query to run.

Authentication Type

No

Optional. Indicates which SQL authentication method will be used for the connection. Use 'SQL Login' to connect to a SQL Server using username and password properties. Use 'Active Directory Password' to connect to an Azure SQL Database/Data Warehouse using an Azure AD principal name and password.

Default is SQL Login.

Username

Yes

Optional. User identity for connecting to the specified database.

Password

Yes

Optional. Password to use to connect to the specified database.

Instance Name

No

Optional. SQL Server instance name to connect to. When it is not specified, a connection is made to the default instance. For the case where both the instanceName and port are specified, see the notes for port. If you specify a Virtual Network Name in the Server connection property, you cannot use instanceName connection property.

Query Timeout

No

Optional. Number of seconds to wait before a timeout has occurred on a query. The default value is -1, which means infinite timeout. Setting this to 0 also implies to wait indefinitely.

Default is -1.

Application Intent

No

Optional. Declares the application workload type when connecting to a server.

Default is ReadWrite.

Connect Timeout

No

Optional. Time in seconds to wait for a connection to the server before terminating the attempt and generating an error.

Default is 0.

Column Encryption

No

Optional. Whether to encrypt data sent between the client and server for encrypted database columns in the SQL server.

Default is Disabled.

Encrypt

No

Optional. Whether to encrypt all data sent between the client and server. This requires that the SQL server has a certificate installed.

Default is No.

Trust Server Certificate

No

Optional. Whether to trust the SQL server certificate without validating it when using SSL encryption for data sent between the client and server.

Default is No.

Workstation ID

No

Optional. Used to identify the specific workstation in various SQL Server profiling and logging tools.

Failover Partner

No

Optional. Name or network address of the SQL Server instance that acts as a failover partner.

Packet Size

No

Optional. Network packet size in bytes to use when communicating with the SQL Server.

Default is -1.

Current Language

No

Optional. Language to use for SQL sessions. The language determines datetime formats and system messages. See sys.syslanguages for the list of installed languages.

Connection Arguments

No

Optional. A list of arbitrary string key/value pairs as connection arguments. These arguments will be passed to the JDBC driver as connection arguments for JDBC drivers that may need additional configurations.

Example

You want to delete all records from SQL Server table "userEvents" of database "prod" running on localhost, port 1433, without authentication using driver "sqlserver42" if the pipeline completes successfully. Ensure that the driver for SQL Server is installed. You can also driver name for some specific driver, otherwise "sqlserver42" will be used, and then configure the plugin with:

Property

Value

Property

Value

Driver Name

sqlserver42

Host

localhost

Port

1433

Database

prod

Query

delete * from userEvents

 

Created in 2020 by Google Inc.