Amazon SQS Sink

The Amazon SQS sink plugin is available in the Hub.

This sink writes messages to Amazon SQS Queue in CSV or JSON format.

Configuration

Property

Macro Enabled?

Description

Property

Macro Enabled?

Description

Reference Name

No

Required. Name used to uniquely identify this sink for lineage, annotating metadata, etc.

Amazon Region

Yes

Required. The Amazon Region where the SQS queue is located. The default is us-west-1.

Queue Name

Yes

Required. The name of SQS queue to persist messages from the plugin.

SQS Endpoint

Yes

Optional. The endpoint of the SQS server to connect to. For example, https://sqs.us-east-1.amazonaws.com

Authentication Method

Yes

Required. Authentication method to access SQS. The default value is Access Credentials. IAM can only be used if the plugin is run in an AWS environment, such as on EMR.

Access ID

Yes

Optional. Amazon access ID required for authentication.

Access Key

Yes

Optional. Amazon access key required for authentication.

Message Format

Yes

Required. Specifies the format of the message published to SQS. Supported formats are CSV and JSON. Default value is JSON.

Default is JSON.

Delay (in seconds)

Yes

Required. The time, in seconds, for which a specific message is delayed. The minimum value is 0 and maximum value is 900.

Default is 0.

Created in 2020 by Google Inc.