Amazon Kinesis Stream Sink (Deprecated)

This plugin is no longer available as of July 26, 2024.

Plugin version: 1.7.0

The Amazon Kinesis Stream sink plugin writes to a specified Amazon Kinesis Stream.

Configuration

Property

Macro Enabled?

Description

Property

Macro Enabled?

Description

Stream Name

No

Required. The name of the Kinesis stream to output to. Must be a valid kinesis stream name. The Kinesis stream will be created if it does not exist.

Access ID

Yes

Required. The Access ID provided by AWS required to access the Kinesis streams.

Access Key

Yes

Required. AWS access key secret having access to Kinesis streams.

Shard count

Yes

Optional. Number of shards to be created, each shard has input of 1 MB. If the stream already exists, the number of shards will not be modified.

Default value is 1.

Distribute

No

Required. Boolean to decide if the data has to be uniformly distributed among all the shards or has to be sent to a single shard.

Default is true.

referenceName

No

Required. Name used to uniquely identify this sink for lineage, annotating metadata, etc.

Example

This example will write to a kinesis stream named 'MyKinesisStream'. The kinesis stream will be created if it does not exists already. Each record it receives will be written as a single stream event. Two shards for this stream will be created and records will be distributed uniformly across the records:

Property

Value

Property

Value

Stream Name

purchases

Access ID

my_aws_access_key

Access Key

my_aws_access_secret

Shard count

2

Distributed

true

referenceName

KinesisStream



Created in 2020 by Google Inc.