Amazon Kinesis Stream Sink (Deprecated)

This plugin is no longer available as of July 26, 2024.

Plugin version: 1.7.0

The Amazon Kinesis Stream sink plugin writes to a specified Amazon Kinesis Stream.

Configuration

Property	Macro Enabled?	Description

Property	Macro Enabled?	Description
Stream Name	No	Required. The name of the Kinesis stream to output to. Must be a valid kinesis stream name. The Kinesis stream will be created if it does not exist.
Access ID	Yes	Required. The Access ID provided by AWS required to access the Kinesis streams.
Access Key	Yes	Required. AWS access key secret having access to Kinesis streams.
Shard count	Yes	Optional. Number of shards to be created, each shard has input of 1 MB. If the stream already exists, the number of shards will not be modified. Default value is 1.
Distribute	No	Required. Boolean to decide if the data has to be uniformly distributed among all the shards or has to be sent to a single shard. Default is `true`.
referenceName	No	Required. Name used to uniquely identify this sink for lineage, annotating metadata, etc.

Example

This example will write to a kinesis stream named 'MyKinesisStream'. The kinesis stream will be created if it does not exists already. Each record it receives will be written as a single stream event. Two shards for this stream will be created and records will be distributed uniformly across the records:

Property	Value

Property	Value
Stream Name	`purchases`
Access ID	`my_aws_access_key`
Access Key	`my_aws_access_secret`
Shard count	`2`
Distributed	`true`
referenceName	`KinesisStream`