YesIntroduction

Confluent Cloud is a streaming data service with Apache Kafka delivered as a managed service.

User Storie(s)

As a pipeline developer, I would like to stream data real time from Confluent cloud based on the specified schema by the user
As a pipeline developer, I would like to stream data real time to Confluent Cloud and perform serialization during event streaming
As a pipeline developer I would like to capture records that did not get delivered downstream to Confluent Cloud for analysis

Real time source

Property	description	Mandatory or Optional
Kafka cluster credential API key	API key in order to connect to the cluster	Yes
Kafka cluster credential secret	Secret in order to connect to the cluster	Yes
Kafka Zookeeper	The connect string location of ZooKeeper. Either that or the list of brokers is required	Required if brokers not specified
Kafka brokers	Comma-separated list of Kafka brokers. Either that or the ZooKeeper quorum is required	Required if zookeeper not specified
Kafka partition	Number of partitions	Yes

Kafka topic	List of topics which we are listening to for streaming	Yes
Schema registry URL	URL endpoint for the schema registry on Confluent Cloud or self hosted schema registry URL	No
Schema registry API key	API key	No
Schema registry secret	Secret	No

Real time sink

Property	description

Kafka topic	List of topics to which the data should be published to
Schema registry URL	URL endpoint for the schema registry on Confluent Cloud
Schema registry API key	API key for
Schema registry secret

References

Design / Implementation Tips

Please attach one or more sample pipeline(s) and associated data.

Table of Contents

Checklist