We're updating the issue view to help you get more done. 

Add Kafka streaming source

Description

None

Release Notes

Added a Kafka source for streaming pipelines.

Activity

Show:
Albert Shau
July 21, 2016, 10:28 PM

after some experimentation, we'll probably have to provide multiple plugins for different versions of kafka.

For example, cdh 5.7 uses Kafka 0.9 while older versions use Kafka 0.8 Their github repo doesn't seem to reflect this, but the KafkaCluster class in spark-assembly.jar in their package uses BrokerEndPoint, which is only present in 0.9+.

Will have to do some research to see what versions of hdp use what as well.

Albert Shau
July 29, 2016, 11:38 PM
Fixed

Assignee

Albert Shau

Reporter

Albert Shau

Labels

None

Docs Impact

None

UX Impact

None

Components

Fix versions

Priority

Major
Configure