Document supported Kafka versions
Description
Release Notes
None
Activity
Show:
Pinned fields
Click on the next to a field label to start pinning.
Details
Details
Assignee

Reporter

Labels
Affects versions
Components
Priority
Created November 18, 2016 at 12:01 AM
Updated September 20, 2018 at 1:32 AM
We currently don't document which version of Kafka we ship with CDAP, nor which version(s) we would support for an external Kafka server.
The current version of Kafka (as of 4.0.0-SNAPSHOT) is 0.8.2.2, and can be found in the pom.xml:
We mention the embedded version on http://docs.cask.co/cdap/current/en/admin-manual/cdap-components.html.
We mention using an external version on:
http://docs.cask.co/cdap/current/en/developers-manual/building-blocks/audit-logging.html#audit-logging-configuring-audit-publishing
http://docs.cask.co/cdap/current/en/developers-manual/building-blocks/metadata-lineage.html#metadata-update-notifications (deprecated)
Each of the installation guides mentions Kafka, such as http://docs.cask.co/cdap/current/en/admin-manual/installation/packages.html?highlight=kafka.
The build script (common_conf.py) could be modified to extract the Kafka version as it does the CDAP version (in get_sdk_version()) and publish it as part of the
rst_epilog
, as the "embedded-kafka-version".We need to test higher versions to see if they are compatible; currently, only 0.8.2.2 is known as.
Not certain where to add this version information, as this is not obvious. Perhaps on either of these:
http://docs.cask.co/cdap/current/en/admin-manual/hadoop-compatibility.html
http://docs.cask.co/cdap/current/en/admin-manual/cdap-components.html