Kafka Server parameters (cdap-site.xml and cdap-default.xml)

kafka.seed.brokers

127.0.0.1:9092

Comma-separated list of CDAP Kafka service brokers; for Distributed CDAP, replace with list of FQDN:port brokers.

kafka.server.default.replication.factor

1

CDAP Kafka service replication factor; used to replicate Kafka messages across multiple machines to prevent data loss in the event of a hardware failure. The recommended setting is to run at least two CDAP Kafka servers. If you are running two CDAP Kafka servers, set this value to 2; otherwise, set it to the maximum number of tolerated machine failures plus one (assuming you have that number of machines).

kafka.server.host.name

0.0.0.0

CDAP Kafka service bind address.

kafka.server.log.dirs

/tmp/kafka-logs

Comma-separated list of CDAP Kafka service log storage directories.

kafka.server.log.flush.interval.messages

10000

The interval length (in number of messages in the CDAP Kafka service) at which to force an fsync of data written to the log.

kafka.server.log.retention.hours

24

The number of hours to keep a log file before deleting it; this is the time-to-live in the CDAP Kafka service, while a log is in-flight between the container and the CDAP log saver.

kafka.server.num.partitions

10

Default number of partitions for a topic in the CDAP Kafka service.

kafka.server.port

9092

CDAP Kafka service bind port.

kafka.server.zookeeper.connection.timeout.ms

1000000

Maximum time in milliseconds that the CDAP Kafka service will wait to establish a connection to ZooKeeper.

kafka.zookeeper.namespace

kafka

CDAP Kafka service ZooKeeper namespace.

kafka.zookeeper.quorum

CDAP Kafka service ZooKeeper quorum and namespace. If set, this will override the ZooKeeper quorum (set by ${zookeeper.quorum}) and the ZooKeeper namespace (set by ${kafka.zookeeper.namespace}) when setting up a connection to the Kafka service used by CDAP. If the same Kafka service ZooKeeper quorum and namespace are shared by multiple CDAP instances, each CDAP instance needs to distinguish its Kafka topics from those of other CDAP instances with unique values for ${log.kafka.topic} and ${metrics.topic.prefix}.