Apache Kudu Batch Source (Deprecated)

This plugin is no longer available as of July 26, 2024.

CDAP plugin for reading data from Apache Kudu table.

Configuration

Property	Macro Enabled?	Description

Property	Macro Enabled?	Description
Reference Name	No	Required. Name used to uniquely identify this source for lineage, annotating metadata, etc.
Table Name	Yes	Required. Kudu table name to which the records will be written. This plugin checks if the table already exists. If it exists, it compares the schema of the existing table with the write schema specified for the plugin, If they don't match an error is thrown at configuration time and If the table doesn't exist, the table is created.
Master Addresses	Yes	Required. List of Kudu master hosts that this plugin will attempt connect to. It's a comma separated list of <hostname>:<port>. Connection is attempt after the plugin is initialized in the pipeline.
Projected Columns	No	Optional. Columns to be projected. By default it's '' for projecting all columns in the table. Default is (asterisk).
User operations timeout	No	Optional. Sets the timeout in milliseconds for user operations with Kudu. If you are writing large sized records it's recommended to increase the this time. Default is 10000 ms.

Usage Notes

This plugin is configured to pull rows from a Kudu table using the Kudu native client. When you configure this plugin in a pipeline, use the Generate Table Schema to automatically create the CDAP schema from the Kudu table. In order to generate the schema, you would have to first specify the Table name and the Master address.

In case, one of the configuration (either Table name or Master address) is a macro the Generate Table Schema will not work. The best way is to provide a non-macros configuration to generate the schema first and then change it the configuration use macro.

The plugin also supports projecting columns efficiently when it's specified. If '*' is specified, then all the columns in the table are projected else only the fields specified will be projected. Make sure the columns to be projected are specified before schema is generated.