CDAP Setup (Replication)

CDAP requires that you provide an extension that will perform HBase-related DDL operations on both clusters instead of only on a single cluster. To create the extension, you must implement the HBaseDDLExecutor class. Details on implementing this class, a sample implementation, and example files are available in the Appendix: HBaseDDLExecutor.

CDAP must have invalid transaction list pruning disabled, as this cannot be used with replication.

To deploy your extension (once compiled and packaged as a JAR file, such as my-extension.jar), run these steps on both your master and slave clusters. These steps assume <cdap-home> is /opt/cdap:

1. Create an extension directory, such as:

   $ mkdir -p /opt/cdap/master/ext/hbase/repl

2. Copy your JAR to the directory:

   $ cp my-extension.jar /opt/cdap/master/ext/hbase/repl/

3. Modify cdap-site.xml to use your implementation of HBaseDDLExecutor:

   <property> <name>hbase.ddlexecutor.extension.dir</name> <value>/opt/cdap/master/ext/hbase</value> </property>

4. Modify cdap-site.xml with any properties required by your executor. Any property prefixed with cdap.hbase.spi.hbase. will be available through the HBaseDDLExecutorContext object passed into your executor's initialize method:

5. Modify cdap-site.xml to disable invalid transaction list pruning, as it cannot be used with replication:

6. Before starting CDAP on the master cluster, run a command on the slave cluster to load the HBase coprocessors required by CDAP onto the slave's HDFS:

7. Start CDAP on the master cluster: