Software Prerequisites

You'll need this software installed:

• A Java runtime on each CDAP node and Hadoop data node.

• A Hadoop, HBase, and Spark environment to run against.

• CDAP nodes require Hadoop and HBase client installation and configuration. Note: No Hadoop services need actually be running.

• We recommend installing an NTP (Network Time Protocol) daemon on all nodes of the cluster, including those with CDAP components.

The latest JDK or JRE version 1.8.xx for Linux, Windows, or Mac OS X must be installed in your environment; we recommend the Oracle JDK.

To check the Java version installed, run the command:

CDAP is tested with both the Oracle JDK and the Open JDK; it may work with other JDKs but it has not been tested with them.

Once you have installed the JDK, you'll need to set the JAVA_HOME environment variable.

• We recommend installing an NTP (Network Time Protocol) daemon on all nodes of the cluster, including those with CDAP components.

• NTP requires that port 123 be open.

• If your cluster does not have access to the internet, you can run a local version of NTP by setting up a master node as an NTP server.

Installing NTP on RPM using Yum

1. Install the NTP service and dependencies:

   $ sudo yum install ntp ntpdate ntp-doc

2. Set the service to start at reboot:

   $ sudo chkconfig ntpd on

3. Start the NTP server. This will continuously adjust the system time from an upstream NTP server:

   $ sudo /etc/init.d/ntpd start

4. Synchronize the system clock with the 0.pool.ntp.org server. You should use this command only once:

   $ sudo ntpdate -u pool.ntp.org

5. Synchronize the hardware clock (to prevent synchronization problems), unless on a virtual server:

   $ sudo hwclock --systohc

Installing NTP on Debian using APT

1. Install the NTP service and dependencies:

   $ sudo apt-get install ntp

2. Start the NTP server. This will continuously adjust the system time from an upstream NTP server:

   $ sudo service ntp start

3. Synchronize the system clock with the 0.pool.ntp.org server. You should use this command only once:

   $ sudo ntpdate -u pool.ntp.org

4. Synchronize the hardware clock (to prevent synchronization problems), unless on a virtual server:

   $ sudo hwclock --systohc

NTP Troubleshooting and Configuration

• To check the synchronization:

  $ ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== +173.44.32.10 18.26.4.105 2 u 5 64 1 78.786 -0.157 1.966 *66.241.101.63 132.163.4.103 2 u 7 64 1 43.085 2.872 0.409 +services.quadra 198.60.22.240 2 u 6 64 1 21.805 3.040 1.033 -hydrogen.consta 200.98.196.212 2 u 7 64 1 114.250 16.011 0.873

• If you need to adjust the configuration (add or delete servers, use servers closer to you, etc.):

  $ vi /etc/ntp.conf