Upgrading CDAP Using Packages

Upgrading CDAP

When upgrading an existing CDAP installation from a previous version, you will need run an upgrade tool to make sure the CDAP table definitions in HBase are up-to-date.

To upgrade existing pipeline applications created using the 4.2.x versions of system artifacts, there are separate instructions on doing so. Run these after upgrading CDAP.

These steps will upgrade from CDAP 4.2.x to 6.5.0. If you are on an earlier version of CDAP, please follow the upgrade instructions for the earlier versions and upgrade first to 4.2.x before proceeding.

Upgrading CDAP Patch Release Versions

Upgrading between patch versions of CDAP refers to upgrading from one 6.5.x version to a higher 6.5.x version.

Upgrading CDAP Major/Minor Release Versions

Upgrading between major/minor versions of CDAP refers to upgrading from a 4.2.x version to 6.5.x. Upgrades between multiple major/minor versions must be done consecutively, and a version cannot be skipped unless otherwise noted.

Upgrade Steps

These steps will stop CDAP, update the installation, run an upgrade tool for the table definitions, and then restart CDAP:

Stop all services, and other programs in all your applications.
Stop all CDAP processes:
$ for i in `ls /etc/init.d/ | grep cdap` ; do sudo service $i stop ; done
Update the CDAP repository definition by running either of these methods:
- On RPM using Yum:
  $ sudo curl -o /etc/yum.repos.d/cask.repo https://repository.cdap.io/centos/6/x86_64/cdap/6.2/cask.repo
  This will create the file /etc/yum.repos.d/cask.repo with:
  [cask] name=Cask Packages baseurl=https://repository.cdap.io/centos/6/x86_64/cdap/6.2 enabled=1 gpgcheck=1
  Add the Cask Public GPG Key to your repository:
  Update your Yum cache:
- On Debian using APT:
  This will create the file /etc/apt/sources.list.d/cask.list with:
  Add the Cask Public GPG Key to your repository:
  Update your APT-cache:
Update the CDAP packages by running either of these methods:
- On RPM using Yum:
- On Debian using APT:
Run the upgrade tool, as the user that runs CDAP Master (the CDAP user, indicated by <cdap-user>):
Note that once you have upgraded an instance of CDAP, you cannot reverse the process; down-grades to a previous version are not possible. Also, note that authorization is disabled in the UpgradeTool so that the cdap user can upgrade all users' data.
The UpgradeTool will produce output similar to the following, prompting you to continue with the upgrade:
You can run the tool in a non-interactive fashion by using the force flag, in which case it will run unattended and not prompt for continuing:
Restart the CDAP processes:
To upgrade existing pipeline applications created using the 4.2.x versions of system artifacts, there are separate instructions on doing so.
Once CDAP has restarted, you can check the status of the upgrade using the Monitor Microservices:
Returning:

Upgrading Hadoop

To upgrade Hadoop, contact your remote Hadoop provider.

Upgrade Steps

These steps cover what to do when upgrading the version of Hadoop of an existing CDAP installation. As the different versions of Hadoop can use different versions of HBase, upgrading from one version to the next can require that the HBase coprocessors be upgraded to the correct version. The steps below will, if required, update the coprocessors appropriately.

It is important to perform these steps as described, otherwise the coprocessors may not get upgraded correctly and HBase regionservers may crash.

Upgrade CDAP to a version that will support the new Hadoop version, following the usual CDAP upgrade procedure for packages.
After upgrading CDAP, start CDAP and check that it is working correctly.
Stop all CDAP applications and services:
Disable all CDAP tables; from an HBase shell, run the command:
Upgrade to the new version of Hadoop.
Run the Post-Hadoop Upgrade Tasks—to upgrade CDAP for the new version of Hadoop—by running the CDAP Upgrade Tool, as the user that runs CDAP Master (the CDAP user, indicated by <cdap-user>):
Enable all CDAP tables; from an HBase shell, run this command:
Restart CDAP: