Upgrading CDAP Using Cloudera Manager

Upgrading CDAP

When upgrading an existing CDAP installation from a previous version, you will need run an upgrade tool to make sure the CDAP table definitions in HBase are up-to-date.

To upgrade existing pipeline applications created using the 4.2.x versions of system artifacts, there are separate instructions on doing so. Run these after upgrading CDAP.

These steps will upgrade from CDAP 4.2.x to 6.2.0. If you are on an earlier version of CDAP, please follow the upgrade instructions for the earlier versions and upgrade first to 4.2.x before proceeding.

Upgrading CDAP Patch Release Versions

Upgrading between patch versions of CDAP refers to upgrading from one 6.2.x version to a higher 6.2.x version. When a new compatible CDAP parcel is released, it will be available via the Parcels page in the Cloudera Manager UI.

Upgrading CDAP Major/Minor Release Versions

Upgrading between major versions of CDAP (for example, from a 4.2.x version to 6.2.x) involves the additional step of upgrading the CSD. Upgrades between multiple Major/Minor versions must be done consecutively, and a version cannot be skipped unless otherwise noted.

The following is the generic procedure for all upgrades. These steps will stop CDAP, update the installation, run an upgrade tool for the table definitions, and then restart CDAP:

1. Stop all services, and other programs in all your applications.

2. Stop all CDAP services.

3. Ensure your installed version of the CSD matches the target version of CDAP. For example, CSD version 3.0.* is compatible with CDAP version 3.0.*. Download the latest version of the CSD.

4. Use the Cloudera Manager UI to download, distribute, and activate the target CDAP parcel version on all cluster hosts.

5. Before starting services, run the CDAP Upgrade Tool to upgrade CDAP. From the CDAP Service page, select "Run CDAP Upgrade" from the Actions menu. Note that authorization is disabled in the UpgradeTool so that the cdap user can upgrade all users' data.

6. Start the CDAP services. At this point it may be necessary to correct for any changes in the CSD. For example, if new CDAP services were added or removed, you must add or remove role instances as necessary. Check the release-specific upgrade notes below for any specific instructions.

7. To upgrade existing pipeline applications created using the 4.2.x versions of the system artifacts, there are separate instructions.

8. You must recompile and then redeploy your applications prior to using them.

9. Once CDAP has restarted, you can check the status of the upgrade using the Monitor Microservices:

   $ curl -w"\n" -X GET "http://<cdap-host>:11015/v3/system/upgrade/status"

   Returning:

   {"metadata":true }

Upgrading CDH

These steps cover what to do when upgrading the version of CDH of an existing CDAP installation. As the different versions of CDH can use different versions of HBase, upgrading from one version to the next can require that the HBase coprocessors be upgraded to the correct version. The steps below will, if required, update the coprocessors appropriately.

It is important to perform these steps as described, otherwise the coprocessors may not get upgraded correctly and HBase regionservers may crash. In the case where something goes wrong, see these troubleshooting instructions for problems while upgrading CDH.

Upgrade Steps

1. Upgrade CDAP to a version that will support the new CDH version, following the usual CDAP-Cloudera Manager upgrade procedure.

2. After upgrading CDAP, start CDAP and check that it is working correctly.

3. Using the CDAP UI, stop all CDAP applications and services.

4. Upgrade to the new version of CDH, following Cloudera's documentation on upgrading.

5. Start CDAP using Cloudera Manager.