CDAP(on-prem) upgrade from 6.1.1 to 6.3.0 - Upgrade steps. CDAP upgrade Failure

Description

Hi Team,

We are trying to upgrade from CDAP 6.1.1 to 6.3.0 and it failed in 6th step(1-5 steps completed successfully). Request your help regarding this. Our apache hadoop version is 2.7.2

1. Backup CDAP files (To be done by root user)

$ cp /etc/cdap/conf/* /etc/cdap/conf_old/

$ cp /opt/cdap/* /opt/cdap_old/

2. Stop all CDAP processes:

$ for i in `ls /etc/init.d/ | grep cdap` ; do sudo service $i stop ; done

3. Download CDAP 6.3.0 packages (To be done by root user)

$ curl -O https://downloads.cdap.io/cdap-distributed-rpm-bundle/cdap-distributed-rpm-bundle-6.3.0.tgz

4. Extract CDAP 6.3.0 packages (To be done by root user)

$ tar -xvzf cdap-distributed-rpm-bundle-6.3.0.tgz -C /opt/

5. Update the CDAP packages (To be done by root user)

$ cd /opt

$ sudo yum upgrade 'cdap*'

6. Run the upgrade tool (To be done by root user)

$ sudo -u cdap /opt/cdap/master/bin/cdap run io.cdap.cdap.data.tools.UpgradeTool upgrade force

7. Start all CDAP processes:

$ for i in `ls /etc/init.d/ | grep cdap` ; do sudo service $i start ; done

8. check the status of the upgrade

$ curl -w"\n" -X GET "http://<cdap-host>:11015/v3/system/upgrade/status

cdap@camdevapp1:/opt# /opt/cdap/master/bin/cdap run io.cdap.cdap.data.tools.UpgradeTool upgrade force

Starting upgrade in non interactive mode.

upgrade - Upgrades CDAP to 6.3.0-1610940078314

The upgrade tool upgrades the following:

1. User and System Datasets (upgrades the coprocessor jars)

2. Stream State Store

Note: Once you run the upgrade tool you cannot rollback to the previous version.

Starting upgrade ...

2021-02-12 15:56:32,081 - INFO [main:i.c.c.d.t.UpgradeTool@258] - Starting Zookeeper Client...

2021-02-12 15:56:32,116 - INFO [main:i.c.c.d.t.UpgradeTool@264] - Starting Transaction Service...

2021-02-12 15:56:32,117 - INFO [main:o.a.t.i.z.LeaderElection@85] - Start leader election on 172.21.86.20:2181,172.21.86.22:2181,172.21.86.21:2181/cdap/tx.service/leader with guid 671fd786-ed03-4239-a7d6-8daa31534e83

2021-02-12 15:56:32,119 - INFO [main:i.c.c.d.t.UpgradeTool@266] - Initializing Dataset Framework...

2021-02-12 15:56:34,493 - INFO [main:i.c.c.d.u.h.DefaultHBaseDDLExecutor@146] - Table 'cdap_system:app.meta' created.

2021-02-12 15:56:34,596 - INFO [main:i.c.c.d.d.InMemoryDatasetFramework@254] - Created dataset dataset:system.app.meta of type io.cdap.cdap.api.dataset.table.Table

2021-02-12 15:57:02,855 - INFO [main:i.c.c.d.u.h.DefaultHBaseDDLExecutor@146] - Table 'cdap_system:workflow.stats' created.

2021-02-12 15:57:02,958 - INFO [main:i.c.c.d.d.InMemoryDatasetFramework@254] - Created dataset dataset:system.workflow.stats of type io.cdap.cdap.api.dataset.table.Table

Failed to perform action 'UPGRADE'. Reason: 'Dataset type 'ProgramScheduleStoreDataset' is neither registered in the 'namespace:system' namespace nor in the system namespace'.

2021-02-12 15:57:02,993 - ERROR [main:i.c.c.d.t.UpgradeTool@437] - Failed to upgrade ...
io.cdap.cdap.api.dataset.DatasetManagementException: Dataset type 'ProgramScheduleStoreDataset' is neither registered in the 'namespace:system' namespace nor in the system namespace
at io.cdap.cdap.data2.dataset2.InMemoryDatasetFramework.addInstance(InMemoryDatasetFramework.java:243) ~[io.cdap.cdap.cdap-data-fabric-6.3.0.jar:na]

at io.cdap.cdap.data2.dataset2.DatasetFramework.addInstance(DatasetFramework.java:140) ~[io.cdap.cdap.cdap-data-fabric-6.3.0.jar:na]

at io.cdap.cdap.internal.app.runtime.schedule.store.ProgramScheduleStoreDataset.setupDatasets(ProgramScheduleStoreDataset.java:707) ~[io.cdap.cdap.cdap-app-fabric-6.3.0.jar:na]

at io.cdap.cdap.data.tools.UpgradeTool.initializeDSFramework(UpgradeTool.java:462) ~[io.cdap.cdap.cdap-master-6.3.0.jar:na]

at io.cdap.cdap.data.tools.UpgradeTool.startUp(UpgradeTool.java:267) ~[io.cdap.cdap.cdap-master-6.3.0.jar:na]

at io.cdap.cdap.data.tools.UpgradeTool.doMain(UpgradeTool.java:316) ~[io.cdap.cdap.cdap-master-6.3.0.jar:na]

at io.cdap.cdap.data.tools.UpgradeTool.main(UpgradeTool.java:435) ~[io.cdap.cdap.cdap-master-6.3.0.jar:na]

Release Notes

None

Assignee

Trishka Fernandes

Reporter

Adithya Cherukupalli

Labels

Docs Impact

None

UX Impact

None

Components

Affects versions

Priority

Major