UpgradeTool is not properly upgrading
Description
UpgradeTool executes and completes seemingly successfully.
However, there are log lines in it which indicate that it does not upgrade certain things.
For instance, I see this in UpgradeTool logs, even though I tried it on an insecure cluster and I can see the specified table in HBase:
Release Notes
Fixed a problem with upgrading CDAP using the CDAP Upgrade Tool.
Activity
Cause is the inconsistency in TableId construction and usage. See CDAP-7344.
PR (against 3.5.2) to fix these two upgraders: https://github.com/caskdata/cdap/pull/6776
The impact of this bug is that DatasetSpecificationUpgrader wil effectively not run and StreamStateStoreUpgrader will not upgrade stream state stores in any non-default namespaces.
However, this bug only affects 3.5.0 and 3.5.1. These upgraders have also been in 3.3.x and 3.4.x, so it should be ok to have run these upgrade steps in those versions, even if these upgrade steps didn't work in 3.5.0 or 3.5.1.
In other words, the migration that these upgrade steps perform should've already happened if the upgrade tool was run in 3.3.x or 3.4.x.
PR to have the fix in 3.6: https://github.com/caskdata/cdap/pull/6823
PR to merge release/3.6 into develop, ensuring that the necessary change is on 4.0: https://github.com/caskdata/cdap/pull/6831
