Running upgrade tool for more than once results in following error:
Steps to reproduce:
1.) Create 3.5 cluster and install 4.1 rpms
2.) Run upgrade tool twice
somehow, that table has the exact same coprocessor twice:
We remove HBaseQueueRegionObserver, then add HBaseQueueRegionObserver, and it fails because there's still one HBaseQueueRegionObserver in the list. Not sure how 2 of the same coprocessor got there to begin with...
this is a tricky bug. It only affects tables that have more than 1 coprocessor.
The root cause is the logic that converts an HTableDescriptor into a TableDescriptor for the HBase SPI. Internally, when you call addCoprocessor() on an HTableDescriptor, it converts the coprocessor information into a property with key 'coprocessor$[num]', where 'num' is the max coprocessor number. So if there is a table with 2 existing coprocessors, when we add the new updated coprocessors, it ends up adding 'coprocessor$3' and 'coprocessor$4' to the coprocessor properties. When we delete the old coprocessors from the descriptor, 'coprocessor$1' and 'coprocessor$2' get removed. Then, when we convert it to a TableDescriptor, we copy all properties into the TableDescriptor, which means 'coprocessor$3' and 'coprocessor$4' properties get copied. In addition, the TableDescriptor contains the CoprocessorDescriptor for the 2 new coprocessor. When we convert that TableDescriptor back into an HTableDescriptor, we copy all properties and we add all coprocessors. This results in 'coprocessor$1' and 'coprocessor$2' from the addCoprocessor() call, as well as 'coprocessor$3' and 'coprocessor$4' from the properties.
To fix, we'll just make sure not to copy coprocessor properties into the TableDescriptor.