Exporting hydrator pipelines should work with newer artifact versions
Description
Release Notes
duplicates
Activity
Bhooshan Mogal September 14, 2017 at 11:41 PM
This has been fixed as part of the upgrade flow for importing pipelines.
Russ Savage July 13, 2016 at 4:29 PM
I think the use case for this is if I have a production hydrator pipeline, and I want to test it on the new version of CDAP that i have on my laptop, i can't import that exported JSON into the latest cdap without manually editing it. Maybe there should just be a popup when you try to import something with old versions that asks if you want to upgrade the pipeline with the big red WARNING.
Ali Anwar June 24, 2016 at 9:39 PM
If the new cluster has both 1.3.1, and 1.3.2 (old cluster also has both versions), then it'll switch to the newer version when imported into the new cluster?
And what if there's 1.3.1, 1.3.2, and a 1.4.0 or 1.5.0 on the new cluster.
My point is that even though backwards compatibility is there, I don't think automatic upgrading is desirable (particularly, in a production environment, where the users would want to freeze on particular versions).
Russ Savage June 24, 2016 at 8:14 AM
We aren't really migrating. We specify that the export should work with at least any minor changes to the version. Most of the time, something old should work with newer minor version of a plugin. core-plugins 1.3.1 vs 1.3.2 shouldn't break any backwards compatibility. So when you export a pipeline with 3.4.1 with 1.3.1 plugins, you should be able to easily import that into a cluster with 3.4.2 with 1.3.2 without manually editing the json. That's not possible today because all the exports use exact versions of every artifact.
Ali Anwar June 23, 2016 at 7:52 PM
It would be a problem if we automatically migrated versions of plugins when importing the config, especially in a production environment.
When exporting a hydrator pipeline config, all versions are absolute. That means when migrating from 3.4.1 to 3.4.2, you need to export, then manually convert the versions. You also need to convert the plugin artifact versions too.
By default, hydrator pipeline exports should work with the current or newer versions of an artifact.