There are a few problems with handling of macros, runtime arguments and preferences in pipelines. To list a few:
1. Every single time you run or schedule a pipeline, you have to click twice, even if your pipeline doesn't use macros
2. The UI automatically populates the macro keys, but their values are not derived from preferences.
3. UI expects every single macro key to be defined (unless of course you explicitly delete it)
4. Macros are not available in preview mode
Creating a top-level jira. Will add subtasks to address each of these issues.
Users can now set both runtime arguments and other pipeline configurations through the new Configure modeless. If there are macro keys defined in the pipeline stages, their values will be derived from Preferences. Users can also mark a macro runtime argument as Provided if the value will be provided by a stage in the pipeline.
2. The UI automatically populates the macro keys, but their values are not derived from preferences.
The UI pulls them from the App level, but not from the namespace or overall cdap level. It needs to pull from all levels properly.
Also, there are plugins that populate values in the workflow token as well (see ssh action). We need to handle that smartly in the UI and not allow users to fill in a value for those.
Also, it would be really great to have the option to save the runtime args entered as app or namespace preferences directly from that UI. This would easily allow users to set "defaults" when building multiple versions of the same pipeline, since the prefs aren't copied over.
I've also linked a bunch of issues to this one related to runtime args. Those should be looked at as well.
As a user, I can add, edit and delete new runtime arguments for that session.
I can also edit runtime arguments I have inherited from preferences.
When I edit an inherited arguments, a "Reset" link appears.
Clicking the link will reset the inherited argument to its original values. New values are used as runtime arguments and do persist as such until reset, but they do NOT modify the original preference values.
After I reset an inherited argument a "Reset Successful" label is displayed inline. The message disappears after 3 seconds
Inherited arguments cannot be deleted.
PR for inital config modeless (with runtime arguments tab): https://github.com/caskdata/cdap/pull/8731