In the Pipeline Studio, you can use macros to add dynamic variables to plugin configurations so that you can specify the variable substitutions at runtime.
With reusable data pipelines, you can have a single pipeline that can apply a data integration pattern to a variety of use cases and datasets. Reusable pipelines are give you better manageability. They let you set most of the configuration of a pipeline at execution time, instead of hard-coding it at design time.
Fields that can be made dynamic using macros have a blue M icon on the far right in the Plugin Configuration section:
...
You can also use the Argument Setter plugins to build a reusable pipelines:
...
...
...
GCS Argument Setter. To learn more, see Creating a reusable pipeline with the GCS Argument Setter.
...
In the Pipeline Studio, you can use macros to add dynamic variables to plugin configurations so that you can specify the variable substitutions at runtime.
That pipeline behavior will be different based on the namespace or the instance it is executed in. Macros let you design the pipeline once, and run it in different environments in various forms.
You can add macros to any plugin, including Argument Setters.
You can set the values for macros in Pipeline Studio in the following areas:
When you run a pipeline, CDAP evaluates the macro values in the same order listed above. For example, if you set the value for a macro as a namespace preference and also in a runtime preference, CDAP uses the value in the runtime preference.