Runtime Arguments

Owned by Robin Rielley
Last updated: Jun 07, 2021
2 min read

Use runtime arguments to set values for macros before you preview data for a pipeline and before you run a deployed pipeline. If the pipeline has macros that are set in preferences, the runtime arguments list the key/value pairs. Optionally, you can override these values in the runtime arguments.

Runtime arguments inherit macros from Application preferences. Application preferences inherit macros from Namespace preferences and Namespace preferences inherit macros from System preferences.

Important: Changes to runtime arguments are saved for all future runs of the pipeline. This might impact any schedules or triggers configured for the pipeline.

Runtime arguments are not saved in the pipeline configuration, which means they are not included in the JSON file when you export a pipeline.

If the pipeline includes an Argument Setter action plugin, leave the value fields blank to notify CDAP that the Argument Setter plugin in the pipeline will set the values of these arguments during runtime.

Macros are resolved in the following order:

  1. Argument Setter plugin

  2. Runtime arguments

  3. Application preferences

Tip: You can use wildcards (*) in the runtime argument values. For example, to read all .txt files that start with POS, enter POS*.txt. To read all .txt files in the directory, enter *.txt. If the folder contains only .txt files, enter / to read all files in the directory.

You can set runtime arguments and preferences with the Pipeline Studio, the CDAP CLI, the Lifecycle Microservices, or Preferences Microservices.

For more information, see Setting Preferences and Macros and macro functions.

Runtime arguments for previewing data

When you preview data for a pipeline, click Configure to set the values for each macro in the pipeline.

Note: Actions and Post-run actions are ignored when running preview.

Runtime arguments for running deployed pipelines

If a pipeline includes macros, after you deploy a pipeline, you add runtime arguments to set the values for the macros.

When you deploy a pipeline with macros, click Run to open Runtime Arguments and set the values for each macro.

To set preferences for a pipeline, follow these steps:

  1. Deploy a pipeline.

  2. On the Deployed Pipeline page, click the down arrow next to the Run button.

     

  3. On the Runtime Arguments page, specify the value for each macro in the pipeline:

 

Created in 2020 by Google Inc.