...
The GCS Argument Setter provides the ability to read a JSON file from GCS to set arguments for macros in the pipeline. It also provides secure access to your source data by allowing you to set the Service Account information for GCS. It is most commonly used when the structure of a pipeline is static, and its configuration needs to be managed outside the pipeline.
| Info |
|---|
Recommended: For better security and ease of use, use the GCS Argument Setter rather than the HTTP Argument Setter, which doesn’t provide authentication support. |
Objectives
Use the GCS Argument Setter plugin to allow the pipeline to read different input in every run.
Use the GCS Argument Setter plugin to allow the pipeline to perform different quality checks in every run.
Use the GCS Argument Setter to ensure your data is secure during when you run a pipeline.
Write the output data of each run to Cloud Storage.
...