Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

For example, if a pipeline has a GCS source, you can set the path to a macro. In the dev environment, you can set up the runtime arguments so that the path resolves to the dev bucket. In the production environment, you can set up the runtime arguments so that the path resolves to the product bucket. Similarly, if the pipeline is writing to BigQuery, the dataset can be set to a macro so that the development pipeline writes to the development dataset and the production pipeline writes to the production dataset.

Note: You can also To export all deployed data pipelines for all namespaces. See Lifecycle Microservices, see Lifecycle Microservices, “Export All Application Details”. To export all data pipelines in Draft mode, see Pipeline Microservices, “List Draft Pipelines”.