Deploying a data pipeline
After you finish designing and debugging a data pipeline and are satisfied with the data you see in Preview, you are ready to deploy the pipeline.Â
Note: A deployed pipeline name must be unique in the namespace. You might be prompted to enter a unique name.
Deploying converts the pipeline into an immutable entity, which is not versioned.
After deployment, the pipeline can be either run manually using the Run button, scheduled on a time basis using the Schedule button, or triggered based on the completion of another pipeline using Incoming Triggers. The Pipeline Studio saves a run history of the pipeline each time it runs. You can toggle between different runtime versions of the pipeline:
When you deploy the pipeline, the Pipeline Studio creates the workflow and corresponding Spark jobs in the background.
Once you deploy a pipeline, you cannot edit it. You can duplicate it or export it and then import it into the Pipeline Studio.
After you deploy a pipeline, you can configure runtime arguments if the pipeline includes macros.
Created in 2020 by Google Inc.