Add job project for BigQuery source


The BigQuery source requires a different set of permissions depending on how it is configured. When reading a view, using a filter, or specifying a partition, the service account needs write access to a BigQuery table because it needs to run a query and store the results in a temp table before reading. When just reading a table, it needs permissions to run an export job.

The docs should include which permissions are required. In addition, a pipeline sometimes needs to read a table from a project where it only has read access and not write access. In those situations, it should be possible to configure the source to run the query and store the results in a temp table in a different dataset where it does have write access. Alternatively, if there is a way to just read the query results directly instead of using a staging table that would be even better.

Release Notes

Your pinned fields
Click on the next to a field label to start pinning.


Bhooshan Mogal


Albert Shau