BigQuery source exports data to temporary tables in the source dataset and then from these temporary tables to the GCS bucket. This workflow is followed (i.e. temporary table creation in source dataset) when one of the following configurations in source is used:
1. partition
2. filter
3. view
For view we need to create temporary table because of the limitation of bigquery view from where data cant be exported directly to GCS. However we need to check if exporting data to temporary tables is really required for partition and filter cases.
Also these temporary tables should be deleted as soon as data is lands in GCS otherwise they can cost users.
PR:
Cherrypicks:
release/0.14 -->
release/0.15 -->
release/0.16 -->