For BigQuery Pushdown, fixed an issue when BigQuery Pushdown was enabled for an existing dataset, the Location where the BigQuery Sink executed jobs was the location specified in the Pushdown configuration, not the BigQuery Dataset location. The configured Location should have only been used when creating resources. Now, if the dataset already exists, the Location for the existing dataset is used.
I noticed during testing that if BQ Pushdown is enabled using an existing dataset, the Location where the BigQuery Sink executed jobs is the location specified in the Pushdown configuration, not the BigQuery Dataset location.
This configured Location should only be used when creating resources. If the dataset already exists, the location for the existing dataset should be used.
The bug can be found in this line:
The correct action can be seen here using location determined using
The simple fix is to replace this location with the execution dataset location.