Skip to:

Skip to Jira Navigation
Skip to Side Navigation
Skip to Main Content

BigQuery Source tries to run jobs in the wrong location

Description

BQ Source Pushdown uses the configured Location when executing jobs.

However, this location should only be used when creating a new Dataset for staging. If the staging dataset already exists, the location for this staging dataset should be the location of the jobs we execute.

See https://github.com/data-integrations/google-cloud/blob/47580e302154c34156c54296063e234e8065fb5d/src/main/java/io/cdap/plugin/gcp/bigquery/sqlengine/BigQueryReadDataset.java#L250

Release Notes

n/a

Linked issues

is duplicated by

BQ pushdown not able to auto-detect dataset location even if it exists

Activity

Show:

Sanket Sahu October 26, 2022 at 5:20 PM

cherrypick : https://github.com/data-integrations/google-cloud/pull/1170

Sanket Sahu October 21, 2022 at 11:19 AM

Initial pr : https://github.com/data-integrations/google-cloud/pull/1169

Avinash Achar October 21, 2022 at 7:59 AM

Should be fixed

Unresolved

Pinned fields

Click on the next to a field label to start pinning.

Details

Assignee

Sanket Sahu

Reporter

Fernando Velasquez

Triaged

Yes

Fix versions

Priority

Major

More fields

Created October 18, 2022 at 8:45 PM

Updated November 4, 2022 at 6:56 PM