We're updating the issue view to help you get more done. 

BigQuery source should not validate schema in the initialize method

Description

Schema validation already happens in prepareRun(), so we don't need it in initialize().

As part of validation, we call BigQuery.getTable(). These calls can be rate limited if they are called concurrently from many exevcutors. We can avoid rate limiting issues by validating schema only in prepareRun.

Release Notes

Removed redundant validations from BQ sink , this should reduce calls to BigQuery.getTable()

Activity

Show:
Greeshma Swaminathan
June 29, 2020, 10:21 PM

Fix available on Jun 25th RC build 

Fixed

Assignee

Greeshma Swaminathan

Reporter

Prashant Jaikumar

Labels

None

Docs Impact

None

UX Impact

None

Components

Fix versions

Priority

Major
Configure