We're updating the issue view to help you get more done. 

BigQuery sink should validate column names are valid

Description

Steps to reproduce,
1. Create a pipeline with joiner
2. Rename a coloumn in the joiner to have `-`
3. Write it to bq sink
4. Clicking on validate returns no error
5. Run preview

After step 5 the pipeline fails with illegal character in schema exception. Have attached exception for reference.

The same pipeline runs fine if `-` in the alias is replaced with `_`

Release Notes

Columns names are validated for big query sink

Activity

Show:
Albert Shau
July 6, 2020, 5:01 PM

This is correct behavior at runtime, as BigQuery doesn't support columns with the dash character '-' in them.

However, this should be caught during validation where we can highlight the problematic field and give a clear reason instead of failing at runtime.

Vinisha Shah
July 13, 2020, 6:37 PM
Fixed

Assignee

Vinisha Shah

Reporter

Ajai Narayanan

Labels

None

Docs Impact

None

UX Impact

None

Components

Fix versions

Priority

Major
Configure