We're updating the issue view to help you get more done. 

SnowflakeRecordReader throws ArrayOutOfBoundsException upon Different Header and Row Lengths

Description

During Snowflake source plugin operation, the plugin first uses a COPY INTO command to copy the data from the source table into staging files, and then each executor creates a new SnowflakeRecordReader to process one of the staging file splits. However, the SnowflakeRecordReader makes the assumption that each file has a header row which matches the lengths of the row length (code), but we have run into some issues where this may not be the case.

We should explicitly be checking the row length matches the header length for each record and either throwing the proper error or logging a warning for row which does not match.

Release Notes

None

Assignee

Unassigned

Reporter

Dennis Li

Labels

None

Dev Complete Date

None

Publish Date

None

Reviewer

None

Components

Fix versions

Priority

Major
Configure