Pipeline failing after deployment with Window Aggregation when using ROW_NUMBER, CONTINUOUS_PERCENTILE, MEDIAN and LEAD function
Description
Release Notes
None
Attachments
3
Activity
Show:
Ankit Jain April 4, 2023 at 12:45 PM
yeah ROW_NUMBER() is working as intended now. I think we can close this bug if other issues are already being tracked in other JIRAs.
Sagar Ahire April 4, 2023 at 12:10 PM
For the Integer → Long cast issue, ROW_NUMBER() outputs the result as an int and the default output schema given by the Window Aggregation plugin is an int for the field added by ROW_NUMBER(). If the schema of the BQ table in the sink is also set to int, the pipeline runs fine, but the pipeline fails if the schema in the sink is set to long.
I believe this is working as intended.
Sanket Sahu April 3, 2023 at 11:49 AM
There are 3 issues / errors are mentioned.
We need to look at java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
Others are already having individual bugs
Unresolved
Pinned fields
Click on the next to a field label to start pinning.
Created March 31, 2023 at 12:02 PM
Updated April 4, 2023 at 12:51 PM
When creating a pipeline with window aggregation plugin using
ROW_NUMBER()orCONTINUOUS_PERCENTILE()orMEDIANfunction it fails after deployment with pushdown enabled.When creating a pipeline with window aggregation plugin using
ROW_NUMBER()function with pushdown disabled.Note: preview run was successful with all functions.
When pushdown is disabled with
ROW_NUMBER():When pushdown is enabled:
With
LEAD():Please find the attached pipeline json and pipeline logs of both runs.