Skip to:
Data profiler plugin does not calculate count for nulls/non nulls correctly due to the condition at
Steps to reproduce:
Create BQ table with null values for a column
Create batch pipelines with BQ Source → Data Profiler → BQ Sink
Run the pipeline
Observe that categorical.nulls is 0 and categorical.non_nulls is the total row count.
Data profiler plugin does not calculate count for nulls/non nulls correctly due to the condition at
Steps to reproduce:
Create BQ table with null values for a column
Create batch pipelines with BQ Source → Data Profiler → BQ Sink
Run the pipeline
Observe that categorical.nulls is 0 and categorical.non_nulls is the total row count.