Data profiler plugin does not calculate nulls/non nulls correctly

Description

Data profiler plugin does not calculate count for nulls/non nulls correctly due to the condition at

Steps to reproduce:

  1. Create BQ table with null values for a column

  2. Create batch pipelines with BQ Source → Data Profiler → BQ Sink

  3. Run the pipeline

  4. Observe that categorical.nulls is 0 and categorical.non_nulls is the total row count.

Release Notes

None

Activity

Show:
Unresolved
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Components

Affects versions

Priority

More fields

Created December 28, 2022 at 7:24 AM
Updated January 9, 2023 at 5:02 PM