Fast Filter Transformation (Spark)
The Fast Filter transformation (Spark) plugin is available in the Hub.
Plugin version: 1.8.0
Filters out messages based on a specified criteria. You can use this transformation in pipelines that use the Spark execution engine.
Configuration
Property | Macro Enabled? | Description |
---|---|---|
Source Field | No | Required. Specifies the source field to use for the filter. If the value in the Source Field is null, the record is dropped. |
Operator | No | Required. Specifies the operator to use for the comparison. Default is contains. |
Criteria | Yes | Required. Enter a value to use for the filter. |
Ignore Case? | No | Required. Select true if you’d like the filter to ignore case. Default is false. |
Example
If you want to filter all values equal to 3 in the Cost field, select Cost
in the Source Field, select =
for the Operator, and enter 3
for the Criteria.
Created in 2020 by Google Inc.