Fast Filter Transformation (Spark)

The Fast Filter transformation (Spark) plugin is available in the Hub.

Plugin version: 1.8.0

Filters out messages based on a specified criteria. You can use this transformation in pipelines that use the Spark execution engine.

Configuration

Property

Macro Enabled?

Description

Property

Macro Enabled?

Description

Source Field

No

Required. Specifies the source field to use for the filter. If the value in the Source Field is null, the record is dropped.

Operator

No

Required. Specifies the operator to use for the comparison.

Default is contains.

Criteria

Yes

Required. Enter a value to use for the filter.

Ignore Case?

No

Required. Select true if you’d like the filter to ignore case.

Default is false.

Example

If you want to filter all values equal to 3 in the Cost field, select Cost in the Source Field, select = for the Operator, and enter 3 for the Criteria.

 

Created in 2020 by Google Inc.