Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Directive

Version Introduced

Description

JSON Path

Uses a DSL (a JSON path expression) for parsing JSON records

Parse as AVRO

Parses an AVRO encoded message - either as binary or json

Parse as AVRO File

Parses an AVRO data file

Parse as CSV

Parses an input record as comma-separated values

Parse as Currency

Parses a currency value that is a string representation of locale currency into a number.

Parse as Date (Deprecated)

Parses dates using natural language processingUse Parse as Simple Date.

Parse as Datetime

6.4.0

Parses string as datetime with a given format.

Parse as Excel

Parsing Excel file.

Parse as Fixed Length

Parses as a fixed length record with specified widths

Parse as HL7

Parses Health Level 7 Version 2 (HL7 V2) messages

Parse as JSON

Parses a JSON object

Parse as Log

Parses access log files as from Apache HTTPD and nginx servers

Parse as Simple Date

Parses date strings

Parse as Timestamp

Parses column values representing unix timestamp as date

Parse XML To JSON

Parses an XML document into a JSON structure

...

Directive

Version Introduced

Description

Current Datetime

6.4.0

Generates current datetime with a given time zone

Datetime to Timestamp

6.4.0

Creates a timestamp from datetime and given time zone

Format Datetime

6.4.0

Formats datetime to a string of a given format

Timestamp to Datetime

6.4.0

Converts a timestamp to datetime

...

Row Operations

Directive

Description

Filter Rows

Filters records based on a condition.

Filter Rows On

Deprecated. Filters records based on a condition.

Filter Row if Matched

Filters rows that match a pattern for a column

Filter Row if True (Deprecated)

Filters rows if the condition is true.

Filter Row Empty of Null

(Obsolete?) Filters rows that are empty of null.

Flatten

Separates the elements in a repeated field

Fail on condition

Fails processing when the condition is evaluated to true.

Send to Error

Filtering of records to an error collector

Send to Error and Continue Processing

Filtering of records to an error collector and continues processing

Split to Rows

Splits based on a separator into multiple records

...