Page Comparison

A collection for Directives available within Wrangler.

Parsers

Directive	Version Introduced	Description
JSON Path		Uses a DSL (a JSON path expression) for parsing JSON records
Parse as AVRO

Parses an AVRO encoded message - either as binary or json

Parse as AVRO

File		Parses an AVRO data file
Parse as CSV		Parses an input record as comma-separated values
Parse as Currency		Parses a currency value that is a string representation of locale currency into a number.
Parse as Date (Deprecated)		Use Parse as Simple Date.
Parse as Datetime	6.4.0	Parses string as datetime with a given format.
Parse as Excel		Parsing Excel file.
Parse as Fixed Length		Parses as a fixed length record with specified widths
Parse as HL7		Parses Health Level 7 Version 2 (HL7 V2) messages
Parse as JSON		Parses a JSON object
Parse as Log		Parses access log files as from Apache HTTPD and nginx servers
Parse as Simple Date		Parses date strings
Parse as Timestamp		Parses column values representing unix timestamp as date
Parse XML To JSON		Parses an XML document into a JSON structure

Directive	Description
Write as CSV	Converts a record into CSV format
Write as JSON Map	Converts the record into a JSON map
Write JSON Object	Composes a JSON object based on the fields specified.
Format as Currency	Formats a number as currency as specified by locale.

Directive	Description
Changing Case	Changes the case of column values
Create Record	Creates a record column with nested values by copying values from source columns into a destination column
Cut Character	Selects parts of a string value
Set Column	Sets the column value to the result of an expression execution
Find and Replace	Transforms string column values using a "sed"-like expression
Index Split	(Deprecated)
Invoke HTTP	Invokes an HTTP Service (Experimental, potentially slow)
Quantization	Quantizes a column based on specified ranges
Extract Regex Groups	Extracts the data from a regex group into its own column
Set Charset	Sets the encoding and then converts the data to a UTF-8 String
Set Record Delim	Sets the record delimiter
Split by Separator (Deprecated)	(Deprecated)
Split Email	Splits an email ID into an account and its domain
Split URL	Splits a URL into its constituents
Text Distance (Fuzzy String Match)	Measures the difference between two sequences of characters
Text Metric (Fuzzy String Match)	Measures the difference between two sequences of characters
URL Decode	Decodes from the `application/x-www-form-urlencoded` MIME format
URL Encode	Encodes to the `application/x-www-form-urlencoded` MIME format
Trimming Spaces	Functions for trimming white spaces around string data

Directive	Description
Decode	Decodes a column value as one of `base32`, `base64`, or `hex`
Encode	Encodes a column value as one of `base32`, `base64`, or `hex`

Directive	Description
Generate UUID	Generates a universally unique identifier (UUID)

Directive	Description
Diff Date	Calculates the difference between two dates
Format Date	Custom patterns for date-time formatting
Format UNIX Timestamp	Formats a UNIX timestamp as a date

Directive	Version Introduced	Description
Current Datetime	6.4.0	Generates current datetime with a given time zone
Datetime to Timestamp	6.4.0	Creates a timestamp from datetime and given time zone
Format Datetime	6.4.0	Formats datetime to a string of a given format
Timestamp to Datetime	6.4.0	Converts a timestamp to datetime

Directive	Description
Catalog Lookup	Static catalog lookup of ICD-9, ICD-10-2016, ICD-10-2017 codes
Table Lookup	Performs lookups into Table datasets

Directive	Description
Hash	Generates a message digest
Mask Number	Applies substitution masking on the column values
Mask Shuffle	Applies shuffle masking on the column values

Directive	Description
Filter Rows	Filters records based on a condition.
Filter Rows On	Deprecated. Filters records based on a condition.
Filter Row if Matched	Filters rows that match a pattern for a column
Filter Row if True (Deprecated)	Filters rows if the condition is true.
Filter Row Empty of Null	(Obsolete?) Filters rows that are empty of null.
Flatten	Separates the elements in a repeated field
Fail on condition	Fails processing when the condition is evaluated to true.
Send to Error	Filtering of records to an error collector
Send to Error and Continue Processing	Filtering of records to an error collector and continues processing
Split to Rows	Splits based on a separator into multiple records

Directive	Description
Change Column Case	Changes column names to either lowercase or uppercase
Changing Case	Changes the case of column values
Cleanse Column Names	Sanitizes column names, following specific rules
Columns Replace	Alters column names in bulk
Copy	Copies values from a source column into a destination column
Drop Column	Drops a column in a record
Fill Null or Empty	Fills column value with a fixed value if null or empty
Keep	Keeps specified columns from the record
Merge	Merges two columns by inserting a third column
Rename	Renames an existing column in the record
Set Headers	Sets the names of columns, in the order they are specified
Split to Columns	Splits a column based on a separator into multiple columns
Swap	Swaps column names of two columns
Set Type	Converts data type of a column

Directive	Description
Stemming Tokenized Words	Applies the Porter stemmer algorithm for English words

Directive	Description
Increment Variable	Increments a transient variable with a record of processing.
Set Variable	Sets a transient variable with a record of processing.