Directives
A collection for Directives available within Wrangler.
Parsers
Directive | Version Introduced | Description |
---|---|---|
| Uses a DSL (a JSON path expression) for parsing JSON records | |
| Parses an AVRO data file | |
| Parses an input record as comma-separated values | |
| Parses a currency value that is a string representation of locale currency into a number. | |
| Use Parse as Simple Date. | |
6.4.0 | Parses string as datetime with a given format. | |
| Parsing Excel file. | |
| Parses as a fixed length record with specified widths | |
| Parses Health Level 7 Version 2 (HL7 V2) messages | |
| Parses a JSON object | |
| Parses access log files as from Apache HTTPD and nginx servers | |
| Parses date strings | |
| Parses column values representing unix timestamp as date | |
| Parses an XML document into a JSON structure |
Output Formatters
Directive | Description |
---|---|
Converts a record into CSV format | |
Converts the record into a JSON map | |
Composes a JSON object based on the fields specified. | |
Formats a number as currency as specified by locale. |
Transformations
Directive | Description |
---|---|
Changes the case of column values | |
Creates a record column with nested values by copying values from source columns into a destination column | |
Selects parts of a string value | |
Sets the column value to the result of an expression execution | |
Transforms string column values using a "sed"-like expression | |
(Deprecated) | |
Invokes an HTTP Service (Experimental, potentially slow) | |
Quantizes a column based on specified ranges | |
Extracts the data from a regex group into its own column | |
Sets the encoding and then converts the data to a UTF-8 String | |
Sets the record delimiter | |
(Deprecated) | |
Splits an email ID into an account and its domain | |
Splits a URL into its constituents | |
Measures the difference between two sequences of characters | |
Measures the difference between two sequences of characters | |
Decodes from the | |
Encodes to the | |
Functions for trimming white spaces around string data |
Encoders and Decoders
Unique ID
Directive | Description |
---|---|
Generates a universally unique identifier (UUID) |
Date Transformations
Directive | Description |
---|---|
Calculates the difference between two dates | |
Custom patterns for date-time formatting | |
Formats a UNIX timestamp as a date |
Datetime Transformations
Directive | Version Introduced | Description |
---|---|---|
6.4.0 | Generates current datetime with a given time zone | |
6.4.0 | Creates a timestamp from datetime and given time zone | |
6.4.0 | Formats datetime to a string of a given format | |
6.4.0 | Converts a timestamp to datetime |
Lookups
Directive | Description |
---|---|
Static catalog lookup of ICD-9, ICD-10-2016, ICD-10-2017 codes | |
Performs lookups into Table datasets |
Hashing and Masking
Directive | Description |
---|---|
Generates a message digest | |
Applies substitution masking on the column values | |
Applies shuffle masking on the column values |
Row Operations
Directive | Description |
---|---|
Filters records based on a condition. | |
Deprecated. Filters records based on a condition. | |
Filters rows that match a pattern for a column | |
Filters rows if the condition is true. | |
(Obsolete?) Filters rows that are empty of null. | |
Separates the elements in a repeated field | |
Fails processing when the condition is evaluated to true. | |
Filtering of records to an error collector | |
Filtering of records to an error collector and continues processing | |
Splits based on a separator into multiple records |