Error Collector

Plugin version: 2.11.0

The Error Collector plugin takes errors emitted from the previous stage and flattens them by adding the error message, code, and stage to the record and outputting the result.

The plugin is used when you want to capture errors emitted from another stage and pass them along with all the error information flattened into the record. For example, you might want to connect a sink to this plugin in order to store and later examine the error records.

The Error Collector takes inputs from a previous stage that outputs errors, such as the Wrangler transformation. For more information, see Sending records to error.

Configuration

Property	Macro Enabled?	Description

Property	Macro Enabled?	Description
Error Message Column Name	No	Optional. The name of the error message field to use in the output schema. The UI will default this to ‘msg’. If no value is specified, the error message will be dropped.
Error Code Column Name	No	Optional. The name of the error code field to use in the output schema. The UI will default this to ‘code’. If no value is specified, the error code will be dropped.
Error Emitter Node Name	No	Optional. The name of the error stage field to use in the output schema. The UI will default this to ‘Stage’. If no value is specified, the error stage will be dropped.
Output Schema	No	Required. The output schema of the data.

Example

This example adds the error message, error code, and error stage as the ‘errMsg’, ‘errCode’, and ‘errStage’ fields.

Property	Value

Property	Value
Error Message Column Name	`errMsg`
Error Code Column Name	`errCode`
Error Emitter Node Name	`errStage`

For example, suppose the plugin receives this error record:

field name	type	value

field name	type	value
A	int	10
B	int	20

with error code 17, error message ‘invalid’, from stage ‘parser’. It will add the error information to the record and output:

field name	type	value

field name	type	value
A	int	10
B	int	20
errMsg	string	invalid
errCode	int	17
errStage	string	parser