Plugin version: 2.11.0
The Error Collector plugin takes errors emitted from the previous stage and flattens them by adding the error message, code, and stage to the record and outputting the result.
The plugin is used when you want to capture errors emitted from another stage and pass them along with all the error information flattened into the record. For example, you may want to connect a sink to this plugin in order to store and later examine the error records.
You might want to add an Error Collector after a Wrangler transformation to store error records. See Sending records to error.
Configuration
Property | Macro Enabled? | Description |
|---|---|---|
Error Message Column Name | No | Optional. The name of the error message field to use in the output schema. The UI will default this to ‘msg’. If no value is specified, the error message will be dropped. |
Error Code Column Name | No | Optional. The name of the error code field to use in the output schema. The UI will default this to ‘code’. If no value is specified, the error code will be dropped. |
Error Emitter Node Name | No | Optional. The name of the error stage field to use in the output schema. The UI will default this to ‘Stage’. If no value is specified, the error stage will be dropped. |
Output Schema | No | Required. The output schema of the data. |
Example
This example adds the error message, error code, and error stage as the ‘errMsg’, ‘errCode’, and ‘errStage’ fields.
Property | Value |
|---|---|
Error Message Column Name |
|
Error Code Column Name |
|
Error Emitter Node Name |
|
For example, suppose the plugin receives this error record:
field name | type | value |
|---|---|---|
A | int | 10 |
B | int | 20 |
with error code 17, error message ‘invalid’, from stage ‘parser’. It will add the error information to the record and output:
field name | type | value |
|---|---|---|
A | int | 10 |
B | int | 20 |
errMsg | string | invalid |
errCode | int | 17 |
errStage | string | parser |