...
Configuring the value mapper fields
Mapping
Source Field Name
Dataset
Target Field Name
Can support one or more such mappings
How the source field should be handled
If NULL, then user can provide a default value or NULL
If EMPTY, then user can provide a default value or EMPTY
Output Schema should allow
Remove Source Field
Include Target Field
Design
Examples
Suppose that user takes the input data( employee details) through the csv file or any other source and wants to perform some mappings on certain fields .
This would be helpful for user to access data in terms of readability.
Source: We are considering the source as CSV file, For Example:
Source |
| Output from Source | Type | Value |
S3 | Id | String | 1234 | |
Path: Path on S3 |
| Name | String | John |
CSV File with fields: |
| Salary | INT | 9000 |
id, name,salary,designation |
| Designation | INT | 2 |
ValueMapper Plugin : For this input will be the StructuredRecord from source and Mapping Dataset.
The transform function in this plugin will apply the mapping on the source fields using Lookup Interface.
StructuredRecord Format:
Id | String |
Name | String |
Salary | INT |
Designation | INT |
Sample structure for Mapping Dataset
Designation | Value |
1 | SE |
2 | SSE |
3 | ML |
Sink : After the transformations from ValueMapper plugin, output will have below structure:
FieldName | Type | Value |
Id | String | 1234 |
Name | String | John |
Salary | INT | 9000 |
Designation | String | SSE |