...
This would be helpful for user to access data in terms of readability.
Source: We are considering the source as CSV file, For Example:
Source |
| Output from Source | Type | Value |
S3 | Id | String | 1234 | |
Path: Path on S3 |
| Name | String | John |
CSV File with fields: |
| Salary | INT | 9000 |
id, name,salary,designation |
| Designation | INTString | 2 |
ValueMapper Plugin : For this input will be the StructuredRecord from source and Mapping/Lookup Dataset.
The transform function in this plugin will apply the mapping on the source fields using Lookup Interface.
...
Id | String |
Name | String |
Salary | INT |
Designation | INTString |
Sample structure for Mapping/Lookup Dataset
Designation | Value |
1 | SE |
2 | SSE |
3 | ML |
Sink : After the transformations from ValueMapper plugin, output will have below structure:
FieldName | Type | Value |
Id | String | 1234 |
Name | String | John |
Salary | INT | 9000 |
Designation | String | SSE |
...