Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

FieldName

Type

Value

Id

String

1234

Name

String

John

Salary

INT

9000

Designation

String

SSE


Properties:

  • mapperField: name of the field that contains the input column name to be mapped. This field should be removed from the output schema.

  • mappingField: name of the field that contains the output column name that will be mapped. This field should be included in output schema.
  • lookup: The configuration of the lookup tables to be used in your transform as mapping dataset. For example, if lookup table "designation" is configured, then you will be able to perform operations with that lookup table in your transform. Currently supports KeyValueTable.

Example:
         {
             "name": "ValueMapper",
             "type": "transform",
             "properties": {
                     "mapperField" : "designationID",
                     "mappingField" : "designationName",
                     "lookup": "{
                              \"tables\":{
                                           \"designation\":{
                                                     \"type\":\"DATASET\",
                                                     \"datasetProperties\":{
                                                               \"dataset_argument1\":\"foo\",
                                                               \"dataset_argument2\":\"bar\"
                                                     }
                                            }
                              }
                    }"
             }
        }

 

The transform takes record that have "id,name,salary,designationID" fields, maps the value of designationID field to generate designationName field based on the mapping dataset and then returns a record containing "id,name,salary,designationName" fields.

For example, if the input is like:

field nametype value 
id string US1
name string samuel
salary string 1000
designationID string D3

it will transform it to this output record:

field nametype value 
id string US1
name string samuel
salary string 1000
designationName string Module Lead