Goal
This is a source plugin that would allow users to read and process mainframe files defined using COBOL Copybook. This should be basic first implementation.
Checklist
- User stories documented
- User stories reviewed
- Design documented
- Design reviewed
- Feature merged
-
Examples and guides - Integration tests
- Documentation for feature
- Short video demonstrating the feature
...
Input Format implementation : here
Design
- If "AbstractFieldValue"(JRecord) type is binary, the data will be encoded to Base64 format.
Integer.parseInt(new String(Base64.decodeBase64(Base64.encodeBase64(value.asString().getBytes()))), 2);
or
Base64.decodeInteger(Base64.encodeInteger(value.asBigInteger()));
...
| JRecord AbstractFieldValue type | JAVA primitive data type | Description | Comments |
|---|---|---|---|
char, char just right , char null terminated, char null padded | java.lang.String | ||
num left justified, num right justified , num zero padded | int | ||
| binary int, binary int positive, positive binary int fields | int | decode it using BASE64 format and then retrieve it. Integer.parseInt(new String(Base64.decodeBase64( Base64.encodeBase64(value.asString().getBytes()))), 2) | The Base64.decodeBase64() accepts either binary or String data, and therefore, first encoding and then decoding the values Decoding it directly results in improper values |
decimal, Mainframe Packed Decimal, Mainframe Packed Decimal, Mainframe Zoned Numeric Numeric, Mainframe Packed Decimal(+ve)
| java.math.BigDecimal | Since CDAP Schema.Type does not have a BigDecimal data type, converting everything to DOUBLE | |
Binary Integer Big Endian (Mainframe, AIX etc) -
| java.math.BigDecimal | decode it using BASE64 format and then retrieve it. Base64.decodeInteger(Base64.encodeInteger(filedValue.asBigInteger())) | The Base64.decodeBase64() accepts either binary or String data, and therefore, first encoding and then decoding the values Decoding it directly resulted in improper values Since CDAP Schema.Type does not have BigInteger converting this to LONG |
| num any decimal, positive num any decimal, assumed decimal | double | ||
| Hex | long | Long.parseLong(value.asHex(), 16) | Since CDAP Schema.Type does not have Hex converting this to LONG |
| Boolean / (Y/N) | java.lang.Boolean | ||
| Default | java.lang.String |
...
"binaryFilePath": "file:///home/cdap/DTAR020_FB.bin",
"maxSplitSize": "5"
}
}
}