CDAP Release 6.7.3

Owned by robin
Last updated: Mar 17, 2023
3 min read

Release Date: February 23, 2023

Changes

PLUGIN-1119: The Oracle Batch Source (version 1.8.5) reads Oracle NUMBER data type with undefined precision and scale as CDAP string. In plugin versions 1.8.3 and earlier, CDAP read Oracle NUMBER data type with undefined precision and scale as decimal (38,0), which could have resulted in data loss. For more information, see Oracle batch source plugin (versions 1.9.1 and 1.8.5) converts Oracle NUMBER data type with undefined precision and scale to CDAP string.

Bug Fixes

CDAP-19599: Fixed an issue in the BigQuery Replication Target plugin that caused replication jobs to fail when the BigQuery target table already existed. The new version of the plugin will automatically be used in new replication jobs. 

CDAP-19622: Fixed upgrade for MySQL and SQL Server replication jobs. You can now upgrade MySQL and SQL Server replication jobs from CDAP 6.7.1 and 6.7.2 to CDAP 6.7.3.

CDAP-20013: Fixed upgrade for Oracle by Datastream replication jobs. You can now upgrade Oracle by Datastream replication jobs from CDAP 6.6.0 and 6.7.x to CDAP 6.7.3 or higher.

CDAP-20235: For Database plugins (version 2.9.3), fixed a security issue where the database username and password were exposed in App Fabric logs.

CDAP-20271: Fixed an issue that caused pipelines to fail when they used a connection that included a secure macro and the secure macro had JSON as the value (for example, the Service Account JSON property).

CDAP-20392: Fixed an issue that occurred in certain upgrade scenarios, where pipelines that didn’t have the Use Connection property set, but the plugin the connection properties (such as Project ID and Service Account Type) were not displayed in the plugin UI. 

CDAP-20394: Fixed an issue where the replication source plugin's event reader was not stopped by the Delta worker in case of errors, leading to leakage of the plugin's resources. 

CDAP-20146: Fixed an issue in security-enabled instances that caused pipeline launches to fail and return a token expired error when evaluating secure macros in provisioner properties.

PLUGIN-1433: In the Oracle Batch Source (version 1.8.4 and earlier), when the source data included fields with the NUMBER data type with undefined precision and scale, CDAP read it as decimal and set the precision to 38 and the scale to 0. If any values in the field had scale other than 0, CDAP rounded these values, which could have resulted in data loss. If the scale for this field is overridden in the plugin output schema, the pipeline fails. 

In the Oracle batch Source (version 1.8.5), you can edit the scale of the CDAP decimal data type in the output schema, and the overridden value is used to map the Oracle NUMBER data type without failing the pipeline. If there are any Numbers present in the fields with the scale greater than the scale defined in the plugin, CDAP rounds the values based on the scale set by the user in the output schema.  For example, if you specify precision=10, scale = 3 and the value 123.4567 comes in, CDAP rounds to 123.457. For more information about setting precision and scale in a plugin, see Changing the precision and scale for decimal fields in the output schema.

PLUGIN-1374: Improved performance for batch pipelines with MySQL sinks.

Created in 2020 by Google Inc.