CDAP Release 6.7.2

Owned by robin
Last updated: Feb 10, 2023
2 min read

Release Date: October 7, 2022

Enhancements

CDAP-19601: For new Dataproc compute profiles, changed the default value of Master Machine Type and Worker Machine Type from n2 to e2.

Bug Fixes

CDAP-19532: Fixed an issue in the Database Batch Source plugin that caused pipelines to fail during runtime when there was a column with precision of 0 in the source returned by JDBC. Now, if a column has a precision of 0, the pipeline no longer fails. This affected CDAP 6.7.1 only. Note: In the Database Batch Source, if a column has precision 0, you must change the data type to Double in the Output Schema to ensure the pipeline runs successfully.

PLUGIN-1373: In the BigQuery Sink plugin (version 0.20.3), fixed an issue that sometimes caused a NullPointerException error when trying to update table metrics.

PLUGIN-1367: In the BigQuery Sink plugin (version 0.20.3), fixed an issue that caused a NullPointerException error when the output schema was not defined. 

PLUGIN-1361: In the Send Email batch pipeline alert, fixed an issue where emails failed to send when the Protocol was set to TLS.

PLUGIN-1072: In the Google Cloud Storage Sink and File Sink, fixed an issue that caused Decimal values for JSON and Delimited/CSV/TSV formats to be formatted incorrectly.

Known Issues

Replication

CDAP-19622: Upgrade for MySQL and SQL Server replication jobs is broken. Upgrading MySQL and SQL Server replication jobs from CDAP 6.7.1 to CDAP 6.7.2 isn’t recommended.

CDAP-20013: Upgrade for Oracle by Datastream replication jobs is broken. Upgrading Oracle by Datastream replication jobs from CDAP 6.6.0 and 6.7.1 to CDAP 6.7.2 isn’t recommended.

Secure Macros

CDAP-20271: Pipelines fail when they use a connection that includes a secure macro and the secure macro has JSON as the value (for example, the Service Account property).

Workarounds

Use one of the following workarounds:

(1) For existing, running pipelines, create a new secure key for the connection and escape all the quotes in the secure macro JSON. Then edit the connection to use the new secure key JSON. Note: If you use this approach, browsing and sampling in Wrangler, and other places that directly use the secure macro will start to fail.

(2) Do not use the connection with the secure macro when you run the pipeline. 

To remove the connection, follow these steps:

  1. Duplicate the deployed pipeline.

  2. In the Pipeline Studio, for each plugin that uses the connection, turn off the connection and then edit the Service Account property to include the secure macro. 

Created in 2020 by Google Inc.