CDAP Release 6.10.1

Owned by Olivia Emery
May 28, 2024
2 min read

Release date: May 21, 2024

Changes

CDAP-21006: You can add a provider using OAuthHandler while reusing credentials stored in Google Cloud Secret Manager.

CDAP-20934: Added support for option string field (keep-strings) in parse-xml-to-json Wrangler directive.

PLUGIN-900: The BigQuery sink plugin doesn’t provide the Dedupe By option while in insert mode.

PLUGIN-1563: The BigQuery plugin supports the JSON type.

PLUGIN-1715: Improved retries in BigQuery plugin.

PLUGIN-1748: Improved error messages in the Spanner source.

PLUGIN-1769: Improved retries in PubSub plugin.

Fixes

PLUGIN-1736: Fixed an issue in Wrangler causing the send-to-error-and-continue directive to not initialize dq_failure when the condition is false.

CDAP-20951: Fixed an issue that occurs if running a replication pipeline when task workers are enabled.

PLUGIN-788, PLUGIN-781, PLUGIN-1318, PLUGIN-782: Improved error reporting in the BigQuery Sink. Fixed an issue in BigQuery Argument Setter where validation error wasn’t displayed correctly.

PLUGIN-1617: Fixed an issue with the Python plugin, where running in native mode doesn’t work as intended.

PLUGIN-1728: Fixed an issue causing certain connection parameters to not propagate in a MySQL connection.

PLUGIN-1735: Fixed an issue causing the Cloud Storage Copy action to timeout while working with large files.

PLUGIN-1738: Fixed an issue causing Copy and Move plugins to not create buckets at the destination path as expected, resulting in a runtime error. 

PLUGIN-1742: Fixed an issue causing empty source input to fail in multiple plugins.

PLUGIN-1778: Fixed an issue with remote execution of Wrangler directives causing type information to not be emitted.

PLUGIN-1771: Streaming pipelines in CDAP support the Excel source. Batch pipelines with an Excel source can consume high memory and fail in large pipelines.

CDAP-21024: Fixed an issue causing a No record field provided error.

CDAP-20890: Fixed an issue with using the Conditional plugin as a source for Wrangler, causing CDAP not to fetch the necessary schema.

CDAP-20999: Fixed an issue with instance upgrades causing existing schedule names to be improperly encoded in the URL, resulting in pre-upgrade failure.

CDAP-20988: Fixed an issue with schedules causing the maximum concurrent run property to not work as intended.

CDAP-20932: Fixed an issue causing committed ID to incorrectly propagate when pushing pipeline configurations to Git.

Breaking

CDAP version 6.10.1 has a known issue in the Cloud Storage plugin causing pipelines to intermittently fail if the plugin contains a * regex pattern and uses Dataproc 2.0. To mitigate this issue:

Created in 2020 by Google Inc.