CDAP Release 6.10.1
Release date: May 21, 2024
Changes
CDAP-21006: You can add a provider using OAuthHandler while reusing credentials stored in Google Cloud Secret Manager.
CDAP-20934: Added support for option string field (keep-strings) in parse-xml-to-json
Wrangler directive.
PLUGIN-900: The BigQuery sink plugin doesn’t provide the Dedupe By option while in insert mode.
PLUGIN-1563: The BigQuery plugin supports the JSON type.
PLUGIN-1715: Improved retries in BigQuery plugin.
PLUGIN-1748: Improved error messages in the Spanner source.
PLUGIN-1769: Improved retries in PubSub plugin.
Fixes
PLUGIN-1736: Fixed an issue in Wrangler causing the send-to-error-and-continue
directive to not initialize dq_failure
when the condition is false.
CDAP-20951: Fixed an issue that occurs if running a replication pipeline when task workers are enabled.
PLUGIN-788, PLUGIN-781, PLUGIN-1318, PLUGIN-782: Improved error reporting in the BigQuery Sink. Fixed an issue in BigQuery Argument Setter where validation error wasn’t displayed correctly.
PLUGIN-1617: Fixed an issue with the Python plugin, where running in native mode doesn’t work as intended.
PLUGIN-1728: Fixed an issue causing certain connection parameters to not propagate in a MySQL connection.
PLUGIN-1735: Fixed an issue causing the Cloud Storage Copy action to timeout while working with large files.
PLUGIN-1738: Fixed an issue causing Copy and Move plugins to not create buckets at the destination path as expected, resulting in a runtime error.Â
PLUGIN-1742: Fixed an issue causing empty source input to fail in multiple plugins.
PLUGIN-1778: Fixed an issue with remote execution of Wrangler directives causing type information to not be emitted.
PLUGIN-1771: Streaming pipelines in CDAP support the Excel source. Batch pipelines with an Excel source can consume high memory and fail in large pipelines.
CDAP-21024: Fixed an issue causing a No record field provided
error.
CDAP-20890: Fixed an issue with using the Conditional plugin as a source for Wrangler, causing CDAP not to fetch the necessary schema.
CDAP-20999: Fixed an issue with instance upgrades causing existing schedule names to be improperly encoded in the URL, resulting in pre-upgrade failure.
CDAP-20988: Fixed an issue with schedules causing the maximum concurrent run property to not work as intended.
CDAP-20932: Fixed an issue causing committed ID to incorrectly propagate when pushing pipeline configurations to Git.
Breaking
CDAP version 6.10.1 has a known issue in the Cloud Storage plugin causing pipelines to intermittently fail if the plugin contains a * regex pattern and uses Dataproc 2.0. To mitigate this issue:
Use an older plugin version.
Created in 2020 by Google Inc.