Release Date: August 11, 2022
Enhancements
CDAP-19050: Enhanced the Dataproc provisioner to avoid making unneeded Compute Engine calls depending on the configuration settings.
CDAP-18336: For new Dataproc compute profiles, changed the default value of Master Machine Type and Worker Machine Type from n1 to n2.
Bug Fixes
CDAP-19381: Fixed an issue in CDAP that created duplicate entries in file cache map, which resulted in multiple attempts to delete the same cache file.
Fixed an issue where the Log service left empty folders, which made the mounting of Persistent Disk slow. This caused the Log service to fail to start in a timely manner.
Fixed an issue that caused pipelines to take a long time to launch or get stuck. This was linked to I/O throttling that occurred on the underlying Persistent Disk.
CDAP-19366: Fixed an issue that caused pipelines to fail when two or more pipelines were scheduled to start simultaneously on a static Dataproc cluster. This was due to a file upload race condition.
CDAP-19353: Fixed an issue in flow control that caused Appfabric to return 5xx error code in rare scenarios instead of 429 (Too Many Requests Error) if the number of concurrently launching or running pipelines were above certain thresholds.
CDAP-19276: Fixed an issue that resulted in an error when a compute profile was exported from the default namespace after switching from a custom namespace.
CDAP-19216: Fixed an issue when you started a pipeline multiple times and then stopped the pipeline before it completed, which resulted in the following UI error: Program <program name> is not running.
CDAP-19211: Removed verbose logs from the BigQuery client libraries in pipeline logs.
PLUGIN-1256: Fixed an issue that caused the BigQuery Execute action plugin configured with an Encryption Key Name (CMEK) to fail when the SQL query contained DDL Statements.
PLUGIN-954: In the BigQuery Execute action plugin, added a property Store Results in a BigQuery Table in the UI, which hides the destination table related properties by default.
Known Issues
CDAP-19622: Upgrade for replication jobs is broken. We recommend not upgrading replication jobs to 6.7.1.