Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Current »

Release Date: November 2, 2021

Enhancements

PLUGIN-883, PLUGIN-897: Added Encryption Key Name property to the following plugins so users can encrypt any new resources created by these plugins with Customer Managed Encryption Keys (CMEK):

PLUGIN-898: Added Location property to GCS Copy and GCS Move action plugins to auto-create destination buckets if they do not exist before running the pipeline. Previously, the bucket had to exist before running the pipeline.

CDAP-18566: The File connection now browses the file system. For example, on a Hadoop cluster, the File connection now browses the HDFS file system. For CDAP Sandbox, the File connection still browses the local file system.

CDAP-18532: Added the following optional cdap-site.xml configs:

  • If a config router.block.request.enabled is true in conf, the request router should respond with a specific response (provided through config) to every user request, hence blocking all the user requests.

  • If a status code is provided using config router.block.request.status.code, the server should respond with this status code, the default value should be 503.

  • If a response message is provided using config router.block.request.message, the server should respond with this response body; otherwise the response body should be empty.

CDAP-18384: Added metrics for authorization in CDAP.

Bug Fixes

CDAP-18571: Fixed an issue where messages couldn’t be retrieved for Kafka topics. This broke in 6.5.0 and is now fixed in 6.5.1.

CDAP-18538, CDAP-184254: Fixed an issue where you couldn’t create a profile for an existing Dataproc cluster.

CDAP-18529: Fixed an issue that caused pipelines to fail when Transformation Pushdown was enabled and used macros as properties.

CDAP-18446: Fixed an issue that caused long running programs, like Replication, to fail within the default Hadoop delegation token timeout. Now, these tokens get renewed so that the job keeps running.

CDAP-18439: Fixed an issue in Replication that caused the Configure button to result in an error when you clicked it.

CDAP-18428: Fixed an issue that caused pipelines to fail with an Access Denied error when the pipeline had BigQuery plugins or Transformation Pushdown configuration that included a Dataset Project ID that was in a different project than the specified Project ID:

The Access Denied error was due to missing permissions on the service account. 

To ensure pipelines with BigQuery or BigQuery Multi Table sinks and pipelines with Transformation Pushdown enabled run successfully, assign the following roles to the Project ID service account:

  • BigQuery Job User role to run jobs

  • GCE Storage Bucket Admin role to create a temporary bucket

If the dataset is not in the same project that the BigQuery job will run in, the Dataset Project ID service account must be granted the following role to write data to a BigQuery dataset or table:

  • BigQuery Data Editor role

CDAP-18423: Fixed an issue in the GCS connection that prevented browsing and parsing files stored in folders under buckets.

CDAP-18335: Fixed an issue where the UI was unusable until an error displayed in the UI was closed by clicking the x icon.

CDAP-18318: Fixed an issue where users did not need permission to restart system services, reset system service log levels, get system service statuses, etc. Now, if authorization is enabled on the cluster, users will need to have the corresponding permissions for these system services in order to access them.

CDAP-18249: Fixed an issue where the Upload window didn’t close after uploading a user-defined directive due to missing properties in the user-defined directive json.

PLUGIN-899: Fixed an issue that caused custom formats to be unusable in the GCS source and sink.


  • No labels

Created in 2020 by Google Inc.