GCS Copy/Move Plugins not creating buckets at the destination path if not present
Description
Release Notes
None
Activity
Show:
Unresolved
Pinned fields
Click on the next to a field label to start pinning.
Created January 23, 2024 at 6:06 PM
Updated January 31, 2024 at 12:51 PM
The GCS Copy/Move Plugins are expected to create a bucket at the Destination Path provided in the plugin property if the bucket does not exist. The method implemented for this purpose does not exhibit expected behaviour as it throws a Runtime Exception if the bucket is not found and hence never attempts to create the bucket.
public void createBucketIfNotExists(GCSPath path, @Nullable String location, @Nullable CryptoKeyName cmekKeyName) { Bucket bucket = null; try { bucket = storage.get(path.getBucket()); } catch (StorageException e) { throw new RuntimeException( String.format("Unable to access bucket %s. ", path.getBucket()) + "Ensure you entered the correct bucket path and have permissions for it.", e); } if (bucket == null) { GCPUtils.createBucket(storage, path.getBucket(), location, cmekKeyName); } }
Link to the code: https://github.com/data-integrations/google-cloud/blob/645fd8056e2bd1e2fb5823725ef988318a902b27/src/main/java/io/cdap/plugin/gcp/gcs/StorageClient.java#L117-129