Google Cloud Storage Delete Action

Plugin version: 0.22.0

This plugin deletes objects in a Google Cloud Storage bucket. Cloud Storage allows world-wide storage and retrieval of any amount of data at any time.

Buckets are the basic containers that hold your data. Everything that is stored in Cloud Storage must be contained in a bucket. Buckets are used to organize and control access to data.

Objects are the individual pieces of data that are stored in Cloud Storage. Object names can contain any combination of Unicode characters (UTF-8 encoded) and must be less than 1024 bytes in length. Object names often look like file paths.

Credentials

If the plugin is run on a Google Cloud Dataproc cluster, the service account key does not need to be provided and can be set to 'auto-detect'. Credentials will be automatically read from the cluster environment.

If the plugin is not run on a Dataproc cluster, the path to a service account key must be provided. The service account key can be found on the Dashboard in the Cloud Platform Console. Make sure the account key has permission to access BigQuery and Google Cloud Storage. The service account key file needs to be available on every node in your cluster and must be readable by all users running the job.

Configuration

Property	Maco Enabled?	Description

Property	Maco Enabled?	Description
Project ID	Yes	Optional. The Google Cloud Project ID, which uniquely identifies a project. It can be found on the Dashboard in the Google Cloud Platform Console. Default is auto-detect.
Objects to Delete	Yes	Required. Comma separated list of objects to delete. Valid formats: Exact object. To delete the object, use the exact path. For example, `gs://MY_BUCKET/file1.csv`, `gs://MY_BUCKET/file2.csv` Wildcard character. Use an `` to match any number of characters within the current directory level. For example, `gs://MY_BUCKET/.csv` deletes all `.csv` files and directories ending with `.csv` under the `MY_BUCKET` directory. It deletes `gs://MY_BUCKET/file1.csv` and `gs://MY_BUCKET/file2.csv` but not `gs://MY_BUCKET/folder1/file3.csv`. You can also use wildcards to delete files and folders in multiple buckets. You must enter the exact path and bucket name. For example, `gs://MY_BUCKET1/.csv, gs://MY_BUCKET2/.csv` deletes all `.csv` files in `MYBUCKET1` and `MYBUCKET2`. You cannot use wildcards in the bucket name.
Service Account Type	Yes	Optional. Select one of the following options: File Path. File path where the service account is located. JSON. JSON content of the service account.
Service Account File Path	Yes	Optional. Path on the local file system of the service account key used for authorization. Can be set to 'auto-detect' when running on a Dataproc cluster. When running on other clusters, the file must be present on every node in the cluster. Default is auto-detect.
Service Account JSON	Yes	Optional. Content of the service account.