Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Wildcard support for GCS Copy/Move

Description

To add wildcard support for the source path.

 

For example gsutil provides functionality that can move all the files under test_*/ folders by:

gsutil mv gs://gcsmove/test_*/* gs://gcsmove-dest/

Release Notes

None

Activity

Show:

Ankit Jain September 26, 2024 at 8:23 AM

PR:

Sumeng Wang May 2, 2022 at 4:48 PM

Ankit Jain December 20, 2021 at 5:34 AM
Edited

Currently, the google storage api only supports passing prefix as an optional parameter. So, the workaround which I can imagine is to recursively parse the directories from the base path and match the regex.
For reference, Apache Beam does something similar, .

Avinash Achar December 8, 2021 at 5:35 AM

IHG is looking for this. Higher priority.

Sree Raman December 7, 2021 at 9:03 PM

File Move plugin does not support service account for the plugin operation. So if there is requirement to use a separate SA for the file move operation then File Move plugin does not statisfy that requirement.

Unresolved
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Labels

Reviewer

Priority

More fields

Created June 3, 2021 at 12:36 AM
Updated December 3, 2024 at 4:05 PM