GCP Plugins need a way to store the Service Account details as a blob of text

Description

Using the GCP plugins as they exist today presents a chicken-and-egg problem for pipelines designed to use the Dataproc profile. The GCP plugins take a path as an input string and require that you have the Service Account JSON file on each node of the cluster, but the nodes don't exist before the job starts therefore the pipeline fails.

Need a way to store the service account as a macro so that it can be provided at runtime or added as part of the profile definition.

Release Notes

None

Activity

Show:
Bhooshan Mogal
July 25, 2018, 6:21 PM

This should be really done via integration with secure store as well. I'm moving this to 5.2.0, because the immediate problem for Dataproc will be fixed by making these properties optional.

Duplicate

Assignee

Bhooshan Mogal

Reporter

Tony Hajdari

Labels

Docs Impact

None

UX Impact

None

Components

Fix versions

Affects versions

Priority

Critical
Configure