Google Cloud BigQuery Execute Action

Project ID

Yes

Optional. Google Cloud Project ID, which uniquely identifies a project. It can be found on the Dashboard in the Google Cloud Platform Console. This is the project that the BigQuery SQL will run in.

Default is auto-detect.

SQL

Yes

Required. SQL command to execute.

Dialect

Yes

Required. Dialect of the SQL command. The value must be 'legacy' or 'standard'. If set to 'standard', the query will use BigQuery's standard SQL: https://cloud.google.com/bigquery/sql-reference/. If set to 'legacy', BigQuery's legacy SQL dialect will be used for this query.

Default is standard.

Mode

Yes

Required. Mode to execute the query in. The value must be 'batch' or 'interactive'. An interactive query is executed as soon as possible and counts towards the concurrent rate limit and the daily rate limit. A batch query is queued and started as soon as idle resources are available, usually within a few minutes. If the query hasn't started within 3 hours, its priority is changed to 'interactive'.

Default is Batch.

Use Cache

Yes

Required. Specifies whether to look for the result in the query cache. The query cache is a best-effort cache that will be flushed whenever tables in the query are modified.

Default is False.

Job Location

Yes

Required. Location of the job. It must match the location of the dataset specified in the query.

Default is US.

Store results in a BigQuery Table

No

0.19.1

Optional. Whether to store results in a BigQuery Table.

Dataset Project ID

Yes

6.7.0/0.20.0

Optional. Project ID of the dataset to store the query results in. This is only required if the dataset is not in the same project that the BigQuery job will run in. If no value is given, it will default to the configured Project ID. BigQuery Data Editor role on this project must be granted to the specified service account to write BigQuery data to this project.

Dataset Name

Yes

Optional. Dataset to store the query results in. If not specified, the results will not be stored.

Table Name

Yes

Optional. Table to store the query results in. If not specified, the results will not be stored.

Encryption Key Name

Yes

6.5.1/0.18.1

Optional. Used to encrypt data written to the dataset or table created by the plugin to store the query results. It is only applicable when users choose to store the query results in a BigQuery table. More information can be found here.

Row As Arguments

Yes

Required. Row as arguments. For example, if the query is select min(id) as min_id, max(id) as max_id from my_dataset.my_table, an arguments for min_id and max_id will be set based on the query results. Plugins further down the pipeline can then reference these values with macros ${min_id} and ${max_id}.

Default is False.

Service Account Type

Yes

Optional. Select one of the following options:

• File Path. File path where the service account is located.

• JSON. JSON content of the service account.

Service Account File Path

Yes

Optional. Path on the local file system of the service account key used for authorization. Can be set to 'auto-detect' when running on a Dataproc cluster. When running on other clusters, the file must be present on every node in the cluster.

Default is auto-detect.

Service Account JSON

Yes

Optional. Contents of the service account JSON file.