Cloud Storage to Snowflake Action

The Cloud Storage to Snowflake action plugin is available in the Hub.

Plugin version: 1.1.0

Loads data from external or internal staged files to an existing Snowflake table.

The Cloud Storage to Snowflake action plugin works on top of COPY INTO. For more information, see the Snowflake docs.

Configuration

Basic

Property	Macro Enabled?	Description

Property	Macro Enabled?	Description
Account Name	Yes	Required. Snowflake account name and region and possibly a cloud specifier. Part of the URL you use to log in to Snowflake, minus the `snowflakecomputing.com`. For example, `myaccount.us-central1.gcp.`
Database	Yes	Required. Database that contains the target table.
Schema	Yes	Required. Schema that contains the target table.
Warehouse	Yes	Optional. Warehouse to use for the data load operation (i.e. compute resources).
Role	Yes	Optional. Role to use (e.g. `ACCOUNTADMIN`).
Source Type	Yes	Required. The source of the data to load. Possible values: `From Path`, `From Query`.
Source Stage / Location	Yes	Optional. Internal or external location where the files containing data are staged. Examples of internal locations formats: `@~[/{path}] @[{namespace}.]{int_stage_name}[/{path}] @[{namespace}.]%{table_name}[/{path}]` Examples of external locations formats: `@[{namespace}.]{ext_stage_name}[/{path}] s3://{bucket}[/{path}] gcs://{bucket}[/{path}] azure://{account}.blob.core.windows.net/{container}[/{path}]`
Source Query	Yes	Optional. An SQL query which specifies an explicit set of fields/columns (separated by commas) to load from the staged data files. The fields/columns are selected from the files using a standard SQL query. Example: `SELECT name, address, phone from s3://bucket/path`
Destination Table	Yes	Required. Snowflake table to load data into. Needs to be present.

Credentials

Property	Macro Enabled?	Description

Property	Macro Enabled?	Description
Username	Yes	Optional. Username to use to connect to your Snowflake account.
Password	Yes	Optional. Password to use to connect to your Snowflake account. Not necessary for key pair or OAuth2 authentication.

Key Pair Authentication

Property	Macro Enabled?	Description

Property

Macro Enabled?

Description

Key Pair Authentication Enabled

No

Optional. If true, the plugin will perform key pair authentication.

Default is False.

Private Key

Yes

Optional. Private key contents.

Key File Passphrase

Yes

Optional. Passphrase for the private key file.

OAuth2

To use OAuth2, you must create a snowflake security integration for it. For more information, see Introduction to OAuth in Snowflake.

Property	Macro Enabled?	Description

Property	Macro Enabled?	Description
OAuth2 Enabled	No	Optional. If true, the plugin will perform OAuth2 authentication. Default is False.
Client ID	Yes	Optional. Client ID obtained via system function SYSTEM$SHOW_OAUTH_CLIENT_SECRETS.
Client Secret	Yes	Optional. Client secret obtained via system function SYSTEM$SHOW_OAUTH_CLIENT_SECRETS.
Refresh Token	Yes	Optional. Token used to receive accessToken, which is end product of OAuth2. Must be generated by user.

Cloud Provider Parameters

More information on this section can be found on the official site.

Property	Macro Enabled?	Description

Property	Macro Enabled?	Description
Use Cloud Provider Parameters	Yes	If true, plugin will use cloud provider parameters to authenticate to source path. Required only for loading from an external private/protected cloud storage location. Not required for public buckets/containers. Default is False.
Cloud Provider		Optional. Cloud provider name. Possible values: `GCP`, `AWS`, `Microsoft Azure`. Default is NONE.
Storage Integration		Specifies the name of the storage integration used to delegate authentication responsibility for external cloud storage to a Snowflake identity and access management (IAM) entity. For more details, see CREATE STORAGE INTEGRATION. Note. We highly recommend the use of storage integrations. This avoids the need to supply cloud storage when creating stages or loading data.
Files Encrypted	Yes	If true, plugin will perform loading from encrypted files. Default is False.
Encryption type	Yes	Encryption of the files. Possible values: `AWS_CSE:` Client-side encryption (requires a `Master Key` value). Currently, the client-side master key you provide can only be a symmetric key. Note that, when a `Master Key` value is provided, Snowflake assumes TYPE = AWS_CSE (i.e. when a MASTER_KEY value is provided, TYPE is not required). `AWS_SSE_S3:` Server-side encryption that requires no additional encryption settings. `AWS_SSE_KMS:` Server-side encryption that accepts an optional `KMS Key Id` parameter. `AZURE_CSE:` Client-side encryption; requires a `Master Key` value. `GCS_SSE_KMS:` Server-side encryption that accepts an optional `KMS Key Id` parameter.
Master Key	Yes	Client-side master key that Snowflake uses to encrypt the files containing the unloaded data. The master key must be a 128-bit or 256-bit key in Base64-encoded form. Required by `AWS_CSE` and `AZURE_CSE` encryption types.
KMS Key Id	Yes	ID for the KMS-managed key that is used to encrypt files unloaded into the bucket. If no value is provided, your default KMS key ID is used to encrypt files on unload. The value is optionally and used by the following encryption types: `AWS_SSE_KMS`, `GCS_SSE_KMS`.
Amazon S3 specific properties
Key Id	Yes	Amazon key ID.
Secret Key	Yes	Amazon secret key.
Token	Yes	Amazon token.
Microsoft Azure specific properties
SAS Token	Yes	SAS (shared access signature) token for connecting to Azure and accessing the private/protected container where the files containing data are staged. Credentials are generated by Azure.

File Format

Property	Macro Enabled?	Description

Property	Macro Enabled?	Description
File Format Filtering Policy	Yes	Policy used to filter source files. Possible value: `Undefined` Do not filter the files. `By File Type` Filter using `Format Type` configuration. `By Existing Format Specification` Filter by existing format saved in Snowflake.
Format Name	Yes	Existing named file format to use for loading data into the table. The named file format determines the format type (CSV, JSON, etc.), as well as any other format options, for the data files. For more information, see CREATE FILE FORMAT.
Format Type	Yes	Type of files to load into the table. Supported values: CSV, JSON, AVRO, ORC, PARQUET, XML.
Format Type Options	Yes	Depending on the file format type specified (`Format Type` property), you can include one or more of the format-specific options. Find the options list here.

Advanced

Property	Macro Enabled?	Description

Property	Macro Enabled?	Description
Files	Yes	Optional. List of one or more files names (separated by commas) to be loaded. The files must already have been staged in either the Snowflake internal location or external location specified in the command. The maximum number of files names that can be specified is 1000. Note. For external stages only (Amazon S3, Google Cloud Storage, or Microsoft Azure), the file path is set by concatenating the URL in the stage definition and the list of resolved file names. However, Snowflake doesn’t insert a separator implicitly between the path and file names. You must explicitly include a separator (/) either at the end of the URL in the stage definition or at the beginning of each file name specified in this parameter.
Pattern	Yes	Optional. Regular expression pattern string, enclosed in single quotes, specifying the file names and/or paths to match. Tip. For the best performance, try to avoid applying patterns that filter on a large number of files.
Copy Options	Yes	Optional. List of copy options. Find them here.
Connection Argument	Yes	Optional. List of arbitrary string tag/value pairs as connection arguments. See JDBC Driver Connection String.