Google Firestore Sink

The Google Firestore sink is available for Preview in the Hub.

Plugin version: 1.0.0

This sink writes data to Google Cloud Firestore.

Credentials

If the plugin is run on a Google Cloud Dataproc cluster, the service account key does not need to be provided and can be set to auto-detect. Credentials will be automatically read from the cluster environment.

If the plugin is not run on a Dataproc cluster, the path to a service account key must be provided. The service account key can be found on the Dashboard in the Cloud Platform Console. Make sure the account key has permission to access Google Cloud Firestore. The service account key file needs to be available on every node in your cluster and must be readable by all users running the job.

Configuration

Property	Macro Enabled?	Description
Reference Name	No	Required. Name used to uniquely identify this sink for lineage, annotating metadata, etc.
Collection	Yes	Required. Name of the collection to write data. If the collection name does not exist in Firestore then new collection will be created first and then data written to it.
Document Id Type	Yes	Required. Type of ID(Key) assigned to documents written to the Cloud Firestore. The type can be one of two values: Auto-generated id. ID(Key) will be generated by Cloud Firestore as a Alpha Numeric ID. Custom name. ID(Key) will be provided as a field from the input records. The id field must not be nullable and must be of type STRING.
Document Id Alias	Yes	Optional. The field that will be used as the document id when writing to Cloud Firestore. This must be provided when the Document Id Type is not auto generated.
Service Account File Path	Yes	Optional. Path on the local file system of the service account key used for authorization. Can be set to `auto-detect` when running on a Dataproc cluster. When running on other clusters, the file must be present on every node in the cluster. Default is auto-detect.
Project ID	Yes	Optional. Google Cloud Project ID, which uniquely identifies a project. It can be found on the Dashboard in the Google Cloud Platform Console. Default is auto-detect.
Batch Size	Yes	Required. Maximum number of documents that can be passed in one batch to a Commit operation. The minimum value is `1` and maximum value is `500`. Default is 25.
serviceAccountJSON	Yes	Optional. Content of the Service Account file.
serviceAccountType	Yes	Optional. Service Account type, file path where the service account is located or the JSON content of the service account.