Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Info

The Google Firestore sink is available for Preview in the Hub.

The Google Firestore sink was introduced in CDAP 6.3.0. Plugin version: 1.1.0

This sink writes data to Google Cloud Firestore.

...

If the plugin is not run on a Dataproc cluster, the path to a service account key must be provided. The service account key can be found on the Dashboard in the Cloud Platform Console. Make sure the account key has permission to access Google Cloud Firestore. The service account key file needs to be available on every node in your cluster and must be readable by all users running the job.

Configuration

Property

Macro Enabled?

Version Introduced

Description

Reference Name

No

Required. Name used to uniquely identify this sink for lineage, annotating metadata, etc.

Database Id

Yes

1.1.0

Optional. Firestore database name. If no value is provided, then (default) will be used.

Collection

Yes

Required. Name of the collection to write data. If the collection name does not exist in Firestore then new collection will be created first and then data written to it.

Document Id Type

Yes

Required. Type of ID(Key) assigned to documents written to the Cloud Firestore. The type can be one of two values:

  • Auto-generated id. ID(Key) will be generated by Cloud Firestore as a Alpha Numeric ID.

  • Custom name. ID(Key) will be provided as a field from the input records. The id field must not be nullable and must be of type STRING.

Document Id Alias

Yes

Optional. The field that will be used as the document id when writing to Cloud Firestore. This must be provided when the Document Id Type is not auto generated.

Service AccountType

Yes

Required. Service Account type, file path where the service account is located or the JSON content of the service account.

Service Account File Path

Yes

Optional. Path on the local file system of the service account key used for authorization. Can be set to auto-detect when running on a Dataproc cluster. When running on other clusters, the file must be present on every node in the cluster.

Default is auto-detect.

Service Account JSON

Yes

Optional. Content of the Service Account file.

Project ID

Yes

Optional. Google Cloud Project ID, which uniquely identifies a project. It can be found on the Dashboard in the Google Cloud Platform Console.

Default is auto-detect.

Batch Size

Yes

Required. Maximum number of documents that can be passed in one batch to a Commit operation. The minimum value is 1 and maximum value is 500.

Default is 25

.

serviceAccountJSON

Yes

Optional. Content of the Service Account file.

serviceAccountType

Yes

Optional. Service Account type, file path where the service account is located or the JSON content of the service account

.