MongoDB Sink

The MongoDB sink plugin is available in the Hub.

Plugin version: 2.0.1

This sink writes to a MongoDB collection.

Configuration

Property

Macro Enabled?

Description

Property

Macro Enabled?

Description

Reference Name

No

Required. Name used to uniquely identify this sink for lineage, annotating metadata, etc.

Host

Yes

Required. Host that MongoDB is running on. Enter the hostname only. The connection string is constructed by the plugin.

Port

Yes

Required. Port that MongoDB is listening to.

Default is 27017.

Database

Yes

Required. MongoDB database name.

Collection

Yes

Required. Name of the database collection to write to.

ID Field

Yes

Optional. Which input field should be used as the document identifier. Identifier is expected to be a 12-byte value or it’s 24-byte hexadecimal string representation. If none is given, an identifier will be automatically generated.

Username

Yes

Optional. User identity for connecting to the specified database.

Password

Yes

Optional. Password to use to connect to the specified database.

Connection Arguments

Yes

Optional. A list of arbitrary string key/value pairs as connection arguments. See Connection String Options for a full description of these arguments.

Data Type Mapping

CDAP Schema Data Type

MongoDB Data Type

Comments

CDAP Schema Data Type

MongoDB Data Type

Comments

boolean

Boolean

 

bytes

Binary data, ObjectId

12-byte value of the identifier field is mapped to ObjectId.

string

String, ObjectId

24-byte hexadecimal string value of the identifier field is mapped to ObjectId.

date

Date

 

double

Double

 

decimal

Decimal128

 

float

Double

 

int

32-bit integer

 

long

64-bit integer

 

time

String

Time string in the following format: HH:mm:ss.SSS

timestamp

Date

 

array

Array

 

record

Object

 

enum

String

 

map

Object

 

union

 

Depends on the actual value. For example, if it's a union ["string","int","long"] and the value is actually a long, the MongoDB document will have the field as a 64-bit integer. If a different record comes in with the value as a string, the MongoDB document will end up with a String for that field.

Created in 2020 by Google Inc.