MongoDB Batch Source

Reference Name

No

Required. Name used to uniquely identify this source for lineage, annotating metadata, etc.

Host

Yes

Required. Host that MongoDB is running on. Enter the hostname only. The connection string is constructed by the plugin.

Port

Yes

Required. Port that MongoDB is listening to.

Default is 27017.

Database

Yes

Required. MongoDB database name.

Collection

Yes

Required. Name of the database collection to read from.

Input Query

Yes

Optional. Optionally filter the input collection with a query. This query must be represented in JSON format and use the MongoDB extended JSON format to represent non-native JSON data types.

Username

Yes

Optional. User identity for connecting to the specified database.

Password

Yes

Optional. Password to use to connect to the specified database.

Authentication Connection String

Yes

Optional. MongoDB connection string to connect to the ‘config’ database of a sharded cluster. It can be omitted if username and password do not differ from the previously provided ones or if ‘config’ database does not require authentication.

On Record Error

Yes

Required. Specifies how to handle error in record processing. An error will be thrown if failed to parse value according to a provided schema.

Default is Fail pipeline.

Connection Arguments

Yes

Optional. A list of arbitrary string key/value pairs as connection arguments. See Connection String Options for a full description of these arguments.

Boolean

boolean

Binary data

bytes

ObjectId

bytes

Double

double

Decimal128

decimal

32-bit integer

int

64-bit integer

long

String

string

Symbol

string

Date

timestamp

Array

array

Object

record, map

Map keys must be a non-nullable string.

Regular expression

Not supported.

DBPointer

Not supported.

JavaScript

Not supported.

JavaScript (with scope)

Not supported.

Timestamp

Not supported. Special type for internal MongoDB use.

Min key

Not supported.

Max key

Not supported.