MariaDB Sink
The MariaDB Batch source plugin is available in the Hub.
Writes records to a MariaDB table. Each record will be written to a row in the table.
Use this sink when you need to write to a MariaDB table. Suppose you periodically build a recommendation model for products on your online store. The model is stored in a FileSet and you want to export the contents of the FileSet to a MariaDB table where it can be served to your users.
Column names would be autodetected from input schema.
Configuration
Property | Macro Enabled? | Description |
|---|---|---|
Reference Name | No | Required. This will be used to uniquely identify this sink for lineage, annotating metadata, etc. |
Host | Yes | Required. Host that MariaDB is running on. Default is localhost. |
Port | Yes | Required. Port that MariaDB is running on. Default is 3306. |
Database | Yes | Required. MariaDB database name. |
Table Name | Yes | Required. Name of the table to export to. |
Username | Yes | Optional. User identity for connecting to the specified database. |
Password | Yes | Optional. Password to use to connect to the specified database. |
Use SSL | No | Optional. Turns on SSL encryption. The connection will fail if SSL is not available. Default is |
Keystore URL | No | Optional. URL to the client certificate KeyStore (if not specified, use defaults). Must be accessible at the same location on host where CDAP Master is running and all hosts on which at least one HDFS, MapReduce, or YARN daemon role is running. |
Keystore Password | No | Optional. Password for the client certificates KeyStore. |
Truststore URL | No | Optional. URL to the trusted root certificate KeyStore (if not specified, use defaults). Must be accessible at the same location on host where CDAP Master is running and all hosts on which at least one HDFS, MapReduce, or YARN daemon role is running. |
Truststore Password | No | Optional. Password for the trusted root certificates KeyStore. |
Driver Name | No | Optional. Name of the JDBC driver to use. Default is mariadb. |
Use Compression | No | Optional. Use zlib compression when communicating with the server. Select this option for WAN connections. Default is No. |
Connection Arguments | No | Optional. A list of arbitrary string key/value pairs as connection arguments. These arguments will be passed to the JDBC driver as connection arguments for JDBC drivers that may need additional configurations. |
Auto Reconnect | No | Optional. Should the driver try to re-establish stale and/or dead connections. Default is No. |
Output Schema | No | The schema of records output by the source. This will be used in place of whatever schema comes back from the query. However, it must match the schema that comes back from the query, except it can mark fields as nullable and can contain a subset of the fields. |
Data Type Mapping
MariaDB Data Type | CDAP Schema Data Type | Comments |
|---|---|---|
TINYINT | int |
|
BOOLEAN, BOOL | boolean |
|
SMALLINT | int |
|
MEDIUMINT | int |
|
INT, INTEGER | int |
|
BIGINT | long |
|
DECIMAL, DEC, NUMERIC, FIXED | decimal |
|
FLOAT | float |
|
DOUBLE, DOUBLE PRECISION, REAL | decimal |
|
BIT | boolean |
|
CHAR | string |
|
VARCHAR | string |
|
BINARY | bytes |
|
CHAR BYTE | bytes |
|
VARBINARY | bytes |
|
TINYBLOB | bytes |
|
BLOB | bytes |
|
MEDIUMBLOB | bytes |
|
LONGBLOB | bytes |
|
TINYTEXT | string |
|
TEXT | string |
|
MEDIUMTEXT | string |
|
LONGTEXT | string |
|
JSON | string | In MariaDB it is alias to LONGTEXT |
ENUM | string | Mapping to String by default |
SET | string |
|
DATE | date |
|
TIME | time_micros |
|
DATETIME | timestamp_micros |
|
TIMESTAMP | timestamp_micros |
|
YEAR | date |
|
Example
Suppose you want to write output records to “users” table of MariaDB database named “prod” that is running on “localhost”, port 3306, as “root” user with “root” password, then configure the plugin with:
Property | Value |
|---|---|
Reference Name | snk1 |
Driver Name | mariadb |
Host | localhost |
Port | 3306 |
Database | prod |
Table Name | users |
Username | root |
Password | root |
Created in 2020 by Google Inc.