Introduction

MarkLogic Server is a powerful software solution for harnessing your digital content all in a single database. MarkLogic enables you to build complex applications that interact with large volumes of JSON, XML, SGML, HTML, RDF triples, binary files, and other popular content formats. The unique architecture of MarkLogic ensures that your applications are both scalable and high-performance, delivering query results at search-engine speeds while providing transactional integrity over the underlying database. These plugins will allow you to integrate data in Marklogic with the rest of your data using CDAP.

User Storie(s)

As a pipeline developer, I would like to read data in Marklogic in batch using CDAP, so that I can integrate it easily with the rest of my data.
As a pipeline developer, I would like to write complex structures (XML, JSON, SGML, HTML, RDF triples, binary data, etc) to Marklogic in batch using CDAP, so that I do not have to develop custom code to load my data into Marklogic, and take advantage of the standardization that CDAP offers.
As a pipeline developer, I would like CDAP to support ELT in Marklogic, so that I can take advantage of Marklogic's powerful search and analytics features after loading the data, while still maintaining standardization and lineage in CDAP

Plugin Type

Batch Source
Batch Sink
Real-time Source
Real-time Sink
Action
Post-Run Action
Aggregate
Join
Spark Model
Spark Compute

Configurables

Marklogic batch source.

Category	User Facing Name	Type	Description	Constraints
Basic	Host	text	The host running the Marklogic REST Server	Should validate URL
	Port	number	The port that the Marklogic REST Server listens on
	User	text	The user to perform operations as. The user should have appropriate read privileges/
	Password	password	The password for the user
	Authentication Type	radio button	The type of authentication to use - Digest or
	Connection Type	radio buton	The type of connection to use - Direct or Gateway

Marklogic batch sink.

Category	User Facing Name	Type	Description	Constraints
Basic	Host	text	The host running the Marklogic REST Server	Should validate URL
	Port	number	The port that the Marklogic REST Server listens on
	User	text	The user to perform operations as. The user should have appropriate read privileges/
	Password	password	The password for the user
	Authentication Type	radio button	The type of authentication to use - Digest or
	Connection Type	radio buton	The type of connection to use - Direct or Gateway
Advanced	Batch size	number	The batch size for writing to Marklogic
Advanced	Max retries	number	The maximum retries for requests to marklogic

Marklogic query executor action.

Category	User Facing Name	Type	Description	Constraints
Basic	Host	text	The host running the Marklogic REST Server	Should validate URL
	Port	number	The port that the Marklogic REST Server listens on
	User	text	The user to perform operations as. The user should have appropriate read privileges/
	Password	password	The password for the user
	Authentication Type	radio button	The type of authentication to use - Digest or
	Connection Type	radio buton	The type of connection to use - Direct or Gateway
	Query	textarea	The query to execute in Marklogic

Design / Implementation Tips

Use Marklogic Server docker image for testing
Marklogic Java Client
Tutorial
Marklogic Hadoop Connector

Design

Approach(s)

Properties

Security

Limitation(s)

Future Work

Some future work – HYDRATOR-99999
Another future work – HYDRATOR-99999

Test Case(s)

Test case #1
Test case #2

Sample Pipeline

Please attach one or more sample pipeline(s) and associated data.

Pipeline #1

Pipeline #2

Table of Contents

Checklist

User stories documented
User stories reviewed
Design documented
Design reviewed
Feature merged
Examples and guides
Integration tests
Documentation for feature
Short video demonstrating the feature

Marklogic plugins