Introduction

Amazon Aurora is a Mysql and Postgres compatible database offered as a service. Users will have needs to write to AuroraDB or read from AuroraDB

Use case(s)

• Users would like to batch build a data pipeline to read complete table from Amazon Aurora DB instance and write to BigTable. 
• Users would like to batch build a data pipeline to perform upserts on AuroraDB tables in batch 
• Users should get relevant information from the tool tip while configuring the AuroraDB source and AuroraDB sink
  • The tool tip for the connection string should be customized specific to the database. 
  • The tool tip should describe accurately what each field is used for
• Users should get field level lineage for the source and sink that is being used
• Reference documentation be available from the source and sink plugins

Deliverables 

Relevant links 

• Data-integrations org: https://github.com/data-integrations/
• Field level lineage: https://docs.cdap.io/cdap/5.1.0-SNAPSHOT/en/developer-manual/metadata/field-lineage.html
• Integration test repos: https://github.com/caskdata/cdap-integration-tests

Plugin Type

• Batch Source
• Batch Sink 
• Real-time Source
• Real-time Sink
• Action
• Post-Run Action
• Aggregate
• Join
• Spark Model
• Spark Compute

Configurables

This section defines properties that are configurable for this plugin. 

Design / Implementation Tips

• Tip #1
• Tip #2

Design

Approach(s)

Properties

Security

Limitation(s)

Future Work

• Some future work – HYDRATOR-99999
• Another future work – HYDRATOR-99999

Test Case(s)

• Test case #1
• Test case #2

Sample Pipeline

Please attach one or more sample pipeline(s) and associated data. 

Pipeline #1

Pipeline #2