An option of IAM role based authentication in the existing S3 source and sink plugins.
Use case(s)
In the S3 source and S3 sink(Avro and Parquet) plugins,there should be a provision for user to select authentication mechanism for S3.User should have an option to select IAM role based authentication in the plugins.
User Storie(s)
As a pipeline user,i want to have an option of IAM role based authentication in the S3 source and sink plugins in Hydrator.
Plugin Type
Batch Source
Batch Sink
Real-time Source
Real-time Sink
Action
Post-Run Action
Aggregate
Join
Spark Model
Spark Compute
Configurables
New Configuration would be added in the S3 plugin
User Facing Name
Type
Description
Constraints
Authentication Method
Select
Authentication method to access S3.
Defaults to Access Credentials
Since this is an EC2+IAM capability,user will need to have AWS environment with roles assigned to all the members.
Design / Implementation Tips
Tip #1
Tip #2
Design
Approach(s)
Properties
Security
Limitation(s)
Future Work
Some future work – HYDRATOR-99999
Another future work – HYDRATOR-99999
Test Case(s)
Test case #1
Test case #2
Sample Pipeline
Please attach one or more sample pipeline(s) and associated data.