Versions Compared
compared with
Key
- This line was added.
- This line was removed.
- Formatting was changed.
Introduction
An option of IAM role based authentication in the existing S3 source and sink plugins.
Use case(s)
- In the S3 source and S3 sink(Avro and Parquet) plugins,there should be a provision for user to select authentication mechanism for S3.User should have an option to select IAM role based authentication in the plugins.
User Storie(s)
- As a pipeline user,i want to have an option of IAM role based authentication in the S3 source and sink plugins in Hydrator.
Plugin Type
- Batch Source
- Batch Sink
- Real-time Source
- Real-time Sink
- Action
- Post-Run Action
- Aggregate
- Join
- Spark Model
- Spark Compute
Configurables
New Configuration would be added in the S3 plugin
| User Facing Name | Type | Description | Constraints |
|---|---|---|---|
| Authentication Method | Select | Authentication method to access S3. Defaults to Access Credentials | Since this is an EC2+IAM capability,user will need to have AWS environment with roles assigned to all the members. |
Design / Implementation Tips
- Tip #1
- Tip #2
Design
Approach(s)
Properties
Security
Limitation(s)
Future Work
- Some future work – HYDRATOR-99999
- Another future work – HYDRATOR-99999
Test Case(s)
- Test case #1
- Test case #2
Sample Pipeline
Please attach one or more sample pipeline(s) and associated data.
Pipeline #1
Pipeline #2
Table of Contents
Table of Contents style circle
Checklist
- User stories documented
- User stories reviewed
- Design documented
- Design reviewed
- Feature merged
- Examples and guides
- Integration tests
- Documentation for feature
- Short video demonstrating the feature