Capabilities to trigger programs from Hydrator pipeline
Activity
Show:
Andreas Neumann November 16, 2016 at 3:48 PM
Another option is unify our Batch pipelines with our Workflow framework. That is a bigger effort, though.
Won't Fix
Pinned fields
Click on the next to a field label to start pinning.
Details
Details
Assignee
Terence Yim
Terence YimReporter
Sreevatsan Raman
Sreevatsan RamanComponents
Priority
MajorCreated November 16, 2016 at 11:42 AM
Updated June 9, 2020 at 1:26 AM
Resolved June 9, 2020 at 1:26 AM
Hydrator should have capabilities to start any CDAP program.
Use-Case:
Simple data transformations are orchestrated using Hydrator pipeline and users would like to perform further processing in spark written using CDAP applications.
https://groups.google.com/forum/#!topic/cdap-user/hpyg8Al5Y2M
In order to make this work with clusters that has perimeter security turned on we should use a triggering mechanism using Transactional messaging system.