Goals
Ability to specify condition for execution-path in the hydrator pipelines.
Checklist
- User stories documented (Shankar/Sagar)
- User stories reviewed (Nitin)
- Design documented (Shankar/Sagar)
- Design reviewed (Terence/Andreas)
- Feature merged ()
- Examples and guides ()
- Integration tests ()
- Documentation for feature ()
- Blog post
Use-case
Use has a multi job pipeline, set of transforms run as a part of MR job doing data cleansing and a spark ML job on training data to output model. wants to add a conditional node after data cleansing, if there were less than 100 records that failed the preprocessing, he likes to continue running spark ML job, else he wants to send an email to the data engineer.