Goals

Ability to specify condition for execution-path in the hydrator pipelines.

Checklist

•  User stories documented (Shankar/Sagar)
•  User stories reviewed (Nitin)
•  Design documented (Shankar/Sagar)
•  Design reviewed (Terence/Andreas)
•  Feature merged ()
•  Examples and guides ()
•  Integration tests () 
•  Documentation for feature ()
•  Blog post

Use-case

Use User has a multi job pipeline, set of transforms run as a part of MR job doing data cleansing and a spark ML job on training data to output model. wants to add a conditional node after data cleansing, if there were less than 100 records that failed the preprocessing, he/she likes to continue running spark ML job, else he/she wants to send an email to the data engineer.

User-Stories

•  Pipeline developer wants to add a conditional node after one job before running the next job. 
•  Pipeline developer wants to add a conditional node at the end of the pipeline and optionally send an email if there were any errors in the final stage.
•  Pipeline developer wants to configure the conditional node with his custom code logic.   
•  Pipeline developer wants to configure the next job to run based on a condition, based on condition pipeline can run either Job-A or Job-B  
•  Plugin developer wants to add a new conditional-plugin supporting a new language or expression for specifying conditional logic.
•  Pipeline developer wants to use workflow tokens and use that information to take action in the conditional node.
•  Pipeline developer wants to test that their custom condition logic is correct without having to deploy and run their pipeline.

...