Conditional Execution in Pipeline - Configuration design
Description
Release Notes
Attachments
relates to
Activity
Tony Bach August 18, 2017 at 11:53 PMEdited
There's not much work to be done on the UI for this, since the design contains simple widgets that the backend can just specify in the spec json. We just need to show a custom 'Output Schema' label, and maybe a placeholder for the textbox. UI PR: https://github.com/caskdata/cdap/pull/9400
Lea Cuniberti-Duran August 14, 2017 at 5:33 PM
As a CDAP user, I want to configure the Conditional stage by writing a JEXL expression and have the system verify its syntax.
Acceptance Criteria
As user, I can view the input Schema and edit the output schema
There is only one output schema, for both conditions
As user, I can type JEXL expression in text box
The instruction for the expression text box is: "Write an expression to specify the condition to be evaluated"
The example expression is the following:
The user validates the expression by clicking "Validate"
As a user I would like to be able to specify multiple conditions for the conditional stage from its configuration panel:
When I add a new condition (or am prompted with my first condition), I only want to see the button for choosing whether I am getting the condition data from "Runtime Arguments" or "Nodes" (see next section for details). Each time I make a successive selection, I would like the next relevant option to appear.
I would like to be able to press a button that will add second, third, fourth, etc. condition
Between each condition, I would like to be able to toggle between a "And"/"Or" selection
As a user, I would like to be able to specify the source of the condition for my conditional stage in the configuration panel:
I would like to be able to specify whether I want to condition on runtime arguments or statistics emitted by a previous stage
If I choose "Stage," I would like to specify which stage from a dropdown list of eligible stages. I only want to be allowed to select stages that come before the condition stage
If I choose "Runtime Arguments," I would like to specify the name of the runtime argument. I would like an error to be thrown at runtime if the runtime argument is not of type numeric
As a user, if I choose "Stage" as the condition source, I would like to select one of the following statistics for my condition:
Number of output records, input records, or error records
As a user, if I choose "Runtime Arguments," I would like to be able to specify my own argument name.
As a user, I would like to be able specify condition on the numeric stage or runtime value:
A condition that is either greater than, less than, equal to, not equal to
A number that comes after this condition ("target value")
As a user, I would like to be able to see metrics that explain the "decision" process that led up to the true/false evaluation of the pipeline:
For each condition I specified, I want to see the name of the node/statistic combination or runtime argument alongside the value on which the condition was evaluated. I would like the evaluation condition (>, <, ==, plus "target value") to be displayed here, too, and see whether the expression evaluated to true/false.
I would like to see whether the overall expression evaluated to true/false
Design
Use "Node" "FileSource."
Evaluate to true if
Value "Number of output records"
is "Greater than" "10"
Use "Runtime Arguments"
Evaluate to true if
Value "myRuntimeArgument"
is "Greater than" "10"