...
Option2: Introduce "runcondition" plugin type. Connections into a run condition imply control flow rather than data flow.
Code Block |
---|
{ "stages": [ { "name": "customersTable", "plugin": { "name": "Database", "type": "batchsource", ... } }, { "name": "customersFiles", "plugin": { "name": "TPFSParquet", "type": "batchsink", ... } }, { "name": "afterDump", "plugin": { "name": "AlwaysRun", "type": "runcondition" } }, { "name": "purchasesTable", "plugin": { "name": "Database", "type": "batchsource" } }, { "name": "purchasesFiles", "plugin": { "name": "TPFSParquet", "type": "batchsink", ... } }, ], "connections": [ { "from": "customersTable", "to": "customersFiles" }, { "from": "customersFiles", "to": "afterDump" }, { "from": "afterDump", "to": "purchasesTable" }, { "from": "purchasesTable", "to": "purchasesFiles" } ] } |
You could also say that certain plugin types imply control flow (runcondition being one of them), whereas other plugin types imply data flow.