We're updating the issue view to help you get more done. 

Hydrator config changes

Description

The etl apps use configs where the plugin types are at the top level. We need to make this more generic because we are starting to add more and more plugin types.

To support additional types, we should move the plugin type down into the plugin section of each stage. The config then becomes a list of stages, a list of connections, resources, and any other options needed.

In addition, we need to start putting the config version in the config, to allow us to handle incompatibilities more fluidly.

Release Notes

Changed the format of Hydrator configuration. All pipeline stages are now in a 'stages' array instead of broken up into 'source', 'transforms', and 'sinks' arrays.

Assignee

Albert Shau

Reporter

Albert Shau

Labels

None

Docs Impact

None

UX Impact

None

Components

Fix versions

Priority

Major
Configure