UI JSON Specification
Specification Audit
Active Version | v2.0 | CDAP >= v3.0 | Initial Version |
|---|---|---|---|
Dev Version | v2.1 | CDAP > 3.4 | Adding label to metadata section to display better plugin names. CDAP-5340 |
Purpose
To help any hydrator plugin developer write a configuration JSON for representing properties of a hydrator plugin, group them and configure the output(s) (the schema of data going out of a plugin) if any required.
Why do we need this?
What this JSON achieves is to help in configuring the configurations of a plugin to be specific components in UI so that overhead of giving the right format of value for a configuration is removed. By default if no configuration is provided for a plugin all the properties appear as text-boxes in UI. It will be difficult to fill in the values for them if it requires it in specific format (for instance JSON or select one from multiple choices etc.,).
What does it have - Structure?
Before we dive into the structure it is imperative we clear up certain terms.
Configuring (Configurations) – A plugin in hydrator has configurations that needs to be set with appropriate value, for the plugin to function. This configuration could be a string or a number or a JSON blob or a Javascript/Python snippet etc., Configuring here refers to setting plugin configurations with appropriate value and for hydrator developers to do that we need to map configurations to widget to set a value.
Output – Every plugin will have data emitted out of it (passing it to subsequent plugins or dump to a database). An output of a plugin specifies the schema of data that is emitted out of the plugin.
Input – Similar to output every plugin will have an input data coming into the plugin (except for source plugins)
Configuration JSON for a plugin is structured as,
metadata
list of groups of configurations
list of output configurations
‘Groups’ refers to collection of configurations of a plugin. An example structure of a configuration JSON would look like this - Sample JSON
CustomPlugin.json
{
"metadata": {...},
"inputs": null,
"configuration-groups": [
{group1},
{group2},
...
],
"output": [
{configuration-output-1},
{configuration-output-2},
...
]
}Metadata
Metadata refers to the top level information of a plugin. Metadata map includes,
label - Label for the plugin in hydrator
type - Type of the plugin
name - Name of the plugin (not sure if name and label should be separate)
artifact-version - Version of the plugin (artifact).
spec-version - Version of SPEC based on which the json is generated.
Following is an example of the metadata section:
{
"metadata": {
"spec-version" : "2.1",
"label" : "Azure Blob Store"
},
...
}Groups
A group is a meaningful collection of configurations where related configs are grouped together to view and edit in hydrator. For instance in a Stream plugin we group together the stream, Processing time window & delay together [more explanation is needed as to why they are grouped together]. A group can essentially have a label , representing the label of the group and a list of fields inside the group. These are represented as ‘label’ and ‘fields’ inside the group map. Based on these information a sample JSON for a custom plugin would look something like this,
CustomPlugin.json
[
{
"label": "My Group Title",
"properties": [
{property1},
{property2},
...
]
}
]Here ‘My Group Title’ is the label of the group that will be grouping fields 1 & 2 under it(as they are related). A field inside a group is nothing but the configuration of the plugin.
Fields
In the above mentioned sample JSON, fields 1 & 2 are configurations of CustomPlugin defined in the backend. We could now configure them in the JSON so that it gets easier to use view/edit them in hydrator.
Configurations for a field
Configurations of a field are the set of properties that we define in the JSON, that governs what/how that particular plugin field gets rendered in hydrator.
‘How’ refers how should it be displayed - like a textbox, passwordbox, dropdown, JSON editor etc., and
‘What’ refers to what should it default to. For instance if the property is rendered as a select widget and if we know that the possible list of values for it would be [‘Male’, ‘Female’], then we could configure that particular plugin property to have those values for the dropdown.
Required
name - Name of the field as specified in the backend.
widget-type - Type of widget to represent the field.
Optional
label - Label for the field. For instance a field coming from the backend like time.ticks.sec could be labelled as ‘Time in Seconds’.
widget-attributes - Attribute/values required by the widget. For instance, list of values for select widget or a default value for a number-textbox. More details should be available in widgets list section.
plugin-function - Function available for the plugin. For instance, Database batchsource plugin have a getSchema endpoint method. output-property signify the plugin property that will get updated from this plugin-function action.
From the above explanation we could enhance the above mentioned sample JSON to something like this,
CustomPlugin.json
{
"configuration-groups": [
{
"label": "My Group Title",
"properties": [
{
"name": "Field1",
"widget-type": "numberbox",
"widget-attributes": {
"default": "1"
},
"plugin-function": {
"method": "POST",
"endpoint": "getSchema",
"output-property": "schema"
}
},
{
"name": "Field2",
"widget-type": "json-editor"
}
]
}
],
"output": [
{output-property1}
{output-property2}
...
]
}Widgets
A widget in UI represents a component that we could use to set the value of a property of a plugin. It could be anything representable in HTML. For instance if our CustomPlugin has a property that requires a JSON value, we could use a json-editor to set value to it. A widget, in general, is a component that we can use to represent a property of a plugin to attach a value to it. The following are the list of widgets that we right now have in CDAP that could be readily be used.
Widget-type | Description | Attributes | Output data type | Sample JSON |
textbox | Default HTML textbox to enter any string |
| string |
{
"name": "property1",
"label": "My Property Label",
"widget-type": "textbox",
"widget-attributes": {
"default": "Some Text"
}
}
|
number | Default HTML number textbox. Can enter only valid numbers. |
| string |
{
"name": "property1",
"label": "My Property Label",
"widget-type": "number",
"widget-attributes": {
"default": "1",
"min": "1",
"max": "100"
}
}
|
password | Default HTML password box. | No attributes | string |
{
"name": "property1",
"label": "My Property Label",
"widget-type": "password",
"widget-attributes": {}
}
|
datetime | Date time picker. Used to set date (in string) for any property | default, format (format could be ISO, long, short or full format dates. Reference | string |
{
"name": "property1",
"label": "My Property Label",
"widget-type": "datetime",
"widget-attributes": {
"default": "current",
"format": "MM-DD-YYYY"
}
}
|
csv | Comma separated values. Each value is entered in a separate box | No attributes | comma-separated string |
{
"name": "property1",
"label": "My Property Label",
"widget-type": "csv",
"widget-attributes": {}
}
|
dsv | Delimiter-separated values; each value is entered in a separate box |
| delimiter-separated string | {
"name": "property1",
"label": "My Property Label",
"widget-type": "dsv",
"widget-attributes": {
"delimiter": ":"
}
} |
json-editor | Json editor to pretty-print and auto format JSON while typing |
| string |
{
"name": "property1",
"label": "My Property Label",
"widget-type": "json-editor",
"widget-attributes": {
"default": "{ \"p1\": \"value\" }"
}
}
|
javascript-editor, python-editor | An editor to write Javscript (javascript-editor) or Python (python-editor) as a value for a property. |
| string |
{
"name": "property1",
"label": "My Property Label",
"widget-type": "javascript-editor",
"widget-attributes": {
"default": "function transform(input, emitter, context) {\n emitter.emit(input);\n}"
}
}
|
keyvalue | A key-value editor which allows you to construct map. |
| string |
{
"name": "property1",
"label": "My Property Label",
"widget-type": "keyvalue",
"widget-attributes": {
"delimiter": ",",
"kv-delimiter": ":"
}
}
|
keyvalue-dropdown | Similar to keyvalue widget, but with a drop-down value list |
| string | {
"name": "property1",
"label": "My Date Property",
"widget-type": "keyvalue-dropdown",
"widget-attributes": {
"delimiter": ",",
"kv-delimiter": ":",
"dropdownOptions": [ "Option1", "Option2"]
}
} |
select | An HTML drop-down with a list of values; allows one choice from the list |
| string |
{
"name": "property1",
"label": "My Property Label",
"widget-type": "select",
"widget-attributes": {
"values": ["Apples", "Oranges", "Bananas"],
"default": "Bananas"
}
} |
dataset-selector, stream-selector | A type-ahead textbox with a list of datasets ( | No attributes | string | {
"name": "property1",
"label": "My Property Label",
"widget-type": "dataset-selector",
"widget-attributes": {}
} |
ds-multiplevalues | A delimiter separated values widget that allows to specify a list of values separated by a delimiter. |
| string | {
"name": "property1",
"label": "My Property Label",
"widget-type": "ds-multiplevalues",
"widget-attributes": {
"delimiter": ",",
"values-delimiter": ":",
"numValues": "3",
"placeholders": ["Input Field", "Lookup", "Output Field"]
}
} |