Page Comparison

...

For example, in a Batch Source plugin, properties such as name, basePath, duration, and delay could be grouped into a Batch Source Configuration.

Setting the hideByDefault property of a configuration group results in the group properties to be hidden by default.

In the case of a Batch Source plugin, it could look like this:

...

A widget in the CDAP UI represents a component that will be rendered and used to set the value of a property of a plugin. These are the different widgets, their type, their attributes (if any), their output data type, a description, sample JSON, that we support in CDAP pipelines as of version 6.2.0.

Widget Type

Widget Attributes

Output Data Type

Description

Example Widget JSON

csv

delimiter: the delimiter between each set of values
value-placeholder: placeholder for each row

Comma-separated string

Comma-separated values; each value is entered in a separate box

Code Block
{ "name": "property-csv", "widget-type": "csv", "widget-attributes": { "delimiter": ",", "value-placeholder": "enter a value" } }

daterange

No attributes

string

Selector for a date and time range using graphical calendar

Code Block
{ "name": "daterange-selector", "widget-type": "daterange", "widget-attributes": {} }

datetime

No attributes

string

Selector for a single date and time using graphical calendar

Code Block
{ "name": "datetime-selector", "widget-type": "datetime", "widget-attributes": {} }

dataset-selector

No attributes

string

A type-ahead textbox with a list of datasets from the CDAP instance

Code Block
{ "name": "property-dataset-selector", "widget-type": "dataset-selector", "widget-attributes": {} }

ds-multiplevalues

delimiter: the delimiter between each set of values
numValues: number of values (number of delimiter-separated values)
placeholders: array of placeholders for each value's textbox
values-delimiter: the delimiter between each value

string

A delimiter-separated values widget that allows specifying lists of values separated by delimiters

Code Block

{
  "name": "property-ds-multiplevalues",
  "widget-type": "ds-multiplevalues",
  "widget-attributes": {
    "delimiter": ",",
    "values-delimiter": ":",
    "numValues": "3",
    "placeholders": [
      "Input Field",
      "Lookup",
      "Output Field"
    ]
  }
}

dsv

delimiter: delimiter used to separate the values

Delimiter-separated string

Delimiter-separated values; each value is entered in a separate box

Code Block
{ "name": "property-dsv", "widget-type": "dsv", "widget-attributes": { "delimiter": ":" } }

hidden

default: default string value for the widget

string

This "hidden" widget allows values to be set for a property but hidden from users. A default can be supplied that will be used as the value for the property.

Code Block
{ "name": "property-hidden", "widget-type": "hidden", "widget-attributes": { "default": "defaultValue" } }

input-field-selector

No attributes

multiselect

string

A dropdown widget with a list of columns taken from the input schema. Selecting sets the input column for that plugin property.

Code Block
{ widget-type": "input-field-selector", "label": "Unique Fields", "name": "uniqueFields", "widget-attributes":{ "multiselect": "true" }

javascript-editor

default: default string value for the widget

string

An editor to write JavaScript code as a value of a property

Code Block
{ "name": "property-javascript-editor", "widget-type": "javascript-editor", "widget-attributes": { "default": "function transform(input, emitter, context) {\ \\n emitter.emit(input);\\n}" } }

json-editor

default: default serialized JSON value for the widget

string

A JSON editor that pretty-prints and auto-formats JSON while it is being entered

Code Block
{ "name": "property-json-editor", "widget-type": "json-editor", "widget-attributes": { "default": "{ \"p1\": \"value\" }" } }

keyvalue

delimiter: delimiter for the key-value pairs
kv-delimiter: delimiter between key and value
key-placeholder: placeholder for the key input
value-placeholder: placeholder for the value input

string

A key-value editor for constructing maps of key-value pairs

Code Block
{ "name": "property-keyvalue", "widget-type": "keyvalue", "widget-attributes": { "delimiter": ",", "kv-delimiter": ":", "key-placeholder": "Enter key", "value-placeholder": "Enter value" } }

multi-select

delimiter: delimiter to concatenate multiple values to form a single string
options: array of options where each option is defined by - id: value to be set for the plugin property - label: a label to be rendered in UI for that specific option
default: a string containing default value(s) to use when rendering the widget for the first time. You must use the specified delimiter to separate values.

string

Multi-select dropdown to choose one or more from a list of available options.

Code Block

{
  "name": "property-keyvalue",
  "widget-type": "multi-select",
  "widget-attributes": {
    "delimiter": ",",
    "default": "value1,value2",
    "options": [
      {
      "id": "value1",
      "label": "Value One"
      },
      {
      "id": "value2",
      "label": "Value two"
      }
    ]
  }
}

radio-group

layout: Layout for the radio buttons to render. Defaults to block to render one below the other. Another option is inline to render side-by-side
options: an array of options where each option is defined by - id: value to be set for the plugin property - label: a label to be rendered in UI for that specific option
default: default value to use when no value is specified

string

A group of radio buttons to choose one among a list of available options.

Code Block

{
  "name": "property-keyvalue",
  "widget-type": "radio-group",
  "widget-attributes": {
    "layout": "inline",
    "default": "value1",
    "options": [
      {
        "id": "value1",
        "label": "Value One"
      },
      {
        "id": "value2",
        "label": "Value two"
      }
    ]
  }
}

keyvalue-dropdown

delimiter: delimiter for the key-value pairs
dropdownOptions: list of drop-down options to display
kv-delimiter: delimiter between key and value

string

Similar to keyvalue widget, but with a drop-down value list

Code Block
{ "name": "property-keyvalue-dropdown", "widget-type": "keyvalue-dropdown", "widget-attributes": { "delimiter": ",", "kv-delimiter": ":", "dropdownOptions": [ "Option1", "Option2"] } }

non-editable-schema-editor

schema: schema that will be used as the output schema for the plugin

string

A non-editable widget for displaying a schema

Code Block
{ "name": "property-non-editable-schema-editor", "widget-type": "non-editable-schema-editor", "widget-attributes": {} }

number

default: default value for the widget
max: maximum value for the number box
min: minimum value for the number box

string

Default HTML number textbox that only accepts valid numbers

Code Block
{ "name": "property-number", "widget-type": "number", "widget-attributes": { "default": "1", "min": "1", "max": "100" } }

password

No attributes

string

Default HTML password entry box

Code Block
{ "name": "property-password", "widget-type": "password", "widget-attributes": {} }

plugin-list

plugin-type: The type of plugin that will be fetched

string

A drop-down with a list of plugins

Code Block
{ "name": "property-plugin-list", "widget-type": "plugin-list", "widget-attributes": { "plugin-type": "jdbc" } }

python-editor

default: default string value for the widget

string

An editor to write Python code as a value of a property

Code Block
{ "name": "property-python-editor", "widget-type": "python-editor", "widget-attributes": { "default": "def transform(input, emitter, context):\ \\n emitter.emit(input)\\n" } }

schema

schema-default-type: default type for each newly-added field in the schema
schema-types: list of schema types for each field from which the user can chose when setting the schema

string

A four-column, editable table for representing the schema of a plugin

Code Block

{
  "name": "property-schema",
  "widget-type": "schema",
  "widget-attributes": {
    "schema-default-type": "string",
    "schema-types": [
      "boolean",
      "int",
      "long",
      "float",
      "double",
      "bytes",
      "string",
      "map<string, string>"
    ]
  }
}

select

default: default value from the list
values: list of values for the drop-down

string

An HTML drop-down with a list of values; allows one choice from the list

Code Block
{ "name": "property-select", "widget-type": "select", "widget-attributes": { "default": "Bananas", "values": ["Apples", "Oranges", "Bananas"] } }

textarea

default: default value for the widget
rows: height of the textarea

string

An HTML textarea element which accepts a default value attribute and a height in rows

Code Block
{ "name": "property-textarea", "widget-type": "textarea", "widget-attributes": { "default": "Default text.", "rows": "1" } }

textarea-validate

placeholder: placeholder text for the textarea
validate-endpoint: plugin function endpoint to hit to validate the contents of the textarea
validate-button-text: label of the validate button
validate-success-message: message to display when validation succeeds

string

An HTML textarea element with a button to validate its contents using a plugin function endpoint

Code Block

{
  "name": "property-to-validate",
  "widget-type": "textarea-validate",
  "widget-attributes": {
    "placeholder": "E.g. ((token['Data Quality']['error'] / token['File']['output']) * 100) > runtime['error_percentage']",
    "validate-endpoint": "validate",
    "validate-button-text": "Validate",
    "validate-success-message": "Expression is valid"
  }
}

textbox

default: default value for the widget

string

An HTML textbox, used to enter any string, with a default value attribute

Code Block
{ "name": "property-textbox", "widget-type": "textbox", "widget-attributes": { "default": "Default text." } }

toggle

on: 'On' state of the toggle, defined by value - value to be set for the plugin property, and label - a label to be rendered in UI, limited to 64 characters.
off: 'Off' state of the toggle, defined by value - value to be set for the plugin property, and label - a label to be rendered in UI, limited to 64 characters.
default: default value for the widget

string

A toggle widget that allows toggling between 'on' and 'off' states

Code Block
{ "name": "property-toggle", "widget-type": "toggle", "widget-attributes": { "on": { "value": "on", "label": "On" }, "off": { "value": "off", "label": "Off" }, "default": "on" } }

Plugin Function

A plugin function is a method exposed by a particular plugin that can be used for a specific task, such as fetching an output schema for a plugin.

...

With plugin functions, if the widget is not supported in the CDAP UI or the plugin function map is not supplied, the user will not see the widget in the CDAP UI.

Widget Type

Widget attributes

Description

Example Widget JSON

outputSchema

label: Label for the button
btnClass: bootstrap css class to add to the button
multiple-inputs: boolean to indicate if there are multiple input schemas

Widget to populate output schema for a plugin. This is specifically used to populate output schema and not any other property.

Code Block

{
  "widget-type": "csv",
  "label": "Group by fields",
  "name": "groupByFields",
  "widget-attributes": {
    "delimiter": ",",
    "value-placeholder": "Field Name"
  },
  "plugin-function": {
    "method": "POST",
    "widget": "outputSchema",
    "label": "Get output schema",
    "button-class": "btn-primary",
    "plugin-method": "outputSchema",
    "required-fields": ["groupByFields", "aggregates"],
    "missing-required-fields-message": "'Group By Fields' & 'Aggregates' properties are required to fetch schema."
  }
}

getPropertyValue

label: Label for the button

Widget to populate any property in a plugin. This widget should be used in the property which needs to fetch its value from a plugin function implemented in the plugin backend

Code Block

{
  "widget-type": "select",
  "label": "Format",
  "name": "format",
  "widget-attributes": {
    "values": [
      "avro",
      "blob",
      "csv",
      "delimited",
      "json",
      "parquet",
      "text",
      "tsv"
    ],
    "default": "text"
  },
  "plugin-function": {
    "method": "POST",
    "widget": "getPropertyValue",
    "widget-attributes": {
      "label": "Get Schema Value"
    },
    "required-fields": ["path"],
    "missing-required-fields-message": "Please provide path field",
    "plugin-method": "getSchema"
  }
}

Example Plugin

In the case of a Batch Source plugin example, the configuration-groups, with additional widgets to show the groupByFields and aggregates properties and using a plugin-function, could be represented by:

...

Versions Compared

Old Version 23

New Version Current

Key

Plugin Function

Example Plugin