Configure - setting up Pipeline Config for both Batch and Realtime Pipelines

Description

As a user, I want to be able to set configuration to a pipeline:

BATCH PIPELINE
Stage Logging - stageLoggingEnabled - true/false
Tooltip: Allows logs from each stage in the pipeline to be queried individually.
Instrumentation - processTimingEnabled - true/false
Tooltip: Emits timing metrics such as total time, mean, standard deviation for pipeline stages.

REALTIME PIPELINE
Stage Logging - stageLoggingEnabled - true/false
Tooltip: Allows logs from each stage in the pipeline to be queried individually.
Instrumentation - processTimingEnabled - true/false
Tooltip: Emits timing metrics such as total time, mean, standard deviation for pipeline stages.
Checkpointing - disableCheckpoints - true/false - Note that this should be set to true if the switch is disabled.
Tooltip: Allows Apache Spark Streaming to checkpoint data (RDDs) to persistent storage so that the pipeline can recover from failures
Graceful Stop - stopGracefully - true/false
Tooltip: When the pipeline is stopped, waits for the processing of all received data to be completed

Release Notes

None

Activity

Show:
Lea Cuniberti-Duran
May 16, 2017, 11:50 PM

please add details to this jira.

Tony Bach
May 23, 2017, 5:31 PM
Fixed

Assignee

Tony Bach

Reporter

Lea Cuniberti-Duran

Labels

Docs Impact

None

UX Impact

None

Components

Sprint

None

Fix versions

Priority

Major
Configure