Data Pipeline Performance Tuning Guide

This document describes how you can tune your pipelines to achieve better performance. Performance depends on the size and characteristics of the data being processed, the structure of the pipeline, and the plugins being used. By understanding a little about how a pipeline is executed, you will be able to understand what settings can be adjusted and what impact they will have.

• Terminology

• Parallel Processing

• Pipeline Structure

• Joins

• Resources

• Cluster Sizing