| Info |
|---|
Data Cacher is designed for pipelines that use Spark as the execution engine. If the pipeline uses MapReduce as the execution engine, Data Cacher has no effect. |
Plugin version: 2.1011.10
The Data Cacher plugin was introduced in 6.1.4 and 6.2.1. It caches any record that is passed through it. This is useful when you have auto-caching disabled at the Engine config level and you want to cache records at certain points in a pipeline. Spark caching prevents unnecessary recomputation of previous stages, which is particularly helpful when you have pipelines with branches.
...