Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Kafka Streaming source with state managment is slow

Description

The new state management version of the Kafka source is an order of magnitude slower than the previous implementation.

 

On closer examination, this is because it is creating the RecordTransform for each record, whereas the previous one creates it once per microbatch.

Release Notes

None

Activity

Show:

Albert ShauOctober 25, 2022 at 10:47 PM

cherry-pick:

Albert ShauOctober 25, 2022 at 9:32 PM

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Labels

Triaged

Yes

Components

Fix versions

Priority

Created October 25, 2022 at 9:26 PM
Updated October 25, 2022 at 10:49 PM
Resolved October 25, 2022 at 10:49 PM