Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Salesforce sink: TooManyLockFailure : Too many lock failure 200 Trying again later

Description

Customers are running into lock contention issues with Salesforce bulk API updated when processing large number of records. Salesforce batch has a limit of 10,000 records. So while processing lets say 100K records, sales force sink creates 10 batches and executes them in parallel ( ) . This has the potential of creating lock contention issues if multiple batches do operations on same record.

See , for the error code and suggested fixes. One suggestion from salesforce is this -

Explore if a “serial” mode option can be set on the failed batches or if this option can be set in the plugin level for customer to choose based on their workload.

Release Notes

In the Salesforce Sink, added the Concurrency Mode property to let you configure the plugin for parallel or serial concurrency. Use this property to help resolve lock contention issues with Salesforce bulk API. In the Salesforce Batch Source, Salesforce Multi Object Batch Source, Salesforce Streaming Source, and Salesforce Sink, added the Connection Timeout property to let you set the maximum time in milliseconds to wait for connection initialization before time out.

Activity

Show:

Greeshma Swaminathan March 29, 2023 at 5:03 PM
Edited

Affects version <=1.4.3. Fix released in 1.4.4 .

prashant singh March 20, 2023 at 3:46 PM

cc:

vikas.rathee March 20, 2023 at 5:26 AM

set job timeout to 600000 for serial jobs to avoid any timeout failure. HUB PR raised

vikas.rathee March 15, 2023 at 7:12 PM

Got timeout issue with last fix also, as jobs are running on multiple nodes and executor doesn’t have exact number of jobs.

vikas.rathee March 15, 2023 at 7:08 PM

PR merged

Unresolved
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Labels

Priority

More fields

Created March 3, 2023 at 9:15 PM
Updated March 29, 2023 at 5:09 PM