We're updating the issue view to help you get more done. 

Set spark.network.timeout to a higher default

Description

When large shuffle blocks are being used, Spark pipelines can fail due to timeouts fetching these blocks.

From my experiments, it is a good amount more reliable if spark.network.timeout is set to a higher value, like 600s.

Release Notes

Changed default value of spark.network.timeout to 10 minutes to make pipeline execution more stable for shuffle heavy pipelines.

Activity

Show:
Albert Shau
June 18, 2020, 4:29 PM
Albert Shau
June 18, 2020, 4:36 PM
Fixed

Assignee

Albert Shau

Reporter

Albert Shau

Labels

None

Docs Impact

None

UX Impact

None

Fix versions

Priority

Major
Configure