A data engineer is running a Lakeflow Job with 5 tasks. Task 3 fails intermittently due to transient network errors. The engineer wants to automatically retry Task 3 up to 3 times with a 2-minute delay between retries before marking the job as failed. How should this be configured?

Question

Accepted Answer

B. Configure the task-level retry policy with max retries = 3 and retry delay = 2 minutes. Lakeflow Jobs supports task-level retry policies. For each task, you can configure the maximum number of retries and the delay between retries. Setting max retries to 3 and the retry delay to 2 minutes will cause the job scheduler to automatically re-run Task 3 up to 3 times with a 2-minute pause before each retry if the task fails. The other options do not implement per-task automatic retry behavior.

A data engineer is running a Lakeflow Job with 5 tasks. Task 3 fails intermittently due to transient network errors. The engineer wants to automatically retry Task 3 up to 3 times with a 2-minute delay between retries before marking the job as failed. How should this be configured?

Related Questions

Discussion