A data engineer needs to pass a runtime parameter `run_date` to all tasks in a Lakeflow Job so each task can filter data for the correct date. Which Lakeflow Jobs feature supports passing shared parameters across all tasks in a job?

Question

Accepted Answer

B. Job-level parameters that are automatically propagated to all tasks. Lakeflow Jobs supports job-level parameters that are automatically pushed to all tasks within the job. The engineer can define `run_date` as a job parameter and all tasks receive it without needing to configure it individually. Spark configuration properties are for Spark settings, not user-defined job parameters. Writing to a Delta table requires tasks to read it themselves. Init script environment variables are for cluster setup, not per-run dynamic parameters.

A data engineer needs to pass a runtime parameter `run_date` to all tasks in a Lakeflow Job so each task can filter data for the correct date. Which Lakeflow Jobs feature supports passing shared parameters across all tasks in a job?

Related Questions

Discussion

A data engineer needs to pass a runtime parameter run_date to all tasks in a Lakeflow Job so each task can filter data for the correct date. Which Lakeflow Jobs feature supports passing shared parameters across all tasks in a job?

Related Questions

Discussion

A data engineer needs to pass a runtime parameter `run_date` to all tasks in a Lakeflow Job so each task can filter data for the correct date. Which Lakeflow Jobs feature supports passing shared parameters across all tasks in a job?