In Apache Spark, what is the difference between a transformation and an action?

Question

Accepted Answer

B. A transformation defines a new DataFrame lazily without executing immediately; an action triggers execution of the computation. PySpark fundamentals documented by Databricks state that transformations (such as `filter`, `select`, `groupBy`, `join`) define a new DataFrame from an existing one lazily — no computation is triggered. Actions (such as `count`, `show`, `display`, `write`) trigger the actual computation by executing all accumulated transformations. This lazy evaluation model allows Spark to optimize the full computation plan before executing.

In Apache Spark, what is the difference between a transformation and an action?

Related Questions

Discussion