A data engineer uses Auto Loader to ingest files from an Azure Data Lake Storage Gen2 path. The source team periodically adds files in large batches — sometimes 50,000 files at once. Processing all 50,000 files in a single micro-batch causes executor out-of-memory errors. Which Auto Loader option limits how many files are processed per micro-batch?
This question requires Pro
Unlock all 14 questions in this certification
Written by certified professionals · Aligned to official exam objectives
View all Pro featuresMore Data Ingestion & Acquisition Questions
14 questions
Full Databricks Certified Data Engineer Professional Practice Test
All topics covered
All Databricks Certified Data Engineer Professional Questions
Browse by topic
Related Questions
What is the primary purpose of the `cloudFiles` source format in Databricks Auto Loader?...
A data engineer configures Auto Loader with `cloudFiles.schemaEvolutionMode` set to `addNewColumns`....
A data engineer needs to perform a one-time batch load of Parquet files from an S3 bucket into an ex...
A data engineer ingests JSON files using Auto Loader with a fixed target schema. Incoming files occa...
In Auto Loader, what is the difference between directory listing mode and file notification mode for...
Educational Content — CertQnA practice questions are written against official exam objectives, covering the same domains tested on the real exam. All content is original and independent — not actual exam questions, not affiliated with any certification vendor. Learn more about our content policy