What is the key difference between offline evaluation and online monitoring in Mosaic AI Agent Evaluation?

Question

Accepted Answer

B. Offline evaluation runs during development against an evaluation set with optional ground-truth labels; online monitoring handles production traffic where ground-truth labels are typically not available.. Agent Evaluation is designed to be consistent between development and production. In development (offline), the team uses an evaluation set with representative queries and optionally ground-truth expected responses — labels enable additional quality metrics. In production (online), real user traffic arrives without ground-truth labels, so evaluation relies on reference-free LLM judges.

What is the key difference between offline evaluation and online monitoring in Mosaic AI Agent Evaluation?

Related Questions

Discussion