An AI system generates 5,000 insurance claim decisions per day. Human review of every decision is infeasible. The team has a budget of 200 human reviews per day. Which review allocation strategy maximizes quality assurance effectiveness?

Question

Accepted Answer

B. Use stratified sampling: classify decisions by risk level and review a higher percentage of high-risk decisions (e.g., large claim amounts, policy edge cases, rare coverage categories). Stratified sampling is the CCA-F-recommended human review pattern. By classifying decisions by risk and reviewing proportionally more high-risk outputs, the review budget is spent where errors have the highest impact. Random sampling (Option A) misses the risk distribution. Option C relies on self-reported confidence, which is poorly calibrated. Option D is a spot check for daily initialization, not a quality assurance strategy for the full day's outputs.

An AI system generates 5,000 insurance claim decisions per day. Human review of every decision is infeasible. The team has a budget of 200 human reviews per day. Which review allocation strategy maximizes quality assurance effectiveness?

Related Questions

Discussion