ORIGINAL COMPANY-CONTEXT PRACTICE
Evaluate a rare-defect inspection model
Original practice scenario inspired by KLA Corporation's public business context; this is not a reported interview question or a description of its internal systems. A fictional electronics inspection model flags rare defects in images from manufacturing lots. Adjacent images from one lot look very similar. How would you assess model quality before proposing an inspection workflow?
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Evaluate at the operational decision grain and keep correlated lots separated. A high overall accuracy can conceal a model that misses almost every defect when negatives dominate.
Work through it
- Split by manufacturing lot and time rather than randomly by image. Document labeling uncertainty and whether one defect can appear in several images.
- Measure recall for defect categories, false alerts per inspected unit, and review workload across thresholds. Report uncertainty for rare categories with few positive examples.
- Replay the proposed review workflow, including duplicate alerts and ambiguous images. Compare against the current inspection baseline and evaluate new equipment or process conditions separately.
Tradeoffs to discuss
- More sensitivity usually increases review effort. Sampling extra defects helps diagnosis but requires prevalence-aware interpretation of precision.
Common pitfalls
- Image-level leakage inflates generalization.
- A balanced evaluation set cannot directly predict production alert volume without the real class prevalence.