ORIGINAL COMPANY-CONTEXT PRACTICE
Evaluate visual inspection across production lines
Original practice scenario: A hypothetical vehicle factory trains an image classifier on one assembly line and deploys it to another with different lighting and camera placement.
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Treat the new production line as a distribution shift, not simply another random sample. Evaluation must separate real defect sensitivity from shortcuts tied to imaging conditions.
Work through it
- Build a labeled synthetic or consented evaluation set covering both lines, defect categories, lighting changes and difficult non-defective cases.
- Split by production batch and time so near-duplicate images do not cross boundaries; compare per-defect recall and false rejection rates.
- Test camera changes and unavailable images, define uncertain-case review and monitor shifts after rollout with delayed verified outcomes.
Tradeoffs to discuss
- Line-specific adaptation may improve quality but increases maintenance and can hide poor transfer to future configurations.
Common pitfalls
- Do not use background cues as evidence of a defect.
- Avoid claiming that a classroom classifier is validated for physical production safety.