ORIGINAL COMPANY-CONTEXT PRACTICE
Detect leakage in assay prediction
Original practice scenario inspired by Amgen Inc.'s public business context; this is not a reported interview question or a description of its internal systems. A synthetic research model predicts an assay result from compound features. Multiple measurements of the same compound and close analogs appear in the dataset. Explain an evaluation suitable for predicting results on genuinely new compound families.
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Choose a split that reflects the intended generalization claim. Randomly separating repeated measurements can test memorization of a compound rather than prediction for a new chemical family.
Work through it
- Group repeated compound identities and define a defensible family or scaffold grouping before splitting. Keep preprocessing and feature selection inside each training fold.
- Check which features existed before the assay result. Remove post-result annotations and examine laboratory batch effects that could reveal labels indirectly.
- Compare simple baselines and report performance across held-out families, with uncertainty and coverage limits. Treat prospective validation as a separate stronger test when feasible.
Tradeoffs to discuss
- Family-based splits are harder and can reduce training coverage, but they better match the proposed deployment. No single split captures every research shift.
Common pitfalls
- A compound identifier can become a memorization shortcut.
- Strong retrospective metrics do not establish biological validity or authorize medical decisions.