ORIGINAL COMPANY-CONTEXT PRACTICE
Design a recommendation experiment with shared viewing
Original practice scenario: A hypothetical entertainment service evaluates a new recommendation row. Several profiles share a household and devices. How would you choose the randomization unit and metrics?
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Choose an assignment unit that matches likely interference. Profile-level randomization can become misleading when shared devices or household members influence each other's viewing.
Work through it
- State the target outcome and candidate population, then examine whether household, account or profile assignment offers a credible separation of treatment experiences.
- Measure meaningful viewing outcomes and satisfaction proxies with latency and diversity guardrails, retaining unsuccessful recommendations and non-engagement in the denominator.
- Estimate uncertainty at the assignment unit and inspect novelty effects and important content or household slices before deciding whether to expand.
Tradeoffs to discuss
- Household assignment reduces contamination but lowers the effective sample count; profile assignment offers more units while requiring stronger independence assumptions.
Common pitfalls
- Do not treat repeated sessions from one assigned household as independent experimental observations.
- A click increase may reflect a distracting presentation rather than better content discovery.