ORIGINAL COMPANY-CONTEXT PRACTICE
Choose an experiment unit for shared accounts
Original practice scenario inspired by The Walt Disney Company's public business context; this is not a reported interview question or a description of its internal systems. A fictional streaming service tests a discovery interface. Several profiles share one household account, and people may watch together. How would you choose randomization and analyze engagement without overstating independent evidence?
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Choose a randomization unit that limits contamination across shared experiences. Household-level assignment may be more credible than profile-level assignment when recommendations and viewing behavior influence other household members.
Work through it
- Define eligibility and stable account assignment before exposure. Specify whether the outcome is per household, per active profile, or per viewing session.
- Analyze using the randomized unit or a method that accounts for clustering. Track exposure, shared-device behavior, and changes in the number of active profiles.
- Report absolute engagement effects and guardrails such as abandonment or playback failures. Allow enough follow-up to distinguish novelty from sustained changes.
Tradeoffs to discuss
- Household assignment reduces the number of independent units and can lower power, but avoids a misleading precision gain from correlated profiles.
Common pitfalls
- Treating every session as independent understates uncertainty.
- Conditioning only on users who watched after assignment can introduce selection caused by the treatment.