Original practice scenario inspired by The Walt Disney Company's public business context; this is not a reported interview question or a description of its internal systems. A fictional streaming service tests a discovery interface. Several profiles share one household account, and people may watch together. How would you choose randomization and analyze engagement without overstating independent evidence?
Read model answer +
Choose a randomization unit that limits contamination across shared experiences. Household-level assignment may be more credible than profile-level assignment when recommendations and viewing behavior influence other household members.
Work through it
Define eligibility and stable account assignment before exposure. Specify whether the outcome is per household, per active profile, or per viewing session.
Analyze using the randomized unit or a method that accounts for clustering. Track exposure, shared-device behavior, and changes in the number of active profiles.
Report absolute engagement effects and guardrails such as abandonment or playback failures. Allow enough follow-up to distinguish novelty from sustained changes.
Tradeoffs to discuss
Household assignment reduces the number of independent units and can lower power, but avoids a misleading precision gain from correlated profiles.
Common pitfalls
Treating every session as independent understates uncertainty.
Conditioning only on users who watched after assignment can introduce selection caused by the treatment.
Original practice scenario inspired by The Walt Disney Company's public business context; this is not a reported interview question or a description of its internal systems. A synthetic streaming app receives subscription events such as renewal, cancellation, and correction. Messages arrive out of order. Explain a state reducer that determines entitlement without letting an older event undo a newer correction.
Read model answer +
Use explicit event identity and a documented ordering or version rule. Event arrival order is unreliable, and cancellation can mean either immediate revocation or access until a paid-through date.
Work through it
Define event semantics, effective timestamps, and authoritative subscription versions. Validate that every event belongs to the correct account and subscription.
Deduplicate event IDs and apply only valid transitions. Keep paid-through time distinct from current status, and record contradictions for reconciliation with the subscription authority.
Test renewal followed by a late cancellation, equal-version conflicts, duplicate corrections, and exact expiry boundaries. Replaying events in different delivery orders should yield the same authoritative state.
Tradeoffs to discuss
A reducer based on monotonic versions is simple, but some event sources require a reconciliation snapshot. Preserve enough history to explain the selected state.
Common pitfalls
Sorting solely by receipt time makes replay unstable.
A generic canceled flag can revoke paid access early or retain access after an immediate security revocation.
