ORIGINAL COMPANY-CONTEXT PRACTICE
Interpret an experiment that changes availability
Original practice scenario inspired by Industria de Diseño Textil, S.A.'s public business context; this is not a reported interview question or a description of its internal systems. A synthetic online fashion experiment changes product recommendations. Conversion rises, but recommended products sell out faster and control shoppers share the same inventory. How would you analyze the experiment?
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Shared inventory creates interference: treatment can change which products remain available to control shoppers. The usual independent-user comparison may therefore estimate a distorted effect under the tested allocation.
Work through it
- Check assignment and exposure, then measure availability trajectories, substitution, cancellations, and net revenue alongside conversion. Keep analysis aligned with the randomized unit.
- Investigate whether treatment demand depleted inventory that control would otherwise buy. Use category or location clusters, switchbacks, or separated stock pools only when operationally credible.
- Report the experiment's allocation and stock conditions, and assess whether a full rollout would reproduce them. Follow returns long enough to observe downstream effects.
Tradeoffs to discuss
- Cluster designs reduce interference but often reduce statistical power. Switchbacks require attention to carryover and replenishment cycles.
Common pitfalls
- Conditioning only on products that remained in stock selects on a treatment-affected variable.
- A user-level significance test cannot repair interference by itself.