ORIGINAL COMPANY-CONTEXT PRACTICE
Evaluate a synthetic merchant offer without redemption bias
Original practice scenario: A hypothetical card program sends selected customers a merchant offer. Analysts propose comparing redeemers with customers who did not redeem it. Improve the design.
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Redemption occurs after assignment and reflects customer preferences, so redeemers are not a comparable control group. Estimate the effect of offering the promotion using the original assignment.
Work through it
- Randomize eligible customers or appropriate clusters before exposure, retain assignment records and define primary spending and cost outcomes over a fixed window.
- Analyze intention-to-treat differences with uncertainty, checking balance, spillover, merchant substitution and the total eligible population rather than only active redeemers.
- Report redemption as a secondary behavioral outcome, and use any treatment-on-treated analysis only with explicit identification assumptions and limitations.
Tradeoffs to discuss
- Assignment-based effects may look smaller than redeemer comparisons but answer a clearer policy question and avoid selection on the outcome path.
Common pitfalls
- Do not exclude customers who received the offer but never used it.
- Measure net incremental value rather than gross promoted-merchant spending alone.