ORIGINAL TECHNICAL PRACTICE
Explain why an observed lift may not be causal
Customers receiving more support calls have higher churn. Does reducing support calls necessarily reduce churn?
Written for independent practice. The model answer presents one approach; clarify constraints and explain alternatives.
A REASONED APPROACH
Model answer.
No. Support calls may be a response to problems that also cause churn, so the observed association does not identify the effect of changing support availability.
Work through it
- Sketch plausible causal relationships, including product failures, customer complexity, prior dissatisfaction and support contact. Consider reverse causation and confounding.
- Define the intervention and outcome precisely. A randomized change to a suitable support policy can answer a different question from comparing customers who happened to call more.
- If using observational methods, state identification assumptions, examine overlap and pre-treatment covariates, and conduct sensitivity checks. Avoid treating adjustment as automatic proof of causality.
Tradeoffs to discuss
- Experiments can be expensive or inappropriate for some interventions. Observational analysis can be useful, but its conclusion depends on assumptions that the data may not fully verify.
Common pitfalls
- Controlling for a variable affected by the intervention can bias an estimate. A predictive model’s feature importance does not establish which actions will change churn.