ORIGINAL TECHNICAL PRACTICE
Explain feature importance carefully
A stakeholder sees that age is the most important model feature. What can and cannot be concluded?
Written for independent practice. The model answer presents one approach; clarify constraints and explain alternatives.
A REASONED APPROACH
Model answer.
Importance describes a model’s use of information under a particular method and dataset; it does not establish a causal effect or validate the appropriateness of using that feature.
Work through it
- Identify the importance method and what it measures, such as loss change after permutation. Evaluate on appropriate held-out data rather than only the training set.
- Examine correlated features, subgroup performance and stability across resamples or time periods. Correlated substitutes can change apparent importance without changing the underlying signal.
- Explain the result in the context of the decision and applicable feature-use rules. If an intervention is proposed, require a causal analysis rather than extrapolating from predictive importance.
Tradeoffs to discuss
- Global summaries are compact but can hide local differences. More detailed explanation methods add interpretive choices and computational cost.
Common pitfalls
- An important feature may be a proxy for another variable or for leakage. Small importance does not prove a feature has no effect in every region of the input space.