ORIGINAL COMPANY-CONTEXT PRACTICE
Evaluate transaction categorization with changing merchant text
Original practice scenario: A hypothetical personal-finance feature assigns categories from transaction descriptions. Merchant names vary, customers correct labels, and new merchants appear. Design training and evaluation.
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Define the category taxonomy and how user-specific preferences differ from general merchant labels. A model should not learn a customer's private corrections as universal truth without an appropriate policy.
Work through it
- Construct labeled examples with source and correction provenance, grouping repeated merchant variants and separating ambiguous transactions for review.
- Split by time and include unseen-merchant evaluation, comparing a simple lookup or rule baseline with the candidate and reporting category-level errors.
- Measure confidence calibration and correction frequency, providing an understandable unknown category and a reversible user override rather than forcing every transaction into a confident label.
Tradeoffs to discuss
- Personalization can improve individual usefulness but increases data requirements and privacy obligations; a shared taxonomy improves consistency while missing personal intent.
Common pitfalls
- Randomly splitting near-identical descriptions can exaggerate generalization.
- Do not treat a frequent customer correction as a verified accounting classification.