ORIGINAL COMPANY-CONTEXT PRACTICE
Evaluate catalog search across languages and measurement systems
Original practice scenario: A hypothetical cross-border marketplace searches listings with translated titles, inconsistent size units and duplicated seller descriptions. Design an evaluation beyond keyword matching.
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Separate semantic relevance from attribute compatibility. A search result can describe the right product category while offering the wrong size, connector or quantity.
Work through it
- Create reviewed query-listing cases across languages and locales, preserving structured attributes and their units alongside original and translated text.
- Compare lexical, embedding and hybrid retrieval baselines, evaluating category relevance and required-attribute satisfaction separately on held-out queries and sellers.
- Inspect ambiguous translations, unit conversions and duplicate listings, then measure useful result coverage with latency and low-confidence handling.
Tradeoffs to discuss
- Structured filters improve compatibility but depend on reliable catalog attributes; semantic retrieval finds varied wording while potentially overlooking hard constraints.
Common pitfalls
- Do not count a near-duplicate seller listing in both training and evaluation as independent evidence.
- A high similarity score is not proof that dimensions or electrical compatibility match the user's requirement.