ORIGINAL COMPANY-CONTEXT PRACTICE
Test a synthetic document-redaction pipeline
Original practice scenario: Design tests for a pipeline that removes designated identifiers from fictional health-service documents before indexing them for search.
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Redaction needs an explicit information contract and adversarial tests, not merely a list of regular expressions. The searchable representation and auxiliary outputs must all follow the same policy.
Work through it
- Create synthetic fixtures with structured fields, free text, tables, scanned-text errors and identifiers split across lines or encoded in unusual formats.
- Define expected retained and removed spans, then test extraction, redaction, indexing, snippets, caches and diagnostic logs end to end.
- Measure missed removals and excessive redaction separately, version the policy and route unsupported formats or uncertain cases to a controlled review path.
Tradeoffs to discuss
- Aggressive removal reduces exposure but can destroy retrieval meaning; narrow rules preserve utility while missing unexpected representations.
Common pitfalls
- Do not test with actual patient data.
- A passing fixture suite is evidence of tested behavior, not proof that every possible document is anonymized.