ORIGINAL TECHNICAL PRACTICE
Choose document chunk boundaries
How would you choose a chunking strategy for a retrieval assistant spanning policy documents and tabular reports?
Written for independent practice. The model answer presents one approach; clarify constraints and explain alternatives.
A REASONED APPROACH
Model answer.
Chunking should preserve useful units of meaning and their provenance. Start with document structure and evaluate retrieval and answer quality; a fixed token size is a tunable baseline, not a universal optimum.
Work through it
- Retain headings, document IDs, revision dates and permission metadata. Keep a table’s column meaning with its rows rather than creating isolated numbers.
- Compare section-aware chunks with a fixed-size baseline using representative short and cross-section questions. Track evidence coverage and redundancy, not just average chunk length.
- Tune overlap and retrieval breadth, then evaluate whether the assembled context supplies the complete supporting statements within the model’s context budget.
Tradeoffs to discuss
- Small chunks can improve focus but lose qualifications; large chunks retain context but may dilute ranking. Overlap adds indexing and retrieval duplication.
Common pitfalls
- Splitting a rule from its exception can reverse its meaning. Good results on a handful of easy lookup questions do not establish that the boundaries work for complex questions.