ORIGINAL TECHNICAL PRACTICE
Choose prompting, retrieval or fine-tuning
How would you decide whether to improve a model with prompting, retrieval or fine-tuning?
Written for independent practice. The model answer presents one approach; clarify constraints and explain alternatives.
A REASONED APPROACH
Model answer.
Match the intervention to the observed failure. Prompting targets instructions and output behavior, retrieval supplies external evidence, and fine-tuning changes learned behavior from examples; none is a universal substitute for evaluation.
Work through it
- Classify failures into missing knowledge, instruction-following, domain language, formatting or task skill. Establish a simple baseline and a held-out evaluation set.
- Use retrieval for information that changes or needs source attribution. Explore prompting and structured validation when the model already has the relevant capability.
- Consider fine-tuning only with suitable rights, representative examples and a clear objective. Compare quality, maintenance effort, latency and regression risk against simpler options.
Tradeoffs to discuss
- Fine-tuning can increase consistency but introduces training-data and lifecycle work. Retrieval can update evidence quickly but adds indexing and retrieval failure modes.
Common pitfalls
- Do not assume fine-tuning reliably memorizes precise changing facts. Tuning on a tiny demonstration set can overfit while appearing impressive on repeated examples.