Design useful judgments
Label answerability and graded evidence at the passage level.
Practical AI · EDITION 1.0.0
Evaluate Retrieval with Evidence
Build a transparent evaluation engine for ranked evidence, graded relevance, unknown questions and paired regression analysis.
By Sanjaay Gowda T S · For python developers responsible for document search and rag evaluation.
You need: Python functions, lists and dictionaries; Basic JSON and terminal commands
Runtime and hardware details
Python 3.10 or later and a terminal. The supplied project runs locally and uses the Python standard library.
No model, API key, cloud account or network calls are used by the companion.
Python functions, lists and dictionaries; Basic JSON and terminal commands.
Full setup and scope ↓$2.99USD · one-time purchase
Prepared with AI assistance. Independent human technical, editorial and assistive-technology reader review remains pending. Automated checks and agent review are not certification.
PDF + EPUB + companion project. Any applicable tax is shown at checkout. Single-reader ebook terms; companion code is MIT licensed.
TRY THE FREE SAMPLE
See how the guide teaches.
Read 7 sample pages to explore the introduction, first chapter and teaching approach before you choose the complete edition.
Read free sample PDF · No checkout required
WHAT YOU WILL LEARN
Understand each part.
Inspect every boundary.
Calculate interpretable metrics
Trace precision, recall, reciprocal rank and nDCG to individual cases.
Compare changes responsibly
Expose missing runs, unknown facts and regressions before release.
INSIDE THE BOOK
9 focused chapters.
- Define the retrieval decision
- Create relevance judgments
- Run the evaluation companion
- Read precision and recall
- Evaluate ranking order
- Inspect unknown questions
- Compare configurations by case
- Diagnose failures in the evidence
- Package a reproducible decision
Your download includes
- 21-page tagged PDF with linked contents
- Reflowable EPUB for ebook readers
- Local Python project, fictional input, recorded expected JSON output and 12 companion tests
- 3 explanatory diagrams
- 9 chapters with exercises and worked answers
Before you begin
Python 3.10 or later and a terminal. The supplied project runs locally and uses the Python standard library.
No model, API key, cloud account or network calls are used by the companion.
Learning prerequisites: Python functions, lists and dictionaries; Basic JSON and terminal commands.
Prepared with AI assistance. Independent human technical, editorial and assistive-technology reader review is pending.
The included examples are synthetic teaching material. Model access, software subscriptions, hosting, ongoing updates and consulting are not included.
CLEAR EXPECTATIONS
Before you buy.
What will I build?
Build a transparent evaluation engine for ranked evidence, graded relevance, unknown questions and paired regression analysis. The local companion uses fictional teaching fixtures. Project limits: Evaluates supplied rankings; it does not retrieve documents or generate answers. Invented fixtures do not establish real retrieval quality.
What was tested?
The 12 companion tests passed on the packaged project. The supplied fictional fixture produced its recorded expected JSON output. EPUBCheck reported no errors or warnings; DAISY Ace and PDF/UA automated checks passed. These local teaching checks are not production accuracy claims or certification. Independent human technical, editorial and assistive-technology reader review remains pending.
How do I get the files?
Complete the secure Lemon Squeezy checkout to access the PDF, EPUB and companion ZIP. Keep your receipt and download access private. This is a one-time purchase of edition 1.0.0; ongoing updates and consulting are not included.
Can I share the book or the companion code?
The full ebook is licensed to one reader. The labelled free sample may be shared unchanged. The companion project is MIT licensed and may be adapted or shared with its licence notice. See the exact ebook terms.
Measure ranking quality without hiding missing labels.
One book. A complete learning project. Evidence you can inspect.
AI-assisted · Independent human review pending
$2.99USDBuy ebook