# RAG evidence-metadata evaluation

Automated status: **automated_fail**

Semantic accuracy, grounding and usefulness: **human review required**.
Passing these gates is not a production release approval or automated hallucination check.

## Macro metrics

| Metric | Value | Eligible cases |
| --- | ---: | ---: |
| retrieval_precision_at_k | 0.380952 | 7 |
| retrieval_recall_at_k | 0.714286 | 7 |
| citation_validity | 0.714286 | 7 |
| required_citation_coverage | 0.714286 | 7 |
| decision_accuracy | 0.750000 | 8 |

## Gate failures

- `{"actual": 0.38095238095238093, "metric": "retrieval_precision_at_k", "minimum": 0.5, "scope": "aggregate"}`
- `{"actual": 0.7142857142857143, "metric": "retrieval_recall_at_k", "minimum": 0.8, "scope": "aggregate"}`
- `{"actual": 0.7142857142857143, "metric": "citation_validity", "minimum": 1.0, "scope": "aggregate"}`
- `{"actual": 0.7142857142857143, "metric": "required_citation_coverage", "minimum": 1.0, "scope": "aggregate"}`
- `{"actual": 0.75, "metric": "decision_accuracy", "minimum": 1.0, "scope": "aggregate"}`
- `{"actual": 0.0, "metric": "retrieval_recall_at_k", "minimum": 1.0, "scope": "C02"}`
- `{"actual": 0.0, "metric": "required_citation_coverage", "minimum": 1.0, "scope": "C02"}`
- `{"actual": 1, "maximum": 0, "metric": "invalid_citations", "scope": "C02"}`
- `{"metric": "retrieval_document_integrity", "scope": "C04", "unknown_document_ids": ["D99"]}`
- `{"actual": 0.0, "metric": "retrieval_recall_at_k", "minimum": 1.0, "scope": "C04"}`
- `{"actual": 0.0, "metric": "required_citation_coverage", "minimum": 1.0, "scope": "C04"}`
- `{"actual": 1, "maximum": 0, "metric": "invalid_citations", "scope": "C04"}`
- `{"actual": "answer", "expected": "refuse", "metric": "expected_decision", "scope": "C07"}`
- `{"actual": "answer", "expected": "refuse", "metric": "expected_decision", "scope": "C08"}`

## Case review queue

| Case | Critical | Decision match | Invalid citations | Missing required citations | Human review |
| --- | --- | --- | ---: | ---: | --- |
| C01 | False | True | 0 | 0 | not performed |
| C02 | True | True | 1 | 2 | not performed |
| C03 | False | True | 0 | 0 | not performed |
| C04 | True | True | 1 | 1 | not performed |
| C05 | True | True | 0 | 0 | not performed |
| C06 | False | True | 0 | 0 | not performed |
| C07 | True | False | 0 | 0 | not performed |
| C08 | True | False | 0 | 0 | not performed |

See report.json for answers, ordered retrieval, citation IDs, input SHA-256 hashes and source line numbers.