Retrieval and RAG
Mean reciprocal rank
Implemented
rag.mrrDefinition
Average over queries of 1 / rank of the first relevant document (0 when none is retrieved within the cutoff).
Formula
MRR = mean_q 1 / rank_q
Range: [0, 1]
Inputs and outputs
- relevant: per query, relevant ids or id -> grade
- retrieved: per query, ranked ids
Returns: MetricResult (float, or per-example array with average=None)
Assumptions
No assumptions beyond valid, aligned inputs of the documented types.
Limitations
No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.
Python API
import evalsuite as es
es.mrr(relevant, retrieved)References
- Voorhees EM. The TREC-8 question answering track report. TREC. 1999.