Retrieval and RAG
Mean average precision
Implemented
rag.mean_average_precision_at_kDefinition
For each query, the mean of Precision@i over the ranks i of relevant retrieved documents, divided by the number of relevant documents (unretrieved ones count as 0); averaged over queries (trec_eval convention).
Formula
AP = (1/|R|) Σ_i P@i · rel_i ; MAP = mean_q AP_q
Range: [0, 1]
Inputs and outputs
- relevant: per query, relevant ids or id -> grade
- retrieved: per query, ranked ids
Returns: MetricResult (float, or per-example array with average=None)
Assumptions
No assumptions beyond valid, aligned inputs of the documented types.
Limitations
No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.
Python API
import evalsuite as es
es.mean_average_precision_at_k(relevant, retrieved, k=10)References
- Manning CD, Raghavan P, Schütze H. Introduction to Information Retrieval. Cambridge University Press; 2008. Chapter 8.