Retrieval and RAG
Context relevance
Implemented
rag.context_relevanceDefinition
Share of the retrieved context that is relevant to the question (per chunk or per sentence), a measure of retrieval noise (ARES context relevance).
Formula
relevant units / retrieved units
Range: [0, 1]
Inputs and outputs
- unit_relevance: per query, relevance of each retrieved chunk or sentence
Returns: MetricResult (float, or per-example array with average=None)
Assumptions
No assumptions beyond valid, aligned inputs of the documented types.
Limitations
No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.
Python API
import evalsuite as es
es.context_relevance(chunk_relevance)References
- Saad-Falcon J, Khattab O, Potts C, Zaharia M. ARES: an automated evaluation framework for retrieval-augmented generation systems. NAACL. 2024:338-354.