Retrieval and RAG
Task success rate
Implemented
rag.task_success_rateDefinition
Share of requests the whole system resolved to specification (end-to-end query resolution, tool-assisted task success), with a Wilson 95% interval.
Formula
successful tasks / tasks
Range: [0, 1]
Inputs and outputs
- success: one boolean per task
Returns: MetricResult (float, or per-example array with average=None)
Assumptions
No assumptions beyond valid, aligned inputs of the documented types.
Limitations
No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.
Python API
import evalsuite as es
es.task_success_rate(success)References
- Wilson EB. Probable inference, the law of succession, and statistical inference. JASA. 1927;22:209-212.