Structured output and tools
API-call success rate
Implemented
structured-output.api_call_success_rateDefinition
Share of tool or API calls that completed successfully (status flags, HTTP status codes below 400, or exceptions recorded as failures).
Formula
successful calls / calls
Range: [0, 1]
Inputs and outputs
- results: per call, success flag, HTTP status or exception
Returns: MetricResult (float, or per-example array with average=None)
Assumptions
No assumptions beyond valid, aligned inputs of the documented types.
Limitations
No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.
Python API
import evalsuite as es
es.api_call_success_rate(call_results)References
- Patil SG, Mao H, Yan F, et al. The Berkeley Function Calling Leaderboard (BFCL): from tool use to agentic evaluation of large language models. ICML. 2025.