Structured output and tools
Unwanted extra-content rate
Implemented
structured-output.extra_content_rateDefinition
Share of outputs that contain text beyond the requested payload, e.g. prose or code fences around JSON that should stand alone.
Formula
outputs with extra content / outputs
Range: [0, 1]
Inputs and outputs
- outputs: model outputs (strings)
Returns: MetricResult (float, or per-example array with average=None)
Assumptions
No assumptions beyond valid, aligned inputs of the documented types.
Limitations
No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.
Python API
import evalsuite as es
es.extra_content_rate(outputs)References
- Zhou J, Lu T, Mishra S, et al. Instruction-following evaluation for large language models (IFEval). arXiv:2311.07911. 2023.