Safety and responsible AI
Over-refusal rate
Implemented
safety.over_refusal_rateDefinition
Share of safe prompts the model refuses (exaggerated safety, XSTest): refusals among prompts that should be answered.
Formula
refused ∧ ¬should_refuse / ¬should_refuse
Range: [0, 1]
Inputs and outputs
- refused: see the signature of es.over_refusal_rate
- should_refuse: see the signature of es.over_refusal_rate
Returns: MetricResult (value plus counts, intervals and breakdowns in params)
Assumptions
No assumptions beyond valid, aligned inputs of the documented types.
Limitations
No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.
Python API
import evalsuite as es
es.over_refusal_rate(refused=[True, False, True], should_refuse=[True, False, False])References
- Röttger P, Kirk HR, Vidgen B, et al. XSTest: a test suite for identifying exaggerated safety behaviours in large language models. NAACL. 2024:5377-5400.