Long context and summarization
Citation coverage
Implemented
long-context.citation_coverageDefinition
Share of output statements that carry at least one citation (ALCE: statements that should be attributable must cite something), pooled over outputs.
Formula
statements with ≥1 citation / statements
Range: [0, 1]
Inputs and outputs
- citations_per_statement: see the signature of es.citation_coverage
Returns: MetricResult (value plus counts, intervals and breakdowns in params)
Assumptions
- ``citations_per_statement``: per output, the number of citations attached to each statement.
Limitations
No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.
Python API
import evalsuite as es
es.citation_coverage([[1, 0, 2]])References
- Gao T, Yen H, Yu J, Chen D. Enabling large language models to generate text with citations. EMNLP. 2023:6465-6488.