Skip to content
EvalSuite
Documentation menu

Text generation

chrF / chrF++

Implementedtext-generation.chrf

Definition

F-beta score (β = 2) over character n-grams (n = 1..6), plus word uni- and bigrams for chrF++ (word_order=2); robust for morphologically rich languages. Computed exactly as sacreBLEU.

Formula

chrF_β = (1 + β²) · chrP · chrR / (β² · chrP + chrR)

Range: [0, 100]

Inputs and outputs

  • references: one reference string (or a list of references) per example
  • predictions: one model output per example

Returns: MetricResult (float, or per-example array with average=None)

Assumptions

No assumptions beyond valid, aligned inputs of the documented types.

Limitations

No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.

Python API

PythonSince v0.4.0
import evalsuite as es

es.chrf(references, predictions)               # word_order=2 for chrF++

References

  1. Popović M. chrF: character n-gram F-score for automatic MT evaluation. WMT. 2015:392-395.
  2. Popović M. chrF++: words helping character n-grams. WMT. 2017:612-618.
  3. Post M. A call for clarity in reporting BLEU scores. WMT. 2018:186-191.

Implementation status