Text generation
Distinct-n
Implemented
text-generation.distinct_nDefinition
Number of distinct n-grams divided by the total number of n-grams across all predictions (whitespace tokens); a simple lexical-diversity indicator.
Formula
|unique n-grams| / |n-grams|
Range: [0, 1]
Inputs and outputs
- predictions: one model output per example
Returns: MetricResult (float, or per-example array with average=None)
Assumptions
No assumptions beyond valid, aligned inputs of the documented types.
Limitations
No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.
Python API
import evalsuite as es
es.distinct_n(predictions, n=2)References
- Li J, Galley M, Brockett C, Gao J, Dolan B. A diversity-promoting objective function for neural conversation models. NAACL. 2016:110-119.