Text generation
Cross-entropy (negative log-likelihood)
Implemented
text-generation.cross_entropyDefinition
Average negative log-probability the model assigns to the observed tokens, pooled over all tokens of all sequences.
Formula
H = −(1/T) Σ_t log p(x_t | x_<t)
Range: [0, ∞)
Inputs and outputs
- token_logprobs: per-token natural-log probabilities, one array per sequence
Returns: MetricResult (float, or per-example array with average=None)
Assumptions
No assumptions beyond valid, aligned inputs of the documented types.
Limitations
No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.
Python API
import evalsuite as es
es.cross_entropy(token_logprobs, base=2) # bits per tokenReferences
- Jelinek F, Mercer RL, Bahl LR, Baker JK. Perplexity—a measure of the difficulty of speech recognition tasks. JASA. 1977;62(S1):S63.