Multilingual
Language identification accuracy
Implemented
multilingual.language_id_accuracyDefinition
Accuracy of predicted language labels, with macro-F1 over languages and per-language recall.
Formula
mean[pred = true]
Range: [0, 1]
Inputs and outputs
- true_languages: see the signature of es.language_id_accuracy
- predicted_languages: see the signature of es.language_id_accuracy
Returns: MetricResult (value plus counts, intervals and breakdowns in params)
Assumptions
No assumptions beyond valid, aligned inputs of the documented types.
Limitations
No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.
Python API
import evalsuite as es
es.language_id_accuracy(["en", "fr", "hi"], ["en", "fr", "en"])References
- Lui M, Baldwin T. langid.py: an off-the-shelf language identification tool. ACL Demos. 2012:25-30.