Skip to content
EvalSuite
Documentation menu

Multilingual

Code-switching robustness

Implementedmultilingual.code_switching_robustness

Definition

Accuracy on code-switched inputs (mixing languages within an utterance) compared with the same content in one language: code-switched accuracy, drop and flip rate.

Formula

acc_cs; drop = acc_mono − acc_cs

Range: [0, 1]

Inputs and outputs

  • monolingual_correct: see the signature of es.code_switching_robustness
  • code_switched_correct: see the signature of es.code_switching_robustness

Returns: MetricResult (value plus counts, intervals and breakdowns in params)

Assumptions

No assumptions beyond valid, aligned inputs of the documented types.

Limitations

No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.

Python API

PythonSince v0.5.0
import evalsuite as es

es.code_switching_robustness(monolingual_correct, code_switched_correct)

References

  1. Aguilar G, Kar S, Solorio T. LinCE: a centralized benchmark for linguistic code-switching evaluation. LREC. 2020:1803-1813.

Implementation status