Statistical tests
McNemar's test
Implemented
statistics.mcnemarDefinition
Compares two classifiers on the same observations using their discordant predictions.
Formula
χ² = (b − c)² / (b + c)
Inputs and outputs
- y_true
- predictions of model A
- predictions of model B
Returns: StatisticalTestResult
Assumptions
No assumptions beyond valid, aligned inputs of the documented types.
Limitations
- Use the exact binomial version when b + c is small.
Python API
import evalsuite as es
es.mcnemar_test(y_true, y_pred_a, y_pred_b)References
- McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 153–157.