Calibration

Brier score

Definition

Mean squared difference between predicted probabilities and binary outcomes.

Formula

Range: [0, 1]

Inputs and outputs

• y_true: array of binary labels {0, 1}
• y_prob: predicted probability of the positive class in [0, 1]

Returns: float

Assumptions

No assumptions beyond valid, aligned inputs of the documented types.

Limitations

• Mixes calibration and discrimination; a low score alone does not show good calibration.

Python API

import evalsuite as es

es.brier_score(y_true, y_prob)

References

1. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3.

Implementation status