Skip to content
EvalSuite
Documentation menu

Code generation

Static-analysis violation rate

Implementedcode.static_analysis_violation_rate

Definition

Linter / static-analyser findings in generated code: violations per 1,000 lines (the value) and the share of programs with at least one finding, from counts your analyser (ruff, pylint, ESLint) reported.

Formula

1000 · Σ violations / Σ lines

Range: [0, ∞)

Inputs and outputs

  • violations: see the signature of es.static_analysis_violation_rate
  • lines: see the signature of es.static_analysis_violation_rate

Returns: MetricResult (value plus counts, intervals and breakdowns in params)

Assumptions

No assumptions beyond valid, aligned inputs of the documented types.

Limitations

No metric-specific limitations are documented yet. Interpret the value alongside the task, data, and other metrics.

Python API

PythonSince v0.5.0
import evalsuite as es

es.static_analysis_violation_rate(violations=[2, 0], lines=[120, 80])

References

  1. Liu Y, Le-Cong T, Widyasari R, et al. Refining ChatGPT-generated code: characterizing and mitigating code quality issues. ACM TOSEM. 2024;33(5):1-26.

Implementation status