Separating signal from noise in coding evaluations

T1Covered by 1 source · first reported by OpenAI News at 08 Jul, 13:00 UTC

A new analysis from OpenAI reveals issues in SWE-Bench Pro, a popular coding benchmark, raising concerns about reliability and accuracy in evaluating AI models.