You are connecting from Lake Geneva Public Library, please login or register to take advantage of your institution's Ground News Plan.
Published 10 hours ago • loading... • Updated 10 hours ago
OpenAI Reports that "Approximately 30% of SWE-Bench Pro Is Broken," Highlighting Numerous Flaws in the Benchmark Used to Measure AI Coding Capabilities.
On July 8, 2026, OpenAI reported that a detailed investigation of its AI coding capability benchmark, "SWE-Bench Pro," revealed that approximately 30% of the tasks had problems that prevented them from being evaluated properly. Read more...
This story is only covered by news sources that have yet to be evaluated by the independent media monitoring agencies we use to assess the quality and reliability of news outlets on our platform. Learn more here.
On July 8, 2026, OpenAI reported that a detailed investigation of its AI coding capability benchmark, "SWE-Bench Pro," revealed that approximately 30% of the tasks had problems that prevented them from being evaluated properly. Read more...