9 stories tagged swe-bench, newest first.
67 points by lasu | 13 comments
Parsimony is an open-source benchmark and leaderboard measuring how much net code AI models add when solving tasks, counting syntax-level coding units from published benchmark patches.
332 points by kmdupree | 172 comments
OpenAI research post explaining that SWE-bench Verified is increasingly contaminated and no longer measures frontier coding ability, recommending SWE-bench Pro instead.
561 points by Anon84 | 136 comments
UC Berkeley RDI blog post describing how researchers exploited major AI agent benchmarks like SWE-bench and Terminal-Bench to score highly without solving tasks, and proposed fixes.
593 points by zixuanlimit | 240 comments
Z.ai research blog post announcing GLM-5.1, a flagship model for agentic engineering with stronger coding and long-horizon task performance on benchmarks like SWE-Bench Pro.
720 points by qwesr123 | 330 comments
Margin Lab tracker that runs daily SWE-Bench-Pro benchmarks on Claude Code with Opus 4.5 to statistically detect performance degradation.
404 points by pretext | 217 comments
Z.ai research blog post announcing GLM-4.7, a language model with improved agentic coding, terminal tasks, and UI generation, with benchmark results.
447 points by mustaphah | 136 comments
GitHub issue on SWE-bench reporting loopholes where agents can access future repository state, like git history and commit messages, that leak solutions during evaluation.
801 points by datadrivenangel | 173 comments
A satirical dictionary-style page defining "vibechart" as charting what you want to see, illustrated with misleading bar charts from OpenAI's GPT-5 release.
331 points by joshwa | 113 comments
arXiv paper introducing SWE-Bench+, an empirical analysis revealing data quality issues in the SWE-bench coding benchmark for LLMs, such as solution leakage and weak tests.