11 stories tagged ai-benchmarks, newest first.
558 points by speckx | 495 comments
Mercor research page presenting a Benchmark Map that measures how well existing AI benchmarks cover tasks across American jobs.
85 points by merksittich | 64 comments
Epoch AI report by David Owen presenting InnovationEval, finding early evidence that AI models cannot yet automate AI R&D or match human algorithmic innovation.
1072 points by jonotime | 938 comments
A personal blog post arguing that DeepSeek 4.1 Flash performs like frontier models such as Opus at a fraction of the cost, with cost-per-task benchmark charts.
188 points by stabbles | 236 comments
Goalposts is a site collecting past Hacker News comments that set challenges for AI, letting visitors vote on which ones AI has since met.
1171 points by meredydd | 1120 comments
A declaration by mathematicians arguing that AI companies' use of math problems as benchmarks harms mathematics and the mathematical community, inviting endorsements.
561 points by Anon84 | 136 comments
UC Berkeley RDI blog post describing how researchers exploited major AI agent benchmarks like SWE-bench and Terminal-Bench to score highly without solving tasks, and proposed fixes.
447 points by mustaphah | 136 comments
GitHub issue on SWE-bench reporting loopholes where agents can access future repository state, like git history and commit messages, that leak solutions during evaluation.
440 points by Davidzheng | 654 comments
X post by OpenAI researcher Alexander Wei announcing an experimental reasoning LLM achieved gold medal-level performance at the 2025 International Math Olympiad.
371 points by owendarko | 596 comments
Ask HN thread inviting users to share AI prompts that no current model can solve, aiming to build a crowdsourced collection of hard prompts.
429 points by wujerry2000 | 156 comments
A LessWrong shortform post by user meemi, linked as reporting that the FrontierMath AI math benchmark was funded by OpenAI.
527 points by mikeknoop | 266 comments
Blog post announcing ARC Prize, a $1M+ public competition by Mike Knoop and François Chollet to beat and open-source a solution to the ARC-AGI benchmark.