46 stories tagged llm-benchmarks, newest first.
160 points by rzk | 56 comments
GitHub repo with code and paper for GVS5H, a training-free self-orchestration harness letting Qwen3.8-27B instances match Claude Fable 5 on hard LiveCodeBench problems.
103 points by theanonymousone | 23 comments
Artificial Analysis model page for Kimi K3.1 Pro, showing its Intelligence Index score of 55, speed, pricing, verbosity, and benchmark results.
159 points by theanonymousone | 58 comments
Artificial Analysis model page benchmarking DeepSeek V4.1 Flash 1017 (Max) on intelligence, speed, and price.
229 points by __cayenne__ | 103 comments
StarSkirmish Bench, a benchmark where LLMs play StarCraft: Brood War by writing code, showing gameplay and results for flagship models.
286 points by dhorthy | 112 comments
A GitHub markdown writeup from HumanLayer reporting October 2026 SlopCodeBench results for coding models including Opus 5.5, Astra, and gpt-5.5.
127 points by pcwelder | 116 comments
Blog post examining why SVGs generated by various LLMs for the prompt "pelican riding a bicycle" all look strikingly similar, with a gallery comparing models.
67 points by lasu | 13 comments
Parsimony is an open-source benchmark and leaderboard measuring how much net code AI models add when solving tasks, counting syntax-level coding units from published benchmark patches.
5 points by luu | 1 comments
A benchmark site measuring whether Codex's "fast mode" priority tier delivers its promised 2x speedup, showing a measured 1.45x throughput gain.
5 points by prithvi2206 | 1 comments
Landing page for DoGBench, a benchmark evaluating AI agents that write and maintain user-facing software documentation, with leaderboard, paper, code and dataset.
71 points by theanonymousone | 29 comments
Artificial Analysis model page benchmarking Google's Gemini 4 Argon (High) on intelligence index, speed, and price.
46 points by theanonymousone | 48 comments
Artificial Analysis article reporting that GPT-6.1 Sol replaces GPT-6 Sol after 7 days, scoring near GPT-6 Astra on its Intelligence Index at under a quarter of the cost.
858 points by bryan0 | 367 comments
GitHub repository for livenerf, a benchmark that tracks AI model capability over time after release to detect whether models like Claude Opus have been nerfed.
99 points by hp6 | 40 comments
Tiny AI Arena shows a leaderboard and match history of four LLMs fighting on an 8×8 grid, with replays viewable for each match.
19 points by tob_scott_a | 4 comments
Blog post by Suha Sabi Hussain critiquing 1Password Off-by-1 Labs' FLAWED report on frontier model vulnerability patches and its implications for industry research.
138 points by Retro_Dev | 81 comments
Artificial Analysis benchmark page for Inception's Mercury 2.5 model, covering intelligence index, output speed of 770 tokens per second, cost, and verbosity rankings.
252 points by plurby | 216 comments
A site called DrivingBench claiming the GPT-6 Astra model can drive a car; no page content was captured to confirm details.
320 points by theanonymousone | 101 comments
Artificial Analysis model page benchmarking Anthropic's Claude Opus 5.5 (max effort) on intelligence, speed, and price.
141 points by theanonymousone | 60 comments
Artificial Analysis benchmark page for Xiaomi's open-weights MiMo-V2.6-Pro model, covering intelligence index, output speed, cost and verbosity rankings.
322 points by benswerd | 143 comments
A report benchmarking large language models playing StarCraft: Brood War against each other, with key takeaways on which models performed best.
407 points by iopapa | 218 comments
EEBench blog post on whether AI models can design circuit boards, describing a benchmark for evaluating the quality of circuits generated by models working in KiCad.
813 points by colinprince | 325 comments
Felony Bench is a benchmark site charting how willing AI models are to assist with illegal requests, with scores by model and evaluator.
761 points by tosh | 454 comments
ARC Prize verified results page for DeepSeek V4 Flash 0731, reporting its ARC-AGI-1 and ARC-AGI-2 scores and cost per task on a leaderboard chart.
458 points by apitman | 291 comments
Artificial Analysis homepage offering independent benchmarks and rankings of AI models and providers, here linked to its agentic intelligence index view.
571 points by delichon | 389 comments
Andrej Karpathy's X post describing Claude Opus 5 writing 5,500 lines of three.js code over two hours to procedurally render the opening of Lord of the Rings.
530 points by theanonymousone | 289 comments
An Artificial Analysis page meant to benchmark DeepSeek V4 Flash's intelligence, performance and price, which returned a 404 Page not found error.
367 points by dhorthy | 100 comments
A markdown report in HumanLayer's GitHub repo benchmarking Anthropic's Claude Opus 5 model on the SlopCodeBench coding benchmark.
358 points by aarondong | 211 comments
Artificial Analysis page comparing AI models on intelligence, output speed, latency, price and context window, showing Claude Opus 5 at the top for intelligence.
632 points by dcastm | 237 comments
Blog post investigating whether AI labs are optimizing their LLMs for Simon Willison's 'SVG of a pelican riding a bicycle' informal benchmark.
816 points by piotrgrabowski | 416 comments
Fireworks AI blog post claiming the Kimi K3 model is competitive with Fable and that combining Kimi K3 with Fable achieves state-of-the-art results.
1038 points by jms703 | 476 comments
Semgrep blog post reporting that the GLM 5.2 model outperformed Claude in Semgrep's cybersecurity benchmarks.
845 points by himata4113 | 411 comments
Artificial Analysis article reporting that Z ai's GLM-5.2 scores 51 on its Intelligence Index, leading open weights models ahead of MiniMax-M3 and DeepSeek V4 Pro.
373 points by bugvader | 217 comments
Endor Labs blog post benchmarking Claude Fable 5 on 200 real-world coding tasks for its Agent Security League, reporting middling functional and security results.
365 points by yogthos | 184 comments
A Runtimewire head-to-head article claiming DeepSeek V4 Pro outperforms GPT-5.5 Pro on precision tasks like instruction following and schema matching.
333 points by GodelNumbering | 124 comments
GitHub repository for Dirac, an open-source coding agent focused on efficiency and context curation, using hash-anchored edits, parallel operations and AST manipulation.
332 points by kmdupree | 172 comments
OpenAI research post explaining that SWE-bench Verified is increasingly contaminated and no longer measures frontier coding ability, recommending SWE-bench Pro instead.
340 points by felix089 | 403 comments
Opper blog post testing 53 AI models on the 'car wash 50 meters away, walk or drive?' logic question, with repeated runs and a human baseline.
720 points by qwesr123 | 330 comments
Margin Lab tracker that runs daily SWE-Bench-Pro benchmarks on Claude Code with Opus 4.5 to statistically detect performance degradation.
325 points by cheeseblubber | 268 comments
AI Trade Arena research post describing a backtested experiment giving five LLMs $100K each to trade stocks over eight months.
1283 points by waxpancake | 361 comments
A web page showing analog clocks rendered every minute by nine different AI models, displaying their often distorted attempts side by side.
397 points by pseudolus | 186 comments
Oxford Internet Institute news item on a study finding that most of 445 LLM benchmarks lack rigorous methods and clear definitions of what they measure.
372 points by emschwartz | 200 comments
Sean Goedecke's blog post arguing that OpenAI's open-weight gpt-oss models resemble Microsoft's Phi series, strong on benchmarks but weak on out-of-domain knowledge.
505 points by rickcarlino | 140 comments
Penrose's AccountingBench page presents a benchmark of LLM agents closing a company's books over months, shown through a simulated accounting reconciliation interface.
866 points by swyx | 216 comments
Simon Willison's writeup of his AI Engineer World's Fair keynote reviewing six months of LLM releases, compared by having each model draw a pelican on a bicycle.
474 points by paulpauper | 375 comments
LessWrong linkpost essay arguing recent AI model improvements feel overstated, based on experience building an AI codebase security-scanning startup.
331 points by joshwa | 113 comments
arXiv paper introducing SWE-Bench+, an empirical analysis revealing data quality issues in the SWE-bench coding benchmark for LLMs, such as solution leakage and weak tests.
430 points by parrt | 193 comments
Page introducing Needle in a Needlestack (NIAN), a benchmark testing LLM attention over long contexts of limericks, showing GPT-4o's strong results versus GPT-4 Turbo and Claude-3.