Measuring AI Ability to Complete Long Tasks(metr.org)

222 points by spicypete 10 months ago | 175 comments

We are changing our developer productivity experiment design(metr.org)

72 points by ej88 8 months ago | 45 comments

ML-Enhanced Code Completion Improves Developer Productivity (2022)(research.google)

17 points by ddadon10 1 year ago | 3 comments

AI slows down open source developers. Peter Naur can teach us why(johnwhiles.com)

303 points by jwhiles 1 year ago | 187 comments

NixOS and reproducible builds could have detected the xz backdoor(luj.fr)

260 points by birdculture 1 year ago | 112 comments

What makes developers productive?(jeremymikkola.com)

130 points by piinbinary 3 years ago | 113 comments

AI tools are spotting errors in research papers(nature.com)

555 points by kgwgk 1 year ago | 200 comments

Nature news article on a growing movement using AI and large language models to detect errors in research papers, sparked by a flawed black plastic utensils study.

Exploiting the most prominent AI agent benchmarks(rdi.berkeley.edu)

561 points by Anon84 6 months ago | 136 comments

UC Berkeley RDI blog post describing how researchers exploited major AI agent benchmarks like SWE-bench and Terminal-Bench to score highly without solving tasks, and proposed fixes.

AI models may be capable of self-replicating without access to a datacenter(anthropic.com)

36 points by artninja1988 14 days ago | 26 comments

Anthropic Frontier Red Team post reporting experiments suggesting small AI models may soon self-replicate on commercial IoT devices without datacenter access.

Study identifies weaknesses in how AI systems are evaluated(oii.ox.ac.uk)

397 points by pseudolus 11 months ago | 186 comments

Oxford Internet Institute news item on a study finding that most of 445 LLM benchmarks lack rigorous methods and clear definitions of what they measure.

MIT Study Finds AI Use Reprograms the Brain, Leading to Cognitive Decline(publichealthpolicyjournal.com)

438 points by cainxinth 1 year ago | 432 comments

Article reporting on an MIT study claiming that using AI tools like ChatGPT alters brain activity and leads to cognitive decline.

One Month Without AI(blog.bustikiller.com)

148 points by saibotk 1 month ago | 153 comments

Personal blog post about a developer spending a month without AI tools at work, reflecting on how AI use made them lazier and a worse developer.