Can We Trust AI Benchmarks? A Review of Current Issues in AI Evaluation (arxiv.org)

Screenshot of arxiv.org: Can We Trust AI Benchmarks? A Review of Current Issues in AI Evaluation

First screen, captured 1 year ago.

Looks similar

Screenshot of arxiv.org: Antagonistic AI

Antagonistic AI (arxiv.org)

65 points by drcwpl 2 years ago | 38 comments

Screenshot of arxiv.org: Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs

Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs (arxiv.org)

512 points by tiny-automates 8 months ago | 329 comments

arXiv paper introducing a 40-scenario benchmark measuring how autonomous AI agents violate ethical, legal, or safety constraints when pressured by KPI-driven outcomes.

ai-agents ai-safety benchmarks alignment

Screenshot of arxiv.org: AI for Scientific Search

AI for Scientific Search (arxiv.org)

108 points by omarsar 1 year ago | 26 comments

Screenshot of arxiv.org: When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation

When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation (arxiv.org)

99 points by doppp 2 months ago | 99 comments

Screenshot of arxiv.org: Kosmos: An AI Scientist for Autonomous Discovery

Kosmos: An AI Scientist for Autonomous Discovery (arxiv.org)

45 points by belter 11 months ago | 9 comments

Screenshot of arxiv.org: Transformers Are Bayesian Networks

Transformers Are Bayesian Networks (arxiv.org)

27 points by Anon84 7 months ago | 16 comments

Looks similar

Antagonistic AI(arxiv.org)

Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs(arxiv.org)

AI for Scientific Search(arxiv.org)

When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation(arxiv.org)

Kosmos: An AI Scientist for Autonomous Discovery(arxiv.org)

Transformers Are Bayesian Networks(arxiv.org)