#evaluation on Hacker News

2 stories tagged evaluation, newest first.

Screenshot of github.com: DishwasherBench – Can Frontier Models Stack a Dishwasher?

DishwasherBench – Can Frontier Models Stack a Dishwasher?(github.com)

74 points by mlukewizard 13 days ago | 41 comments

GitHub repository for DishwasherBench, a dishwasher-loading game on a physical simulation used to benchmark language models, also playable in a browser.

llm-benchmark ai-agents physics-simulation browser-game

Screenshot of kaggle.com: Evidence of inconsistencies in evaluation process and selection of winners

Evidence of inconsistencies in evaluation process and selection of winners (kaggle.com)

446 points by twerkmeister 3 months ago | 276 comments

A Kaggle discussion post alleging inconsistencies in the judging and winner selection of the 'Measuring Progress Toward AGI' competition, citing the Medley Bench grand prize winner.

kaggle agi benchmarks ml-competitions

#evaluation on Hacker News

DishwasherBench – Can Frontier Models Stack a Dishwasher?(github.com)

Evidence of inconsistencies in evaluation process and selection of winners(kaggle.com)

Evidence of inconsistencies in evaluation process and selection of winners (kaggle.com)