First screen, captured 12 hours ago. Hover to scroll the page, or open the video.

Zombie stats: Where the numbers AI quotes come from

What HN thinks

An AI-written post about AI-made-up stats, critiqued by humans quoting made-up stats about made-up stats.

AI should trace sources, mostly doesn't
Many wish AI did the drudge work of verifying every claim back to a source; instead products deliver fast unsourced answers, and users often don't realise it.
It's the product, not the tech
Some argue well-prompted LLMs with retrieval tools can cite properly, though others counter that RAG doesn't guarantee faithful use of sources and citations often don't say what's claimed.
Humans were already like this
Joking and serious replies note humans repeat made-up stats and citogenesis too, as with the oft-repeated '70% of websites use PHP' figure, and peer-reviewed sources are often wrong.

Consensus: Most agree AI-quoted statistics and inline citations shouldn't be trusted without checking, and many dismiss the article itself as AI-written slop.

Recommended in the comments

Wikipedia (service): Held up as a cheaper, higher-quality model for source-backed information at scale.
pgvector (library): Part of a simple self-built RAG setup for answering only from a trusted corpus.
llama.cpp (tool): Suggested for local chunking, labeling, embedding and reranking in a DIY RAG pipeline.
Veracity ↗ (product): Offered by its maker for detecting hallucinated citations and verifying claims.
HTTP Archive ↗ (service): Used to check the PHP usage claim against real crawl data.