Using LLM for confidence scoring is a bad idea(rumple.github.io)
185 points by jimauthors | 100 comments
Blog post arguing that LLM-generated confidence scores are uncalibrated text continuations, and describing how to build a calibrated confidence rule from recorded outcomes instead.