Understanding GPT tokenizers(simonwillison.net)

380 points by simonw 3 years ago | 110 comments

Simon Willison blog post explaining how GPT tokenizers convert text to integer tokens, with an Observable notebook tool for encoding, decoding and searching tokens.

The soloists and populars of the software world(mihaiolteanu.me)

5 points by molteanu 3 years ago | 0 comments

Should LLMs just treat text content as an image?(seangoedecke.com)

159 points by ingve 1 year ago | 94 comments

The Ken Thompson Hack(wiki.c2.com)

98 points by stephc_int13 2 years ago | 27 comments

All your OpenCodes belong to us(johncodes.com)

33 points by jpmcb 9 months ago | 7 comments

OpenTelemetry Tracing in < 200 lines of code(jeremymorrell.dev)

213 points by masterj 2 years ago | 28 comments