Looks similar
Understanding GPT tokenizers(simonwillison.net)
380 points by simonw | 110 comments
Simon Willison blog post explaining how GPT tokenizers convert text to integer tokens, with an Observable notebook tool for encoding, decoding and searching tokens.
Chinchilla’s death(espadrine.github.io)
156 points by KolmogorovComp | 37 comments
Understanding ChatGPT Work(simonwillison.net)
281 points by gmays | 149 comments






