Understanding GPT tokenizers(simonwillison.net)
380 points by simonw | 110 comments
Simon Willison blog post explaining how GPT tokenizers convert text to integer tokens, with an Observable notebook tool for encoding, decoding and searching tokens.
380 points by simonw | 110 comments
Simon Willison blog post explaining how GPT tokenizers convert text to integer tokens, with an Observable notebook tool for encoding, decoding and searching tokens.
21 points by EfrainGaray | 9 comments
Blog post benchmarking TabPFN and TabICL foundation models against tuned XGBoost on fourteen tabular datasets, finding the no-training models won all fourteen.
43 points by abelanger | 6 comments
64 points by pantalaimon | 19 comments