Show HN: TokenDagger – A tokenizer faster than OpenAI's Tiktoken (github.com) | HN Screenshots

YHacker Newsscreenshots

Screenshot of github.com: Show HN: TokenDagger – A tokenizer faster than OpenAI's Tiktoken

First screen, captured 1 year ago.

Looks similar

Screenshot of github.com: GigaToken: ~1000x faster Language model tokenization

GigaToken: ~1000x faster Language model tokenization (github.com)

585 points by syrusakbary 3 months ago | 116 comments

GitHub repository for gigatoken, an open-source library for fast language model tokenization at GB/s speeds, MIT licensed.

tokenization llm nlp rust

Screenshot of github.com: Show HN: Tokenflood – simulate arbitrary loads on instruction-tuned LLMs

Show HN: Tokenflood – simulate arbitrary loads on instruction-tuned LLMs (github.com)

18 points by twerkmeister 11 months ago | 0 comments

Screenshot of github.com: Publisher: The Malloy Semantic Model Server

Publisher: The Malloy Semantic Model Server (github.com)

16 points by cpard 1 year ago | 0 comments

Screenshot of github.com: Show HN: Tegon: Open-source alternative to Jira, Linear

Show HN: Tegon: Open-source alternative to Jira, Linear (github.com)

139 points by harshithmul 2 years ago | 106 comments

Screenshot of github.com: Show HN: Txt2plotter – True centerline vectors from Flux.2 for pen plotters

Show HN: Txt2plotter – True centerline vectors from Flux.2 for pen plotters (github.com)

29 points by tsanummy 9 months ago | 7 comments

Screenshot of github.com: Zlob.h 100% POSIX and glibc compatible globbing lib that is faste and better

Zlob.h 100% POSIX and glibc compatible globbing lib that is faste and better (github.com)

17 points by neogoose 8 months ago | 10 comments