Show HN: Quicktok, an exact BPE tokenizer 7x faster than tiktoken
**Brief summary (2‑3 sentences)** Quicktok is an open‑source BPE tokenizer that reproduces the exact output of OpenAI’s tiktoken but runs up to seven times fast
Original source**Brief summary (2‑3 sentences)** Quicktok is an open‑source BPE tokenizer that reproduces the exact output of OpenAI’s tiktoken but runs up to seven times fast
Original source