tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models
Installation
To install this package, run one of the following:
Usage Tracking
0.12.0
0.11.0
0.9.0
0.8.0
0.7.0
5 / 8 versions selected
Downloads (Last 6 months): 0
About
Summary
tiktoken is a fast BPE tokeniser for use with OpenAI's models
Last Updated
Oct 7, 2025 at 17:15
License
MIT
Total Downloads
1.7M
Supported Platforms
linux-aarch64
linux-ppc64le
macOS-arm64
win-64
macOS-64
linux-64
GitHub Repository
https://github.com/openai/tiktoken