tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models
tiktoken is a fast BPE tokeniser for use with OpenAI's models
To install this package, run one of the following:
Summary
tiktoken is a fast BPE tokeniser for use with OpenAI's models
Information Last Updated
Aug 20, 2025 at 15:50
License
MIT
Total Downloads
57
Platforms