tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
- Overview
- Files 160
- Labels 1
- Badges
Installation
To install this package, run one of the following:
Usage Tracking
0.13.2
0.13.1
0.13.0
0.13.0.dev0
0.12.1
5 / 8 versions selected
Total downloads: 0
About
Summary
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Information Last Updated
Mar 25, 2025 at 16:27
License
Apache License 2.0
Total Downloads
428.2K
Platforms
Linux 64 Version: 0.13.2
macOS 64 Version: 0.13.2
Win 64 Version: 0.13.2