tokenizers
Fast State-of-the-Art Tokenizers optimized for Research and Production
| Name | Type | Version | Platform | Labels | Updated | Size | Downloads | Actions |
|---|
win-64/tokenizers-0.22.2-py314h3230b74_0.tar.bz2 | conda | 0.22.2 | win-64 | main | 5 days ago | 2.46 MB | 5 | |
win-64/tokenizers-0.22.2-py313h4510a99_0.tar.bz2 | conda | 0.22.2 | win-64 | main | 5 days ago | 2.47 MB | 3 | |
win-64/tokenizers-0.22.2-py312h4510a99_0.tar.bz2 | conda | 0.22.2 | win-64 | main | 5 days ago | 2.49 MB | 6 | |
win-64/tokenizers-0.22.2-py311h4510a99_0.tar.bz2 | conda | 0.22.2 | win-64 | main | 5 days ago | 2.5 MB | 3 | |
win-64/tokenizers-0.22.2-py310h4510a99_0.tar.bz2 | conda | 0.22.2 | win-64 | main | 5 days ago | 2.49 MB | 3 | |
osx-arm64/tokenizers-0.22.2-py314h402bb5a_0.tar.bz2 | conda | 0.22.2 | osx-arm64 | main | 5 days ago | 2.57 MB | 4 | |
osx-arm64/tokenizers-0.22.2-py313h99e1e97_0.tar.bz2 | conda | 0.22.2 | osx-arm64 | main | 5 days ago | 2.58 MB | 4 | |
osx-arm64/tokenizers-0.22.2-py312h99e1e97_0.tar.bz2 | conda | 0.22.2 | osx-arm64 | main | 5 days ago | 2.57 MB | 4 | |
osx-arm64/tokenizers-0.22.2-py311h99e1e97_0.tar.bz2 | conda | 0.22.2 | osx-arm64 | main | 5 days ago | 2.58 MB | 5 | |
osx-arm64/tokenizers-0.22.2-py310h99e1e97_0.tar.bz2 | conda | 0.22.2 | osx-arm64 | main | 5 days ago | 2.57 MB | 4 | |
linux-aarch64/tokenizers-0.22.2-py314h178e3a9_0.tar.bz2 | conda | 0.22.2 | linux-aarch64 | main | 5 days ago | 2.87 MB | 3 | |
linux-aarch64/tokenizers-0.22.2-py313hb555b49_0.tar.bz2 | conda | 0.22.2 | linux-aarch64 | main | 5 days ago | 2.86 MB | 3 | |
linux-aarch64/tokenizers-0.22.2-py312hb555b49_0.tar.bz2 | conda | 0.22.2 | linux-aarch64 | main | 5 days ago | 2.87 MB | 3 | |
linux-aarch64/tokenizers-0.22.2-py311hb555b49_0.tar.bz2 | conda | 0.22.2 | linux-aarch64 | main | 5 days ago | 2.87 MB | 3 | |
linux-64/tokenizers-0.22.2-py314h9d79900_0.tar.bz2 | conda | 0.22.2 | linux-64 | main | 5 days ago | 2.91 MB | 4 | |
linux-aarch64/tokenizers-0.22.2-py310hb555b49_0.tar.bz2 | conda | 0.22.2 | linux-aarch64 | main | 5 days ago | 2.87 MB | 3 | |
linux-64/tokenizers-0.22.2-py312h4b931ed_0.tar.bz2 | conda | 0.22.2 | linux-64 | main | 5 days ago | 2.91 MB | 4 | |
linux-64/tokenizers-0.22.2-py313h4b931ed_0.tar.bz2 | conda | 0.22.2 | linux-64 | main | 5 days ago | 2.87 MB | 4 | |
linux-64/tokenizers-0.22.2-py310h4b931ed_0.tar.bz2 | conda | 0.22.2 | linux-64 | main | 5 days ago | 2.88 MB | 4 | |
linux-64/tokenizers-0.22.2-py311h4b931ed_0.tar.bz2 | conda | 0.22.2 | linux-64 | main | 5 days ago | 2.91 MB | 5 | |
osx-arm64/tokenizers-0.22.1-py314h402bb5a_0.tar.bz2 | conda | 0.22.1 | osx-arm64 | main | 23 days ago | 2.62 MB | 4 | |
win-64/tokenizers-0.22.1-py313h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:09 AM | 2.51 MB | 12 | |
win-64/tokenizers-0.22.1-py312h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:09 AM | 2.49 MB | 12 | |
win-64/tokenizers-0.22.1-py311h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:09 AM | 2.5 MB | 11 | |
win-64/tokenizers-0.22.1-py310h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:09 AM | 2.51 MB | 10 |