tokenizers
Fast State-of-the-Art Tokenizers optimized for Research and Production
| Name | Type | Version | Platform | Labels | Updated | Size | Downloads | Actions |
|---|
osx-arm64/tokenizers-0.22.2-py314h402bb5a_0.tar.bz2 | conda | 0.22.2 | osx-arm64 | main | 19 days ago | 2.57 MB | 0 | |
osx-arm64/tokenizers-0.22.2-py313h99e1e97_0.tar.bz2 | conda | 0.22.2 | osx-arm64 | main | 19 days ago | 2.58 MB | 3 | |
osx-arm64/tokenizers-0.22.2-py312h99e1e97_0.tar.bz2 | conda | 0.22.2 | osx-arm64 | main | 19 days ago | 2.57 MB | 3 | |
osx-arm64/tokenizers-0.22.2-py311h99e1e97_0.tar.bz2 | conda | 0.22.2 | osx-arm64 | main | 19 days ago | 2.58 MB | 3 | |
linux-aarch64/tokenizers-0.22.2-py314h178e3a9_0.tar.bz2 | conda | 0.22.2 | linux-aarch64 | main | 19 days ago | 2.87 MB | 0 | |
osx-arm64/tokenizers-0.22.2-py310h99e1e97_0.tar.bz2 | conda | 0.22.2 | osx-arm64 | main | 19 days ago | 2.57 MB | 3 | |
linux-aarch64/tokenizers-0.22.2-py313hb555b49_0.tar.bz2 | conda | 0.22.2 | linux-aarch64 | main | 19 days ago | 2.86 MB | 3 | |
linux-aarch64/tokenizers-0.22.2-py312hb555b49_0.tar.bz2 | conda | 0.22.2 | linux-aarch64 | main | 19 days ago | 2.87 MB | 3 | |
linux-64/tokenizers-0.22.2-py314h9d79900_0.tar.bz2 | conda | 0.22.2 | linux-64 | main | 19 days ago | 2.91 MB | 7 | |
linux-aarch64/tokenizers-0.22.2-py311hb555b49_0.tar.bz2 | conda | 0.22.2 | linux-aarch64 | main | 19 days ago | 2.87 MB | 3 | |
linux-aarch64/tokenizers-0.22.2-py310hb555b49_0.tar.bz2 | conda | 0.22.2 | linux-aarch64 | main | 19 days ago | 2.87 MB | 3 | |
linux-64/tokenizers-0.22.2-py310h4b931ed_0.tar.bz2 | conda | 0.22.2 | linux-64 | main | 19 days ago | 2.88 MB | 11 | |
linux-64/tokenizers-0.22.2-py311h4b931ed_0.tar.bz2 | conda | 0.22.2 | linux-64 | main | 19 days ago | 2.91 MB | 10 | |
linux-64/tokenizers-0.22.2-py313h4b931ed_0.tar.bz2 | conda | 0.22.2 | linux-64 | main | 19 days ago | 2.87 MB | 10 | |
linux-64/tokenizers-0.22.2-py312h4b931ed_0.tar.bz2 | conda | 0.22.2 | linux-64 | main | 19 days ago | 2.91 MB | 10 | |
win-64/tokenizers-0.22.2-py314h3230b74_0.tar.bz2 | conda | 0.22.2 | win-64 | main | 19 days ago | 2.46 MB | 6 | |
win-64/tokenizers-0.22.2-py311h4510a99_0.tar.bz2 | conda | 0.22.2 | win-64 | main | 19 days ago | 2.5 MB | 9 | |
win-64/tokenizers-0.22.2-py310h4510a99_0.tar.bz2 | conda | 0.22.2 | win-64 | main | 19 days ago | 2.49 MB | 9 | |
win-64/tokenizers-0.22.2-py313h4510a99_0.tar.bz2 | conda | 0.22.2 | win-64 | main | 19 days ago | 2.47 MB | 9 | |
win-64/tokenizers-0.22.2-py312h4510a99_0.tar.bz2 | conda | 0.22.2 | win-64 | main | 19 days ago | 2.49 MB | 9 | |
osx-arm64/tokenizers-0.22.1-py314h402bb5a_0.tar.bz2 | conda | 0.22.1 | osx-arm64 | main | Dec 11, 2025, 09:41 AM | 2.62 MB | 1 | |
win-64/tokenizers-0.22.1-py313h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:10 AM | 2.51 MB | 13 | |
win-64/tokenizers-0.22.1-py312h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:10 AM | 2.49 MB | 13 | |
win-64/tokenizers-0.22.1-py311h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:10 AM | 2.5 MB | 13 | |
win-64/tokenizers-0.22.1-py310h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:10 AM | 2.51 MB | 13 |