tokenizers
Fast State-of-the-Art Tokenizers optimized for Research and Production
- Overview
- Files 288
- Labels 1
- Badges
| Name | Type | Version | Platform | Labels | Updated | Size | Downloads | Actions |
|---|
osx-arm64/tokenizers-0.22.1-py314h402bb5a_0.tar.bz2 | conda | 0.22.1 | osx-arm64 | main | 3 days ago | 2.62 MB | 0 | |
win-64/tokenizers-0.22.1-py313h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:10 AM | 2.51 MB | 9 | |
win-64/tokenizers-0.22.1-py312h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:10 AM | 2.49 MB | 9 | |
win-64/tokenizers-0.22.1-py311h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:10 AM | 2.5 MB | 9 | |
win-64/tokenizers-0.22.1-py310h4510a99_0.tar.bz2 | conda | 0.22.1 | win-64 | main | Nov 5, 2025, 09:10 AM | 2.51 MB | 9 | |
osx-arm64/tokenizers-0.22.1-py313h99e1e97_0.tar.bz2 | conda | 0.22.1 | osx-arm64 | main | Nov 5, 2025, 09:10 AM | 2.59 MB | 1 | |
osx-arm64/tokenizers-0.22.1-py312h99e1e97_0.tar.bz2 | conda | 0.22.1 | osx-arm64 | main | Nov 5, 2025, 09:10 AM | 2.59 MB | 1 | |
osx-arm64/tokenizers-0.22.1-py310h99e1e97_0.tar.bz2 | conda | 0.22.1 | osx-arm64 | main | Nov 5, 2025, 09:10 AM | 2.62 MB | 17 | |
osx-arm64/tokenizers-0.22.1-py311h99e1e97_0.tar.bz2 | conda | 0.22.1 | osx-arm64 | main | Nov 5, 2025, 09:10 AM | 2.63 MB | 1 | |
linux-aarch64/tokenizers-0.22.1-py313hb555b49_0.tar.bz2 | conda | 0.22.1 | linux-aarch64 | main | Nov 5, 2025, 09:10 AM | 2.88 MB | 3 | |
linux-aarch64/tokenizers-0.22.1-py312hb555b49_0.tar.bz2 | conda | 0.22.1 | linux-aarch64 | main | Nov 5, 2025, 09:10 AM | 2.88 MB | 3 | |
linux-aarch64/tokenizers-0.22.1-py311hb555b49_0.tar.bz2 | conda | 0.22.1 | linux-aarch64 | main | Nov 5, 2025, 09:10 AM | 2.89 MB | 3 | |
linux-aarch64/tokenizers-0.22.1-py310hb555b49_0.tar.bz2 | conda | 0.22.1 | linux-aarch64 | main | Nov 5, 2025, 09:10 AM | 2.88 MB | 3 | |
linux-64/tokenizers-0.22.1-py313h4b931ed_0.tar.bz2 | conda | 0.22.1 | linux-64 | main | Nov 5, 2025, 09:10 AM | 2.93 MB | 13 | |
linux-64/tokenizers-0.22.1-py311h4b931ed_0.tar.bz2 | conda | 0.22.1 | linux-64 | main | Nov 5, 2025, 09:10 AM | 2.93 MB | 15 | |
linux-64/tokenizers-0.22.1-py310h4b931ed_0.tar.bz2 | conda | 0.22.1 | linux-64 | main | Nov 5, 2025, 09:10 AM | 2.92 MB | 19 | |
linux-64/tokenizers-0.22.1-py312h4b931ed_0.tar.bz2 | conda | 0.22.1 | linux-64 | main | Nov 5, 2025, 09:10 AM | 2.9 MB | 18 | |
win-64/tokenizers-0.21.0-py39h5bc5163_0.tar.bz2 | conda | 0.21.0 | win-64 | main | Mar 7, 2025, 07:52 PM | 2.23 MB | 11 | |
win-64/tokenizers-0.21.0-py313hbd09cc8_0.tar.bz2 | conda | 0.21.0 | win-64 | main | Mar 7, 2025, 07:52 PM | 2.21 MB | 11 | |
win-64/tokenizers-0.21.0-py312h482ea96_0.tar.bz2 | conda | 0.21.0 | win-64 | main | Mar 7, 2025, 07:52 PM | 2.23 MB | 11 | |
win-64/tokenizers-0.21.0-py311hbb1481a_0.tar.bz2 | conda | 0.21.0 | win-64 | main | Mar 7, 2025, 07:52 PM | 2.23 MB | 11 | |
osx-arm64/tokenizers-0.21.0-py39hd743ec3_0.tar.bz2 | conda | 0.21.0 | osx-arm64 | main | Mar 7, 2025, 07:52 PM | 2.35 MB | 4 | |
win-64/tokenizers-0.21.0-py310h5bc5163_0.tar.bz2 | conda | 0.21.0 | win-64 | main | Mar 7, 2025, 07:52 PM | 2.23 MB | 13 | |
osx-arm64/tokenizers-0.21.0-py313h5b23da0_0.tar.bz2 | conda | 0.21.0 | osx-arm64 | main | Mar 7, 2025, 07:52 PM | 2.35 MB | 12 | |
osx-arm64/tokenizers-0.21.0-py312he2d9c3e_0.tar.bz2 | conda | 0.21.0 | osx-arm64 | main | Mar 7, 2025, 07:52 PM | 2.35 MB | 9 |