tokenizers
Fast State-of-the-Art Tokenizers optimized for Research and Production
- Overview
- Files 35
- Labels 1
- Badges
| Name | Type | Version | Platform | Labels | Updated | Size | Downloads | Actions |
|---|
win-64/tokenizers-0.21.0-py313_0.conda | conda | 0.21.0 | win-64 | main | Mar 10, 2025, 04:46 PM | 1.76 MB | 1.5K | |
win-64/tokenizers-0.21.0-py312_0.conda | conda | 0.21.0 | win-64 | main | Mar 10, 2025, 04:46 PM | 1.77 MB | 23 | |
win-64/tokenizers-0.21.0-py311_0.conda | conda | 0.21.0 | win-64 | main | Mar 10, 2025, 04:46 PM | 1.78 MB | 4.8K | |
linux-64/tokenizers-0.21.0-py313_0.conda | conda | 0.21.0 | linux-64 | main | Mar 10, 2025, 04:05 PM | 2.09 MB | 60 | |
linux-64/tokenizers-0.21.0-py312_0.conda | conda | 0.21.0 | linux-64 | main | Mar 10, 2025, 04:04 PM | 2.1 MB | 13 | |
linux-64/tokenizers-0.21.0-py311_0.conda | conda | 0.21.0 | linux-64 | main | Mar 10, 2025, 04:04 PM | 2.1 MB | 48 | |
linux-64/tokenizers-0.19.1-py39_0.conda | conda | 0.19.1 | linux-64 | main | Sep 12, 2024, 02:35 PM | 2.6 MB | 32 | |
linux-64/tokenizers-0.19.1-py312_0.conda | conda | 0.19.1 | linux-64 | main | Sep 11, 2024, 05:02 PM | 2.62 MB | 20 | |
linux-64/tokenizers-0.19.1-py311_0.conda | conda | 0.19.1 | linux-64 | main | Sep 11, 2024, 05:02 PM | 2.61 MB | 378 | |
linux-64/tokenizers-0.19.1-py310_0.conda | conda | 0.19.1 | linux-64 | main | Sep 11, 2024, 05:02 PM | 2.6 MB | 40 | |
win-64/tokenizers-0.19.1-py39_0.conda | conda | 0.19.1 | win-64 | main | Sep 11, 2024, 05:00 PM | 1.62 MB | 85 | |
win-64/tokenizers-0.19.1-py312_0.conda | conda | 0.19.1 | win-64 | main | Sep 11, 2024, 05:00 PM | 1.62 MB | 23 | |
win-64/tokenizers-0.19.1-py311_0.conda | conda | 0.19.1 | win-64 | main | Sep 11, 2024, 05:00 PM | 1.63 MB | 5.0K | |
win-64/tokenizers-0.19.1-py310_0.conda | conda | 0.19.1 | win-64 | main | Sep 11, 2024, 05:00 PM | 1.62 MB | 33 | |
win-64/tokenizers-0.15.0-py39_0.conda | conda | 0.15.0 | win-64 | main | Mar 14, 2024, 06:44 AM | 1.62 MB | 70 | |
win-64/tokenizers-0.15.0-py311_0.conda | conda | 0.15.0 | win-64 | main | Mar 14, 2024, 06:44 AM | 1.63 MB | 5.4K | |
win-64/tokenizers-0.15.0-py310_0.conda | conda | 0.15.0 | win-64 | main | Mar 14, 2024, 06:44 AM | 1.63 MB | 36 | |
linux-64/tokenizers-0.15.0-py39_0.conda | conda | 0.15.0 | linux-64 | main | Mar 11, 2024, 01:49 AM | 2.75 MB | 34 | |
linux-64/tokenizers-0.15.0-py311_0.conda | conda | 0.15.0 | linux-64 | main | Mar 11, 2024, 01:49 AM | 2.76 MB | 394 | |
linux-64/tokenizers-0.15.0-py310_0.conda | conda | 0.15.0 | linux-64 | main | Mar 11, 2024, 01:49 AM | 2.77 MB | 32 | |
win-64/tokenizers-0.10.1-py39_0.conda | conda | 0.10.1 | win-64 | main | Sep 11, 2023, 05:52 PM | 1.49 MB | 6.4K | |
linux-64/tokenizers-0.10.1-py39_0.tar.bz2 | conda | 0.10.1 | linux-64 | main | Jul 16, 2021, 02:44 AM | 2.8 MB | 1.2K | |
linux-64/tokenizers-0.10.1-py38_0.tar.bz2 | conda | 0.10.1 | linux-64 | main | Jul 16, 2021, 02:43 AM | 2.8 MB | 359 | |
linux-64/tokenizers-0.10.1-py37_0.tar.bz2 | conda | 0.10.1 | linux-64 | main | Jul 16, 2021, 02:43 AM | 2.8 MB | 364 | |
linux-64/tokenizers-0.10.1-py36_0.tar.bz2 | conda | 0.10.1 | linux-64 | main | Jul 16, 2021, 02:43 AM | 2.8 MB | 71 |