tokenizers
Fast State-of-the-Art Tokenizers optimized for Research and Production
| Name | Type | Version | Platform | Labels | Updated | Size | Downloads | Actions |
|---|
linux-64/tokenizers-0.21.0-py311_1.conda | conda | 0.21.0 | linux-64 | main | Apr 3, 2025, 01:46 PM | 2.13 MB | 10 | |
linux-64/tokenizers-0.15.0-py39_0.conda | conda | 0.15.0 | linux-64 | main | Feb 11, 2024, 02:09 AM | 2.75 MB | 10 | |
linux-64/tokenizers-0.15.0-py311_0.conda | conda | 0.15.0 | linux-64 | main | Feb 11, 2024, 02:09 AM | 2.76 MB | 23 | |
linux-64/tokenizers-0.15.0-py310_0.conda | conda | 0.15.0 | linux-64 | main | Feb 11, 2024, 02:09 AM | 2.75 MB | 12 | |
win-64/tokenizers-0.15.0-py311_0.conda | conda | 0.15.0 | win-64 | main | Jan 22, 2024, 06:13 PM | 1.63 MB | 51 |