About Anaconda Help Download Anaconda
Sign In
Anaconda.com

conda-forge / packages / tokenizers

Fast State-of-the-Art Tokenizers optimized for Research and Production

copied from cf-post-staging / tokenizers

Filters
Type Size Name Uploaded Downloads Labels
conda 1.7 MB | win-64/tokenizers-0.9.3-py37h537c2b9_0.tar.bz2  5 years and 1 month ago 391 main
conda 1.7 MB | win-64/tokenizers-0.9.3-py38h291c280_0.tar.bz2  5 years and 1 month ago 395 main
conda 1.7 MB | win-64/tokenizers-0.9.3-py36h01e22a4_0.tar.bz2  5 years and 1 month ago 383 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py36hc3ca637_0.tar.bz2  5 years and 1 month ago 258 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py37h7bba54e_0.tar.bz2  5 years and 1 month ago 262 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py37h17e0dd7_0.tar.bz2  5 years and 1 month ago 2744 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py38h0582b0e_0.tar.bz2  5 years and 1 month ago 257 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py36h2bc52f9_0.tar.bz2  5 years and 1 month ago 2704 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py38h781e378_0.tar.bz2  5 years and 1 month ago 2720 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py36hc84fb85_0.tar.bz2  5 years and 1 month ago 2660 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py36h999cc91_0.tar.bz2  5 years and 1 month ago 265 main
© 2026 Anaconda, Inc. All Rights Reserved. (v4.2.13) Legal | Privacy Policy