About Anaconda Help Download Anaconda
Sign In
Anaconda.com

conda-forge / packages / tokenizers

Fast State-of-the-Art Tokenizers optimized for Research and Production

copied from cf-post-staging / tokenizers

Filters
Type Size Name Uploaded Downloads Labels
conda 1.7 MB | win-64/tokenizers-0.9.3-py37h537c2b9_0.tar.bz2  5 years and 1 month ago 389 main
conda 1.7 MB | win-64/tokenizers-0.9.3-py38h291c280_0.tar.bz2  5 years and 1 month ago 393 main
conda 1.7 MB | win-64/tokenizers-0.9.3-py36h01e22a4_0.tar.bz2  5 years and 1 month ago 382 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py36hc3ca637_0.tar.bz2  5 years and 1 month ago 257 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py37h7bba54e_0.tar.bz2  5 years and 1 month ago 261 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py37h17e0dd7_0.tar.bz2  5 years and 1 month ago 2729 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py38h0582b0e_0.tar.bz2  5 years and 1 month ago 256 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py36h2bc52f9_0.tar.bz2  5 years and 1 month ago 2690 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py38h781e378_0.tar.bz2  5 years and 1 month ago 2707 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py36hc84fb85_0.tar.bz2  5 years and 1 month ago 2647 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py36h999cc91_0.tar.bz2  5 years and 1 month ago 264 main
© 2025 Anaconda, Inc. All Rights Reserved. (v4.2.13) Legal | Privacy Policy