About Anaconda Help Download Anaconda
Sign In
Anaconda.com

conda-forge / packages / tokenizers

Fast State-of-the-Art Tokenizers optimized for Research and Production

copied from cf-post-staging / tokenizers

Filters
Type Size Name Uploaded Downloads Labels
conda 1.7 MB | osx-64/tokenizers-0.8.1-py38h2cef62f_0.tar.bz2  5 years and 5 months ago 378 main
conda 1.7 MB | osx-64/tokenizers-0.8.1-py37h4c3b196_0.tar.bz2  5 years and 5 months ago 379 main
conda 1.7 MB | osx-64/tokenizers-0.8.1-py36h327c840_0.tar.bz2  5 years and 5 months ago 327 main
conda 1.7 MB | osx-64/tokenizers-0.8.1-py36hf1545a4_0.tar.bz2  5 years and 5 months ago 330 main
conda 2.5 MB | linux-64/tokenizers-0.8.1-py36h327c840_0.tar.bz2  5 years and 5 months ago 3083 main
conda 2.5 MB | linux-64/tokenizers-0.8.1-py37h4c3b196_0.tar.bz2  5 years and 5 months ago 5771 main
conda 2.5 MB | linux-64/tokenizers-0.8.1-py38h2cef62f_0.tar.bz2  5 years and 5 months ago 4870 main
conda 2.5 MB | linux-64/tokenizers-0.8.1-py36hf1545a4_0.tar.bz2  5 years and 5 months ago 4671 main
© 2026 Anaconda, Inc. All Rights Reserved. (v4.2.13) Legal | Privacy Policy