About Anaconda Help Download Anaconda
Sign In
Anaconda.com

anaconda / packages / tokenizers

Fast State-of-the-Art Tokenizers optimized for Research and Production

Filters
Type Size Name Uploaded Downloads Labels
conda 2.4 MB | osx-arm64/tokenizers-0.20.1-py39hd743ec3_1.tar.bz2  1 year and 1 month ago 39 main
conda 2.4 MB | osx-arm64/tokenizers-0.20.1-py312he2d9c3e_1.tar.bz2  1 year and 1 month ago 59 main
conda 2.4 MB | osx-arm64/tokenizers-0.20.1-py311hac6c8b7_1.tar.bz2  1 year and 1 month ago 46 main
conda 2.4 MB | osx-arm64/tokenizers-0.20.1-py310hd743ec3_1.tar.bz2  1 year and 1 month ago 31 main
conda 2.2 MB | osx-64/tokenizers-0.20.1-py39h753fa65_1.tar.bz2  1 year and 1 month ago 16 main
conda 2.2 MB | osx-64/tokenizers-0.20.1-py312h74fea66_1.tar.bz2  1 year and 1 month ago 22 main
conda 2.2 MB | osx-64/tokenizers-0.20.1-py311h497a92b_1.tar.bz2  1 year and 1 month ago 26 main
conda 2.2 MB | osx-64/tokenizers-0.20.1-py310h753fa65_1.tar.bz2  1 year and 1 month ago 20 main
conda 2.5 MB | linux-s390x/tokenizers-0.20.1-py39ha3609b1_1.tar.bz2  1 year and 1 month ago 14 main
conda 2.5 MB | linux-s390x/tokenizers-0.20.1-py312hc707890_1.tar.bz2  1 year and 1 month ago 14 main
conda 2.5 MB | linux-s390x/tokenizers-0.20.1-py311he4aaa16_1.tar.bz2  1 year and 1 month ago 14 main
conda 2.5 MB | linux-s390x/tokenizers-0.20.1-py310ha3609b1_1.tar.bz2  1 year and 1 month ago 14 main
conda 2.4 MB | linux-aarch64/tokenizers-0.20.1-py39hdc1999c_1.tar.bz2  1 year and 1 month ago 55 main
conda 2.4 MB | linux-aarch64/tokenizers-0.20.1-py310hdc1999c_1.tar.bz2  1 year and 1 month ago 21 main
conda 2.4 MB | linux-aarch64/tokenizers-0.20.1-py311hf2d19f8_1.tar.bz2  1 year and 1 month ago 16 main
conda 2.4 MB | linux-aarch64/tokenizers-0.20.1-py312h377adab_1.tar.bz2  1 year and 1 month ago 133 main
conda 2.2 MB | win-64/tokenizers-0.20.1-py39h5bc1c3f_0.tar.bz2  1 year and 2 months ago 43 main
conda 2.2 MB | win-64/tokenizers-0.20.1-py38h5bc1c3f_0.tar.bz2  1 year and 2 months ago 85 main
conda 2.2 MB | win-64/tokenizers-0.20.1-py312h4b3af92_0.tar.bz2  1 year and 2 months ago 47 main
conda 2.2 MB | win-64/tokenizers-0.20.1-py311h3eac673_0.tar.bz2  1 year and 2 months ago 67 main
conda 2.2 MB | win-64/tokenizers-0.20.1-py310h5bc1c3f_0.tar.bz2  1 year and 2 months ago 38 main
conda 2.4 MB | osx-arm64/tokenizers-0.20.1-py39he44fafa_0.tar.bz2  1 year and 2 months ago 27 main
conda 2.4 MB | osx-arm64/tokenizers-0.20.1-py38he44fafa_0.tar.bz2  1 year and 2 months ago 32 main
conda 2.4 MB | osx-arm64/tokenizers-0.20.1-py312h654b0c7_0.tar.bz2  1 year and 2 months ago 39 main
conda 2.4 MB | osx-arm64/tokenizers-0.20.1-py311h120959f_0.tar.bz2  1 year and 2 months ago 39 main
© 2025 Anaconda, Inc. All Rights Reserved. (v4.2.13) Legal | Privacy Policy