About Anaconda Help Download Anaconda
Sign In
Anaconda.com

conda-forge / packages / tokenizers

Fast State-of-the-Art Tokenizers optimized for Research and Production

copied from cf-post-staging / tokenizers

Filters
Type Size Name Uploaded Downloads Labels
conda 2.0 MB | osx-arm64/tokenizers-0.10.1-py39hda0fb44_0.tar.bz2  4 years and 9 months ago 609 main
conda 2.0 MB | osx-arm64/tokenizers-0.10.1-py38h8521912_0.tar.bz2  4 years and 9 months ago 725 main
conda 1.9 MB | win-64/tokenizers-0.10.1-py39h8c9392a_0.tar.bz2  4 years and 9 months ago 2537 main
conda 1.9 MB | win-64/tokenizers-0.10.1-py36h01e22a4_0.tar.bz2  4 years and 9 months ago 735 main
conda 1.9 MB | win-64/tokenizers-0.10.1-py37h537c2b9_0.tar.bz2  4 years and 9 months ago 2514 main
conda 1.9 MB | win-64/tokenizers-0.10.1-py38h291c280_0.tar.bz2  4 years and 9 months ago 2240 main
conda 2.0 MB | osx-64/tokenizers-0.10.1-py38heb1ef26_0.tar.bz2  4 years and 9 months ago 1317 main
conda 1.9 MB | osx-64/tokenizers-0.10.1-py39h6561821_0.tar.bz2  4 years and 9 months ago 725 main
conda 2.0 MB | osx-64/tokenizers-0.10.1-py36h502fc6b_0.tar.bz2  4 years and 9 months ago 237 main
conda 2.0 MB | osx-64/tokenizers-0.10.1-py37hc5d47e5_0.tar.bz2  4 years and 9 months ago 1531 main
conda 1.9 MB | osx-64/tokenizers-0.10.1-py36h33fedaa_0.tar.bz2  4 years and 9 months ago 834 main
conda 2.8 MB | linux-64/tokenizers-0.10.1-py36h589faa6_0.tar.bz2  4 years and 9 months ago 5031 main
conda 2.8 MB | linux-64/tokenizers-0.10.1-py36h002baa2_0.tar.bz2  4 years and 9 months ago 2573 main
conda 2.8 MB | linux-64/tokenizers-0.10.1-py38hb63a372_0.tar.bz2  4 years and 9 months ago 13605 main
conda 2.8 MB | linux-64/tokenizers-0.10.1-py39hd6d55de_0.tar.bz2  4 years and 9 months ago 7988 main
conda 2.8 MB | linux-64/tokenizers-0.10.1-py37hcb7a40c_0.tar.bz2  4 years and 9 months ago 14763 main
© 2025 Anaconda, Inc. All Rights Reserved. (v4.2.13) Legal | Privacy Policy