About Anaconda Help Download Anaconda
Sign In
Anaconda.com

conda-forge / packages / tokenizers

Fast State-of-the-Art Tokenizers optimized for Research and Production

copied from cf-post-staging / tokenizers

Filters
Type Size Name Uploaded Downloads Labels
conda 1.7 MB | win-64/tokenizers-0.9.3-py37h537c2b9_0.tar.bz2  5 years and 4 months ago 402 main
conda 1.7 MB | win-64/tokenizers-0.9.3-py38h291c280_0.tar.bz2  5 years and 4 months ago 406 main
conda 1.7 MB | win-64/tokenizers-0.9.3-py36h01e22a4_0.tar.bz2  5 years and 4 months ago 392 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py36hc3ca637_0.tar.bz2  5 years and 4 months ago 267 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py37h7bba54e_0.tar.bz2  5 years and 4 months ago 271 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py37h17e0dd7_0.tar.bz2  5 years and 4 months ago 2924 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py38h0582b0e_0.tar.bz2  5 years and 4 months ago 267 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py36h2bc52f9_0.tar.bz2  5 years and 4 months ago 2884 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py38h781e378_0.tar.bz2  5 years and 4 months ago 2880 main
conda 2.6 MB | linux-64/tokenizers-0.9.3-py36hc84fb85_0.tar.bz2  5 years and 4 months ago 2837 main
conda 1.8 MB | osx-64/tokenizers-0.9.3-py36h999cc91_0.tar.bz2  5 years and 4 months ago 275 main
© 2026 Anaconda, Inc. All Rights Reserved. (v4.2.15) Legal | Privacy Policy