Files | Anaconda.org

HuggingFace / packages / tokenizers

1

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Filters

Type	Size	Name	Uploaded	Downloads	Labels
conda	3.3 MB	\| osx-64/tokenizers-0.13.0.dev0-py38_0.tar.bz2	2 years and 10 months ago	1401	main
conda	3.3 MB	\| osx-64/tokenizers-0.13.0.dev0-py37_0.tar.bz2	2 years and 10 months ago	392	main
conda	3.3 MB	\| osx-64/tokenizers-0.13.0.dev0-py39_0.tar.bz2	2 years and 10 months ago	3024	main
conda	2.9 MB	\| win-64/tokenizers-0.13.0.dev0-py37_0.tar.bz2	2 years and 10 months ago	1982	main
conda	3.0 MB	\| win-64/tokenizers-0.13.0.dev0-py38_0.tar.bz2	2 years and 10 months ago	6214	main
conda	3.0 MB	\| win-64/tokenizers-0.13.0.dev0-py39_0.tar.bz2	2 years and 10 months ago	10078	main
conda	4.5 MB	\| linux-64/tokenizers-0.13.0.dev0-py39_0.tar.bz2	2 years and 10 months ago	14669	main
conda	4.5 MB	\| linux-64/tokenizers-0.13.0.dev0-py38_0.tar.bz2	2 years and 10 months ago	11578	main
conda	4.5 MB	\| linux-64/tokenizers-0.13.0.dev0-py37_0.tar.bz2	2 years and 10 months ago	2875	main