Files | Anaconda.org

HuggingFace / packages / tokenizers

1

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Filters

Type	Size	Name	Uploaded	Downloads	Labels
conda	1.8 MB	\| win-64/tokenizers-0.10.2-py37_0.tar.bz2	4 years and 5 months ago	370	main
conda	1.9 MB	\| win-64/tokenizers-0.10.2-py36_0.tar.bz2	4 years and 5 months ago	200	main
conda	1.8 MB	\| win-64/tokenizers-0.10.2-py38_0.tar.bz2	4 years and 5 months ago	778	main
conda	2.0 MB	\| osx-64/tokenizers-0.10.2-py35_0.tar.bz2	4 years and 5 months ago	10	main
conda	1.8 MB	\| win-64/tokenizers-0.10.2-py35_0.tar.bz2	4 years and 5 months ago	23	main
conda	2.0 MB	\| osx-64/tokenizers-0.10.2-py38_0.tar.bz2	4 years and 5 months ago	749	main
conda	2.8 MB	\| linux-64/tokenizers-0.10.2-py35_0.tar.bz2	4 years and 5 months ago	22	main
conda	2.0 MB	\| osx-64/tokenizers-0.10.2-py37_0.tar.bz2	4 years and 5 months ago	459	main
conda	2.0 MB	\| osx-64/tokenizers-0.10.2-py36_0.tar.bz2	4 years and 5 months ago	343	main
conda	2.8 MB	\| linux-64/tokenizers-0.10.2-py38_0.tar.bz2	4 years and 5 months ago	2238	main
conda	2.8 MB	\| linux-64/tokenizers-0.10.2-py36_0.tar.bz2	4 years and 5 months ago	745	main
conda	2.8 MB	\| linux-64/tokenizers-0.10.2-py37_0.tar.bz2	4 years and 5 months ago	1422	main