Files | Anaconda.org

HuggingFace / packages / tokenizers

1

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Filters

Type	Size	Name	Uploaded	Downloads	Labels
conda	3.1 MB	\| osx-64/tokenizers-0.11.6-py39_0.tar.bz2	2 years and 8 months ago	36	main
conda	3.1 MB	\| osx-64/tokenizers-0.11.6-py38_0.tar.bz2	2 years and 8 months ago	11	main
conda	3.1 MB	\| osx-64/tokenizers-0.11.6-py37_0.tar.bz2	2 years and 8 months ago	13	main
conda	2.9 MB	\| win-64/tokenizers-0.11.6-py38_0.tar.bz2	2 years and 8 months ago	26	main
conda	2.9 MB	\| win-64/tokenizers-0.11.6-py37_0.tar.bz2	2 years and 8 months ago	18	main
conda	2.9 MB	\| win-64/tokenizers-0.11.6-py39_0.tar.bz2	2 years and 8 months ago	152	main
conda	4.2 MB	\| linux-64/tokenizers-0.11.6-py37_0.tar.bz2	2 years and 8 months ago	28	main
conda	4.2 MB	\| linux-64/tokenizers-0.11.6-py39_0.tar.bz2	2 years and 8 months ago	81	main
conda	4.2 MB	\| linux-64/tokenizers-0.11.6-py38_0.tar.bz2	2 years and 8 months ago	33	main