Files | Anaconda.org

HuggingFace / packages / tokenizers

1

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Filters

Type	Size	Name	Uploaded	Downloads	Labels
conda	3.2 MB	\| osx-64/tokenizers-0.12.1-py38_0.tar.bz2	2 years and 11 months ago	178	main
conda	3.2 MB	\| osx-64/tokenizers-0.12.1-py39_0.tar.bz2	2 years and 11 months ago	637	main
conda	3.0 MB	\| win-64/tokenizers-0.12.1-py37_0.tar.bz2	2 years and 11 months ago	257	main
conda	3.0 MB	\| win-64/tokenizers-0.12.1-py39_0.tar.bz2	2 years and 11 months ago	1510	main
conda	3.0 MB	\| win-64/tokenizers-0.12.1-py38_0.tar.bz2	2 years and 11 months ago	479	main
conda	3.2 MB	\| osx-64/tokenizers-0.12.1-py37_0.tar.bz2	2 years and 11 months ago	112	main
conda	4.3 MB	\| linux-64/tokenizers-0.12.1-py39_0.tar.bz2	2 years and 11 months ago	2206	main
conda	4.3 MB	\| linux-64/tokenizers-0.12.1-py38_0.tar.bz2	2 years and 11 months ago	1693	main
conda	4.3 MB	\| linux-64/tokenizers-0.12.1-py37_0.tar.bz2	2 years and 11 months ago	659	main