tokenizers - HuggingFace | Anaconda.org

tokenizers

Community

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

win-64/tokenizers-0.13.2-py38_0.tar.bz2

conda

0.13.2

win-64

main

Dec 19, 2022, 11:44 AM

2.98 MB

229

win-64/tokenizers-0.13.2-py310_0.tar.bz2

conda

0.13.2

win-64

main

Dec 19, 2022, 11:43 AM

2.96 MB

963

win-64/tokenizers-0.13.2-py37_0.tar.bz2

conda

0.13.2

win-64

main

Dec 19, 2022, 11:41 AM

2.96 MB

116

osx-64/tokenizers-0.13.2-py310_0.tar.bz2

conda

0.13.2

osx-64

main

Dec 19, 2022, 11:40 AM

3.26 MB

201

osx-64/tokenizers-0.13.2-py37_0.tar.bz2

conda

0.13.2

osx-64

main

Dec 19, 2022, 11:39 AM

3.28 MB

20

win-64/tokenizers-0.13.2-py39_0.tar.bz2

conda

0.13.2

win-64

main

Dec 19, 2022, 11:37 AM

2.96 MB

420

osx-64/tokenizers-0.13.2-py39_0.tar.bz2

conda

0.13.2

osx-64

main

Dec 19, 2022, 11:36 AM

3.3 MB

144

osx-64/tokenizers-0.13.2-py38_0.tar.bz2

conda

0.13.2

osx-64

main

Dec 19, 2022, 11:36 AM

3.27 MB

64

linux-64/tokenizers-0.13.2-py38_0.tar.bz2

conda

0.13.2

linux-64

main

Dec 19, 2022, 11:32 AM

4.47 MB

523

linux-64/tokenizers-0.13.2-py37_0.tar.bz2

conda

0.13.2

linux-64

main

Dec 19, 2022, 11:30 AM

4.47 MB

185

linux-64/tokenizers-0.13.2-py39_0.tar.bz2

conda

0.13.2

linux-64

main

Dec 19, 2022, 11:30 AM

4.46 MB

804

osx-64/tokenizers-0.13.1-py38_0.tar.bz2

conda

0.13.1

osx-64

main

Oct 6, 2022, 12:50 PM

3.3 MB

82

osx-64/tokenizers-0.13.1-py37_0.tar.bz2

conda

0.13.1

osx-64

main

Oct 6, 2022, 12:50 PM

3.3 MB

19

win-64/tokenizers-0.13.1-py38_0.tar.bz2

conda

0.13.1

win-64

main

Oct 6, 2022, 12:48 PM

2.97 MB

175

win-64/tokenizers-0.13.1-py39_0.tar.bz2

conda

0.13.1

win-64

main

Oct 6, 2022, 12:48 PM

2.97 MB

921

win-64/tokenizers-0.13.1-py37_0.tar.bz2

conda

0.13.1

win-64

main

Oct 6, 2022, 12:46 PM

2.96 MB

51

osx-64/tokenizers-0.13.1-py39_0.tar.bz2

conda

0.13.1

osx-64

main

Oct 6, 2022, 12:45 PM

3.3 MB

367

linux-64/tokenizers-0.13.1-py37_0.tar.bz2

conda

0.13.1

linux-64

main

Oct 6, 2022, 12:43 PM

4.49 MB

88

linux-64/tokenizers-0.13.1-py38_0.tar.bz2

conda

0.13.1

linux-64

main

Oct 6, 2022, 12:41 PM

4.51 MB

214

linux-64/tokenizers-0.13.1-py39_0.tar.bz2

conda

0.13.1

linux-64

main

Oct 6, 2022, 12:41 PM

4.49 MB

661

win-64/tokenizers-0.13.0-py37_0.tar.bz2

conda

0.13.0

win-64

main

Sep 21, 2022, 10:38 AM

2.98 MB

23

osx-64/tokenizers-0.13.0-py39_0.tar.bz2

conda

0.13.0

osx-64

main

Sep 21, 2022, 10:37 AM

3.3 MB

16

osx-64/tokenizers-0.13.0-py38_0.tar.bz2

conda

0.13.0

osx-64

main

Sep 21, 2022, 10:37 AM

3.3 MB

16

win-64/tokenizers-0.13.0-py39_0.tar.bz2

conda

0.13.0

win-64

main

Sep 21, 2022, 10:36 AM

2.96 MB

34

win-64/tokenizers-0.13.0-py38_0.tar.bz2

conda

0.13.0

win-64

main

Sep 21, 2022, 10:34 AM

2.97 MB

32