Tokenizers | Anaconda.org

conda-forge / packages / tokenizers 0.21.0

1

Fast State-of-the-Art Tokenizers optimized for Research and Production

copied from cf-staging / tokenizers

License: Apache-2.0
Home: https://pypi.org/project/tokenizers/
Development: https://github.com/huggingface/tokenizers
2366619 total downloads
Last upload: 24 days and 13 hours ago

Installers

linux-64 v0.21.0
linux-ppc64le v0.21.0
linux-aarch64 v0.21.0
osx-arm64 v0.21.0
osx-64 v0.21.0
win-64 v0.21.0

conda install

To install this package run one of the following:
conda install conda-forge::tokenizers

Description

© 2024 Anaconda, Inc. All Rights Reserved. (v4.0.6) Legal | Privacy Policy