tokenizers

Community

Fast State-of-the-Art Tokenizers optimized for Research and Production

Versions

To install this package, run one of the following:

$conda install services::tokenizers

Version

1 / 8 versions selected

Downloads (Last 6 months): 0

Summary

Fast State-of-the-Art Tokenizers optimized for Research and Production

Last Updated

Nov 18, 2021 at 16:05

License

Apache-2.0

Total Downloads

102

Supported Platforms

macOS-arm64

linux-64

linux-ppc64le

linux-aarch64

macOS-64

Home

GitHub Repository