tokenizers

Community

Fast State-of-the-Art Tokenizers optimized for Research and Production

Versions

To install this package, run one of the following:

$conda install rocketce::tokenizers

Version

2 / 8 versions selected

Downloads (Last 6 months): 0

Summary

Fast State-of-the-Art Tokenizers optimized for Research and Production

Last Updated

Jul 12, 2024 at 11:24

License

Apache-2.0

Total Downloads

1.5K

Supported Platforms

linux-ppc64le

Home

GitHub Repository

Documentation