Fast State-of-the-Art Tokenizers optimized for Research and Production
conda install rocketce::tokenizers
conda install rocketce/label/rocketce-1.10.0::tokenizers
conda install rocketce/label/rocketce-1.11.2::tokenizers
conda install rocketce/label/rocketce-1.11.5::tokenizers
conda install rocketce/label/rocketce-1.11.6::tokenizers