Unsupervised text tokenizer focused on computational efficiency. Wraps the 'YouTokenToMe' library <https://github.com/VKCOM/YouTokenToMe> which is an implementation of fast Byte Pair Encoding (BPE) <https://www.aclweb.org/anthology/P16-1162>.
| Label | Latest Version |
|---|---|
| main | 0.1.0 |
