Badges | Anaconda.org

r / packages / r-tokenizers.bpe

0

Unsupervised text tokenizer focused on computational efficiency. Wraps the 'YouTokenToMe' library <https://github.com/VKCOM/YouTokenToMe> which is an implementation of fast Byte Pair Encoding (BPE) <https://aclanthology.org/P16-1162/>.

Click on a badge to see how to embed it in your web page

https://anaconda.org/r/r-tokenizers.bpe/badges/version.svg

https://anaconda.org/r/r-tokenizers.bpe/badges/latest_release_date.svg

https://anaconda.org/r/r-tokenizers.bpe/badges/latest_release_relative_date.svg

https://anaconda.org/r/r-tokenizers.bpe/badges/platforms.svg

https://anaconda.org/r/r-tokenizers.bpe/badges/license.svg

https://anaconda.org/r/r-tokenizers.bpe/badges/downloads.svg