CMD + K

libsentencepiece

Community

Unsupervised text tokenizer for Neural Network-based text generation.

Installation

To install this package, run one of the following:

Conda
$conda install esri::libsentencepiece

Usage Tracking

0.2.0
1 / 8 versions selected
Downloads (Last 6 months): 0

Description

SentencePiece is an unsupervised text tokenizer and detokenizer mainly for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training.

About

Summary

Unsupervised text tokenizer for Neural Network-based text generation.

Last Updated

Sep 19, 2025 at 01:53

License

Apache-2.0

Total Downloads

1.5K

Supported Platforms

win-64