CMD + K

pytokens

Community

A Fast, spec compliant Python 3.14+ tokenizer that runs on older Pythons.

Installation

To install this package, run one of the following:

Conda
$conda install conda-forge::pytokens

Usage Tracking

0.4.1
0.4.0
0.3.0
3 / 8 versions selected
Downloads (Last 6 months): 0

About

Summary

A Fast, spec compliant Python 3.14+ tokenizer that runs on older Pythons.

Last Updated

Jan 30, 2026 at 08:53

License

MIT

Supported Platforms

macOS-arm64
linux-64
macOS-64
linux-aarch64
linux-ppc64le
win-64

Unsupported Platforms

noarch Last supported version: 0.3.0