pytokens
A fast, spec-compliant Python tokenizer
Installation
To install this package, run one of the following:
Usage Tracking
0.4.1
0.3.0
0.2.0
3 / 8 versions selected
Downloads (Last 6 months): 0
Description
A fast, spec-compliant Python tokenizer that runs on older Pythons.
About
Summary
A fast, spec-compliant Python tokenizer
Last Updated
Apr 2, 2026 at 15:08
License
MIT
Supported Platforms
linux-64
linux-aarch64
macOS-arm64
win-64
GitHub Repository
https://github.com/tusharsadhwani/pytokens