pytokens
A fast, spec-compliant Python tokenizer
Installation
To install this package, run one of the following:
Usage Tracking
0.3.0
0.2.0
2 / 8 versions selected
Downloads (Last 6 months): 0
Description
A fast, spec-compliant Python tokenizer that runs on older Pythons.
About
Summary
A fast, spec-compliant Python tokenizer
Last Updated
Nov 26, 2025 at 10:26
License
MIT
Total Downloads
3.3K
Supported Platforms
macOS-arm64
win-64
linux-64
linux-aarch64
GitHub Repository
https://github.com/tusharsadhwani/pytokens