A tiny sentence/word tokenizer for Japanese text written in Python
copied from cf-post-staging / konohaKonoha is a Python library for providing easy-to-use integrated interface of various Japanese tokenziers, which enables you to switch a tokenizer and boost your pre-processing.
