Convert natural language text into tokens. The tokenizers have a consistent interface and are compatible with Unicode, thanks to being built on the 'stringi' package. Includes tokenizers for shingled n-grams, skip n-grams, words, word stems, sentences, paragraphs, characters, lines, and regular expressions.
| Name | Type | Version | Platform | Labels | Updated | Size | Downloads | Actions |
|---|
linux-64/r-tokenizers-0.1.4-r343h599a50d_0.tar.bz2 | conda | 0.1.4 | linux-64 | maintest-label | Feb 2, 2018, 11:17 PM | 91.3 KB | 7 | |
linux-64/r-tokenizers-0.1.4-r342h0fb3b52_0.tar.bz2 | conda | 0.1.4 | linux-64 | maintest-label | Feb 2, 2018, 11:17 PM | 91.34 KB | 7 | |
linux-64/r-tokenizers-0.1.4-r3.4.1_0.tar.bz2 | conda | 0.1.4 | linux-64 | maintest-label | Feb 2, 2018, 11:17 PM | 117.87 KB | 6 | |
linux-64/r-tokenizers-0.1.4-r3.3.2_0.tar.bz2 | conda | 0.1.4 | linux-64 | maintest-label | Feb 2, 2018, 11:17 PM | 107.46 KB | 6 | |
linux-64/r-tokenizers-0.1.4-mro343hf484d3e_0.tar.bz2 | conda | 0.1.4 | linux-64 | maintest-label | Feb 2, 2018, 11:17 PM | 91.26 KB | 6 |