| sebastianbergmann/php-token-stream |
6,457 |
|
104,288 |
188 |
over 4 years ago |
36 |
November 30, 2020 |
0 |
other |
PHP |
| Wrapper around PHP's tokenizer extension. |
| datalust/superpower |
848 |
|
64 |
44 |
almost 4 years ago |
99 |
June 23, 2022 |
10 |
apache-2.0 |
C# |
| A C# parser construction toolkit with high-quality error reporting |
| glayzzle/php-parser |
500 |
|
327 |
100 |
over 2 years ago |
74 |
June 07, 2023 |
34 |
bsd-3-clause |
JavaScript |
| :herb: NodeJS PHP Parser - extract AST or tokens |
| lydell/js-tokens |
410 |
|
586,360 |
1,870 |
over 2 years ago |
21 |
September 09, 2023 |
0 |
mit |
JavaScript |
| Tiny JavaScript tokenizer. |
| timtadh/lexmachine |
370 |
|
1 |
45 |
over 3 years ago |
6 |
July 15, 2022 |
5 |
other |
Go |
| Lex machinary for go. |
| daniel-kukiela/nmt-chatbot |
309 |
|
0 |
0 |
about 6 years ago |
0 |
|
47 |
gpl-3.0 |
Python |
| NMT Chatbot |
| coccoc/coccoc-tokenizer |
295 |
|
0 |
0 |
almost 5 years ago |
0 |
|
3 |
lgpl-3.0 |
C++ |
| high performance tokenizer for Vietnamese language |
| bruce-dunwiddie/tsql-parser |
278 |
|
3 |
15 |
about 3 years ago |
27 |
July 15, 2022 |
6 |
apache-2.0 |
C# |
| Library Written in C# For Parsing SQL Server T-SQL Scripts in .Net |
| linonetwo/segmentit |
208 |
|
1 |
6 |
about 3 years ago |
17 |
December 22, 2019 |
6 |
mit |
JavaScript |
| 任何 JS 环境可用的中文分词包，fork from leizongmin/node-segment |
| fnl/syntok |
158 |
|
5 |
12 |
about 4 years ago |
16 |
March 12, 2022 |
5 |
mit |
Python |
| Text tokenization and sentence segmentation (segtok v2) |
