Awesome Open Source

Search

Programming Languages

About

The Top 10 Word Segmentation Open Source Projects

Open source projects categorized as Word Segmentation

Categories > Machine Learning > Word Segmentation

Edit Category

google/sentencepiece ⭐ 8,851

Unsupervised text tokenizer for Neural Network-based text generation.

dependent packages 0 total releases 0 most recent commit about 2 years ago

lancopku/pkuseg-python ⭐ 6,001

pkuseg多领域中文分词工具; The pkuseg toolkit for multi-domain Chinese word segmentation

dependent packages 0 total releases 0 most recent commit over 3 years ago

baidu/lac ⭐ 3,644

百度NLP：分词，词性标注，命名实体识别，词重要性

dependent packages 0 total releases 0 most recent commit almost 5 years ago

wolfgarbe/SymSpell ⭐ 2,970

SymSpell: 1 million times faster spelling correction & fuzzy search through Symmetric Delete spelling correction algorithm

dependent packages 0 total releases 0 most recent commit over 2 years ago

rsennrich/subword-nmt ⭐ 1,937

Unsupervised Word Segmentation for Neural Machine Translation and Text Generation

dependent packages 0 total releases 0 most recent commit over 3 years ago

VKCOM/YouTokenToMe ⭐ 926

Unsupervised text tokenizer focused on computational efficiency

dependent packages 0 total releases 0 most recent commit almost 3 years ago

PyThaiNLP/pythainlp ⭐ 902

Thai Natural Language Processing in Python.

dependent packages 0 total releases 0 most recent commit about 2 years ago

fastnlp/fastHan ⭐ 730

fastHan是基于fastNLP与pytorch实现的中文自然语言处理工具，像spacy一样调用方便。

dependent packages 0 total releases 0 most recent commit over 2 years ago

mammothb/symspellpy ⭐ 693

Python port of SymSpell: 1 million times faster spelling correction & fuzzy search through Symmetric Delete spelling correction algorithm

dependent packages 0 total releases 0 most recent commit almost 3 years ago

messense/jieba-rs ⭐ 585

The Jieba Chinese Word Segmentation Implemented in Rust

dependent packages 0 total releases 0 most recent commit over 2 years ago

Get A Weekly Email With Trending Word Segmentation Projects

No Spam. Unsubscribe easily at any time.

Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2026 Awesome Open Source. All rights reserved.