Rust Tokenizers Alternatives

Rust-tokenizer offers high-performance tokenizers for modern language models, including WordPiece, Byte-Pair Encoding (BPE) and Unigram (SentencePiece) models
Suggest Alternative
Alternatives To guillaume-be/rust-tokenizers
Project Name Stars Downloads Repos Using This Packages Using This Most Recent Commit Total Releases Latest Release Open Issues License Language
zhanghang1989/PyTorch-Encoding 1,974 1 0 about 3 years ago 130 August 08, 2020 143 mit Python
A CV toolkit for my papers.
AxeldeRomblay/MLBox 1,403 0 0 about 3 years ago 21 August 25, 2020 26 other Python
MLBox is a powerful Automated Machine Learning python library.
kexinhuang12345/DeepPurpose 826 0 0 over 2 years ago 13 October 26, 2021 12 bsd-3-clause Jupyter Notebook
A Deep Learning Toolkit for DTI, Drug Property, PPI, DDI, Protein Function Prediction (Bioinformatics)
RelevanceAI/vectorhub 525 0 1 over 3 years ago 45 September 28, 2021 8 apache-2.0 Python
Vector Hub - Library for easy discovery, and consumption of State-of-the-art models to turn data into vectors. (text2vec, image2vec, video2vec, graph2vec, bert, inception, etc)
ashishpatel26/Amazing-Feature-Engineering 485 0 0 over 2 years ago 0 0 Jupyter Notebook
Feature engineering is the process of using domain knowledge to extract features from raw data via data mining techniques. These features can be used to improve the performance of machine learning algorithms. Feature engineering can be considered as applied machine learning itself.
tatp22/multidim-positional-encoding 429 0 0 over 2 years ago 14 June 17, 2022 1 mit Python
An implementation of 1D, 2D, and 3D positional encoding in Pytorch and TensorFlow
vector-ai/vectorai 299 0 4 over 2 years ago 9 June 04, 2021 12 apache-2.0 Python
Vector AI — A platform for building vector based applications. Encode, query and analyse data using vectors.
guillaume-be/rust-tokenizers 232 0 7 over 2 years ago 34 October 01, 2023 4 apache-2.0 Rust
Rust-tokenizer offers high-performance tokenizers for modern language models, including WordPiece, Byte-Pair Encoding (BPE) and Unigram (SentencePiece) models
Vadikus/practicalDL 176 0 0 about 4 years ago 0 0 apache-2.0 Jupyter Notebook
A Practical Guide to Deep Learning with TensorFlow 2.0 and Keras materials for Frontend Masters course
songtaohe/Sat2Graph 112 0 0 about 3 years ago 0 11 mit OpenEdge ABL
Sat2Graph: Road Graph Extraction through Graph-Tensor Encoding
Alternatives To guillaume-be/rust-tokenizers
Select To Compare


Alternative Project Comparisons
Popular Encoding Projects
Popular Deep Learning Projects
Popular Data Formats Categories
Related Searches
Get A Weekly Email With Trending Projects
No Spam. Unsubscribe easily at any time.
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2026 Awesome Open Source.  All rights reserved.