Multilingual UnigramLM

company

https://cimeister.github.io/blog/unigramlm/

AI & ML interests

Multilingual Tokenization

Recent Activity

suchirsalhan updated a model about 20 hours ago

MultilingualUnigramLM/las-tokenizers-granite-3.0-8b-base-khm

suchirsalhan published a model about 20 hours ago

MultilingualUnigramLM/las-tokenizers-granite-3.0-8b-base-khm

suchirsalhan updated a model about 20 hours ago

MultilingualUnigramLM/las-tokenizers-granite-3.0-8b-base-tam

View all activity

MultilingualUnigramLM 's datasets 32

MultilingualUnigramLM/FineWeb2-khm_Khmr-100M

Viewer • Updated about 23 hours ago • 1.01M • 5

MultilingualUnigramLM/FineWeb2-tha_Thai-100M

Viewer • Updated about 23 hours ago • 710k • 5

MultilingualUnigramLM/FineWeb2-tam_Taml-100M

Viewer • Updated about 23 hours ago • 264k • 4

MultilingualUnigramLM/FineWeb2-arb_Arab-100M

Viewer • Updated about 23 hours ago • 188k • 4

MultilingualUnigramLM/FineWeb2-ben_Beng-100M

Viewer • Updated about 23 hours ago • 254k • 4

MultilingualUnigramLM/FineWeb2-amh_Ethi-100M

Viewer • Updated about 23 hours ago • 197k • 4

MultilingualUnigramLM/FineWeb2-yor_Latn-100M

Viewer • Updated about 23 hours ago • 80k • 4

MultilingualUnigramLM/FineWeb2-wol_Latn-100M

Viewer • Updated about 23 hours ago • 6.62k • 4

MultilingualUnigramLM/FineWeb2-hin_Deva-100M

Viewer • Updated 1 day ago • 193k • 8

MultilingualUnigramLM/FineWeb2-swh_Latn-100M

Viewer • Updated 1 day ago • 240k • 6

MultilingualUnigramLM/FineWeb2-deu_Latn-100M

Viewer • Updated 1 day ago • 234k • 7

MultilingualUnigramLM/FineWeb2-tur_Latn-100M

Viewer • Updated 3 days ago • 263k • 38

MultilingualUnigramLM/FineWeb2-fin_Latn-100M

Viewer • Updated 3 days ago • 261k • 41

MultilingualUnigramLM/FineWeb2-hun_Latn-100M

Viewer • Updated 3 days ago • 198k • 33

MultilingualUnigramLM/LangMap-TheStack-go-100M

Viewer • Updated 23 days ago • 35k • 56

MultilingualUnigramLM/LangMap-TheStack-typescript-100M

Viewer • Updated 23 days ago • 64.2k • 53

MultilingualUnigramLM/LangMap-TheStack-markdown-100M

Viewer • Updated 23 days ago • 56.5k • 51

MultilingualUnigramLM/LangMap-TheStack-css-100M

Viewer • Updated 23 days ago • 18.6k • 53

MultilingualUnigramLM/LangMap-TheStack-html-100M

Viewer • Updated 23 days ago • 20.3k • 61

MultilingualUnigramLM/LangMap-TheStack-php-100M

Viewer • Updated 23 days ago • 78.3k • 48

MultilingualUnigramLM/LangMap-TheStack-csharp-100M

Viewer • Updated 23 days ago • 79.5k • 96 • 1

MultilingualUnigramLM/LangMap-TheStack-cpp-100M

Viewer • Updated 23 days ago • 26.4k • 52

MultilingualUnigramLM/LangMap-TheStack-c-100M

Viewer • Updated 23 days ago • 28.6k • 55

MultilingualUnigramLM/LangMap-TheStack-java-100M

Viewer • Updated 23 days ago • 73k • 52

MultilingualUnigramLM/LangMap-TheStack-javascript-100M

Viewer • Updated 23 days ago • 47.4k • 55

MultilingualUnigramLM/LangMap-TheStack-python-100M

Viewer • Updated 23 days ago • 49.9k • 107 • 1

MultilingualUnigramLM/FineWeb2-100M

Updated 25 days ago • 19

MultilingualUnigramLM/The-Stack-10K

Viewer • Updated Apr 4 • 300k • 107

MultilingualUnigramLM/FineWeb2-100M-olmo3-7b-toks

Viewer • Updated Mar 14 • 1.22M • 63

MultilingualUnigramLM/FineWeb2-10M

Viewer • Updated Jan 20 • 228k • 7