Arabic
arabic
tokenizer
morphology
nlp
dialect
df-arc / tokenizer.model

Commit History

vocab_size Increased from 64000 to 128000
34fc84b
verified

fr3on commited on

Upload custom Unigram tokenizer (v1)
ea23987
verified

fr3on commited on