qikp/wordmix
Viewer โข Updated โข 26.1k โข 35
How to use qikp/pika-3 with Transformers:
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("qikp/pika-3", dtype="auto")๐ You are looking at pika 3, which uses the wordmix dataset!
pika is a simple and public domain-like tokenizer.
[EOS] (ID 0)[PAD] (ID 1)pika was trained on qikp/wordmix.
Some uncommon special tokens aren't present, you'll have to add them manually if needed.