🔄 In a Training Loop

Joseph Jones PRO

KlondikeDev

5 1 5

https://oc.kunix.org/

AI & ML interests

Restoring the Digital Millennium. https://kunix.org/digital-millennium.html

Recent Activity

updated a collection about 4 hours ago

Lucy

repliedto their post about 14 hours ago

Boris-2 coming soon! The Models: Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th. Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th. Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th. Why does 125M get more tokens than 250M? Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model. Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed! We hope to end up in the ballpark of https://huggingface.co/AxiomicLabs/GPT-X2.5-135M or https://huggingface.co/BananaMind/BananaMind-2-Pro-Preview Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon!

repliedto their post about 14 hours ago

View all activity

Organizations

updated a collection about 4 hours ago

Lucy

Collection

Lucy Image Generation Models • 1 item • Updated about 4 hours ago

replied to their post about 14 hours ago

That's not a bad idea -- I was planning on 4096 context, but I might scale up, there's not a reason not to

replied to their post about 14 hours ago

That's not too far off from our original plan, which was FineWeb-Edu, Cosmopedia V2, and DCLM, then some FineMath, but I will definitely look into those other ones, I've been wanting to find some better datasets

updated a model about 14 hours ago

opencerebral/Lucy

Unconditional Image Generation • 71.4M • Updated about 14 hours ago

published a model about 15 hours ago

opencerebral/Lucy

Unconditional Image Generation • 71.4M • Updated about 14 hours ago

New activity in BananaMind/BananaMindBench-Leaderboard 1 day ago

OpenCerebral

#6 opened 1 day ago by

KlondikeDev

OpenCerebral

#5 opened 1 day ago by

KlondikeDev

updated a Space 1 day ago

README

🏢

posted an update 1 day ago

Post

1006

Boris-2 coming soon!

The Models:

Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th.
Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th.
Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.

Why does 125M get more tokens than 250M?

Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.

Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!

We hope to end up in the ballpark of AxiomicLabs/GPT-X2.5-135M or BananaMind/BananaMind-2-Pro-Preview

Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon!