And if you want long context then add the stack v2 and finepdf but only use that if you want +8K context window
Banaxi PRO
Banaxi-Tech
AI & ML interests
SLMs, training from scratch, LoRA, TTS, Ternary models. AI Interpretability. BCI. Contact at banaxitech@gmail.com
Recent Activity
new activity 7 minutes ago
BananaMind/BananaMindBench-Leaderboard:models to add repliedto KlondikeDev's post about 1 hour ago
Boris-2 coming soon!
The Models:
Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th.
Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th.
Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.
Why does 125M get more tokens than 250M?
Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.
Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!
We hope to end up in the ballpark of https://huggingface.co/AxiomicLabs/GPT-X2.5-135M or https://huggingface.co/BananaMind/BananaMind-2-Pro-Preview
Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon! repliedto KlondikeDev's post about 1 hour ago
Boris-2 coming soon!
The Models:
Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th.
Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th.
Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.
Why does 125M get more tokens than 250M?
Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.
Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!
We hope to end up in the ballpark of https://huggingface.co/AxiomicLabs/GPT-X2.5-135M or https://huggingface.co/BananaMind/BananaMind-2-Pro-Preview
Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon!Organizations
replied to KlondikeDev's post about 1 hour ago
replied to KlondikeDev's post about 1 hour ago
I reccomended you to use
Fineweb edu
Fine phrase
Cosmopedia v2
Finemath
Npset2
Dclm
reacted to ProCreations's post with 🔥 about 5 hours ago
Post
439
new day, new models! Introducing the Auto series, a series of classifiers for determining if agentic tool calls are safe to run or not to prevent any harmful actions from happening. They have high performance compared to other LLMs commonly used for this task with incredible speed and small memory footprints.
ProCreations/auto-1b (recommended generally, much higher accuracy)
ProCreations/auto-0.4b (faster but worse)
Comes with datasets as well (open source ftw)!
A GitHub repo with pi extensions etc will come soon with this model.
ProCreations/auto-1b (recommended generally, much higher accuracy)
ProCreations/auto-0.4b (faster but worse)
Comes with datasets as well (open source ftw)!
A GitHub repo with pi extensions etc will come soon with this model.
- 2 replies
replied to their post about 7 hours ago
Today it will start train, not released.
reacted to their post with 🚀🔥 about 7 hours ago
Post
287
Today.
- 3 replies
reacted to KlondikeDev's post with 🔥 about 11 hours ago
Post
470
Boris-2 coming soon!
The Models:
Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th.
Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th.
Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.
Why does 125M get more tokens than 250M?
Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.
Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!
We hope to end up in the ballpark of AxiomicLabs/GPT-X2.5-135M or BananaMind/BananaMind-2-Pro-Preview
Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon!
The Models:
Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th.
Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th.
Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.
Why does 125M get more tokens than 250M?
Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.
Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!
We hope to end up in the ballpark of AxiomicLabs/GPT-X2.5-135M or BananaMind/BananaMind-2-Pro-Preview
Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon!
- 4 replies
reacted to KlondikeDev's post with 👀 about 12 hours ago
Post
470
Boris-2 coming soon!
The Models:
Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th.
Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th.
Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.
Why does 125M get more tokens than 250M?
Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.
Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!
We hope to end up in the ballpark of AxiomicLabs/GPT-X2.5-135M or BananaMind/BananaMind-2-Pro-Preview
Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon!
The Models:
Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th.
Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th.
Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.
Why does 125M get more tokens than 250M?
Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.
Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!
We hope to end up in the ballpark of AxiomicLabs/GPT-X2.5-135M or BananaMind/BananaMind-2-Pro-Preview
Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon!
- 4 replies
replied to their post about 13 hours ago
or if you really wanna get small, with only 3-4MB minimum ram then 5m
replied to their post about 13 hours ago
but ill also get rose-mini to work at 2 bit and 4 bit
replied to their post about 13 hours ago
10M, or if you want a bit more then 15M
replied to their post about 14 hours ago
you gotta make Rose-Micro...
replied to their post about 14 hours ago
Though your rose models are a bit too big so ill only add 2 bit and 4 bit no 8 bit
replied to their post about 14 hours ago
ill add it, but code only at 100 followers onhttps://huggingface.co/BananaMind 🔒
replied to their post about 14 hours ago
But it would need a architecture implementations, but if you want me to add any, i can add.
replied to their post about 14 hours ago
Yes it could run other models!
replied to their post about 14 hours ago
Its from scratch a minimal kernel
You could use LFM2.5 230m, lfm 2.5 350m or bananamind 2 pro, or if you can wait use bananaminf 2 ultra with 400m
wdym newer models i could choose? You mean ones to finetune?