GestaltLabs/Ornstein3.6-35B-A3B-RYS-SABER-GGUF
Text Generation • 36B • Updated • 259 • 13
E.M. Rabani PRO
SkyMind
AI & ML interests
Neuromorphic NNs
Recent Activity
liked a model about 20 hours ago
GestaltLabs/Ornstein3.6-35B-A3B-RYS-SABER-GGUF liked a dataset 2 days ago
CritPt-Benchmark/CritPt liked a model 5 days ago
bartowski/Hy3-GGUFOrganizations
None yet
liked a
model about 20 hours ago
liked a
dataset 2 days ago
liked a
model 5 days ago
InternScience/Agents-A1; https://huggingface.co/InternScience/Agents-A1
5
#2642 opened 29 days ago
by SkyMind
IDEA: Bitnet 1.58 (a4.8) version in future variants would be so incredible!
❤️ 6
4
#40 opened 23 days ago
by 
apiarium
liked 2
models 6 days ago
liked a
model 7 days ago
updated a
model 8 days ago
published a
model 8 days ago
liked a
dataset 9 days ago
liked 2
models 10 days ago
reacted to SeaWolf-AI's post with ❤️ 10 days ago
Post
5136
A small gift for anyone building or studying foundation models.
Most "open" models hand you the weights and stop there. With Aether-7B-5Attn we wanted to hand over the whole thing — so you can actually learn from it, reproduce it, and build on it: the data recipe, the training code, every hyperparameter, the complete logs, and the intermediate checkpoints. All Apache-2.0, reproducible byte-for-byte.
What you can do with it:
🔁 Rebuild it from scratch, or fork the recipe for your own model
🔬 Study a real heterogeneous-attention MoE — 49 layers place 5 attention mechanisms on a 7×7 Latin square, arranged as a clean, attributable ablation
📈 Trace training dynamics across the released checkpoints (110k / 115k / 162k)
It's a modest 6.59B model, and an honest one — the limitations (no KV-cache in this build, small scale) are written right in the card. We're not claiming it's special. If any piece of it saves you time or teaches you something, that's exactly what we hoped for. 🤗
📖 Full write-up →
[blog] · https://huggingface.co/blog/FINAL-Bench/opensource-llm
📦 5 Attention Base · FINAL-Bench/Aether-7B-5Attn
🎯 5 Attention Instruct · FINAL-Bench/Aether-7B-5Attn-it
🚀 5 Attention Live demo · FINAL-Bench/Aether-Sovereign-AI
📦 7 Attention Base · https://huggingface.co/FINAL-Bench/Aether-7B-7Attn-base
📦 11 Attention Base · FINAL-Bench/Aether-6B-11Attn-base
🧬 Collection · https://huggingface.co/collections/FINAL-Bench/aether-foundation-model
#opensource #LLM #MoE #reproducibility #Apache2
Most "open" models hand you the weights and stop there. With Aether-7B-5Attn we wanted to hand over the whole thing — so you can actually learn from it, reproduce it, and build on it: the data recipe, the training code, every hyperparameter, the complete logs, and the intermediate checkpoints. All Apache-2.0, reproducible byte-for-byte.
What you can do with it:
🔁 Rebuild it from scratch, or fork the recipe for your own model
🔬 Study a real heterogeneous-attention MoE — 49 layers place 5 attention mechanisms on a 7×7 Latin square, arranged as a clean, attributable ablation
📈 Trace training dynamics across the released checkpoints (110k / 115k / 162k)
It's a modest 6.59B model, and an honest one — the limitations (no KV-cache in this build, small scale) are written right in the card. We're not claiming it's special. If any piece of it saves you time or teaches you something, that's exactly what we hoped for. 🤗
📖 Full write-up →
[blog] · https://huggingface.co/blog/FINAL-Bench/opensource-llm
📦 5 Attention Base · FINAL-Bench/Aether-7B-5Attn
🎯 5 Attention Instruct · FINAL-Bench/Aether-7B-5Attn-it
🚀 5 Attention Live demo · FINAL-Bench/Aether-Sovereign-AI
📦 7 Attention Base · https://huggingface.co/FINAL-Bench/Aether-7B-7Attn-base
📦 11 Attention Base · FINAL-Bench/Aether-6B-11Attn-base
🧬 Collection · https://huggingface.co/collections/FINAL-Bench/aether-foundation-model
#opensource #LLM #MoE #reproducibility #Apache2
- 5 replies
updated a
model 13 days ago
published a
model 13 days ago
liked a
model 13 days ago
liked 3
models 15 days ago
