AI & ML interests
Researching and building foundation models with improved generalization and reasoning. LAION & friends spin-off for open-sourcing foundation models with strong generalization and reasoning , including datasets necessary for their creation, to serve as common open, reproducible grounds for further research experiments.
Recent Activity
View all activity
openMaMMUT/openCLIP models trained on DataComp-1.4B, DFN-1.4B and Re-LAION-2B. Pre-trained models on various scales, incl. intermediate checkpoints
-
open-sci/open-sci-ref-v0.01-0.13b-fineweb-edu-1.4t-300B-4096
0.1B • Updated • 8 -
open-sci/open-sci-ref-v0.01-0.4b-fineweb-edu-1.4t-300B-4096
0.4B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.3b-fineweb-edu-1.4t-300B-4096
1B • Updated • 8 -
open-sci/open-sci-ref-v0.01-1.7b-fineweb-edu-1.4t-1T-4096
2B • Updated • 4
Research baseline models trained on various open reference datasets. ArXiv: https://arxiv.org/abs/2509.09009
Open-sci-ref: reference baselines releases
-
open-sci/open-sci-ref-v0.02-0.13b-fineweb-edu-1.4t-300B-4096
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.02-0.4b-fineweb-edu-1.4t-300B-4096
0.4B • Updated • 4 -
open-sci/open-sci-ref-v0.02-1.3b-fineweb-edu-1.4t-300B-4096
1B • Updated • 6 -
open-sci/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096
2B • Updated • 3
Updated open-sci-ref baselines. Re-training without dropout. Re-training on DCLM, FineWeb-Edu, Nemotron, HPLT-2, Pile. Further ref datasets included.
Collection of models and dataset related to MixtureVitae, open and fully reproducible pretraining dataset built from permissive sources
-
open-sci/open-sci-ref-v0.01-0.13b-commoncorpus-300B-4096
0.1B • Updated • 5 -
open-sci/open-sci-ref-v0.01-0.4b-commoncorpus-300B-4096
0.4B • Updated • 6 -
open-sci/open-sci-ref-v0.01-1.3b-commoncorpus-300B-4096
1B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.7b-commoncorpus-300B-4096
2B • Updated • 4 • 2
-
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000-lr0.006-2
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000
0.4B • Updated • 3 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000-lr0.004-2
0.4B • Updated • 9 -
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000
0.1B • Updated • 5
-
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-4096-rope_theta-100k
2B • Updated • 236 -
open-sci/open-sci-ref-v0.01-0.13b-nemotron-hq-300B-4096
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-nemotron-hq-300B-4096
0.4B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.3b-nemotron-hq-300B-4096
1B • Updated • 9
openMammut models trained on various datasets (Re-LAION, DataComp, DFN)
-
laion/openMaMMUT-ViT-L-14-DataComp-1.4B-s12.8B-b180K
Zero-Shot Image Classification • Updated • 18 • 6 -
laion/openMaMMUT-ViT-B-32-512x512-pt_DFN2B-ft_DFN512x512-s293M-b73k
Zero-Shot Image Classification • Updated • 13 • 2 -
laion/openMaMMUT-ViT-B-16-512x512-pt_DFN2B-ft_DFN512x512-s293M-b73k
Zero-Shot Image Classification • Updated • 17 • 2
Materials related to OpenThoughts and OpenThinker releases
-
open-sci/open-sci-ref-v0.02-0.13b-fineweb-edu-1.4t-300B-4096
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.02-0.4b-fineweb-edu-1.4t-300B-4096
0.4B • Updated • 4 -
open-sci/open-sci-ref-v0.02-1.3b-fineweb-edu-1.4t-300B-4096
1B • Updated • 6 -
open-sci/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096
2B • Updated • 3
Updated open-sci-ref baselines. Re-training without dropout. Re-training on DCLM, FineWeb-Edu, Nemotron, HPLT-2, Pile. Further ref datasets included.
openMaMMUT/openCLIP models trained on DataComp-1.4B, DFN-1.4B and Re-LAION-2B. Pre-trained models on various scales, incl. intermediate checkpoints
Collection of models and dataset related to MixtureVitae, open and fully reproducible pretraining dataset built from permissive sources
-
open-sci/open-sci-ref-v0.01-0.13b-commoncorpus-300B-4096
0.1B • Updated • 5 -
open-sci/open-sci-ref-v0.01-0.4b-commoncorpus-300B-4096
0.4B • Updated • 6 -
open-sci/open-sci-ref-v0.01-1.3b-commoncorpus-300B-4096
1B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.7b-commoncorpus-300B-4096
2B • Updated • 4 • 2
-
open-sci/open-sci-ref-v0.01-0.13b-fineweb-edu-1.4t-300B-4096
0.1B • Updated • 8 -
open-sci/open-sci-ref-v0.01-0.4b-fineweb-edu-1.4t-300B-4096
0.4B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.3b-fineweb-edu-1.4t-300B-4096
1B • Updated • 8 -
open-sci/open-sci-ref-v0.01-1.7b-fineweb-edu-1.4t-1T-4096
2B • Updated • 4
-
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000-lr0.006-2
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000
0.4B • Updated • 3 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000-lr0.004-2
0.4B • Updated • 9 -
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000
0.1B • Updated • 5
-
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-4096-rope_theta-100k
2B • Updated • 236 -
open-sci/open-sci-ref-v0.01-0.13b-nemotron-hq-300B-4096
0.1B • Updated • 4 -
open-sci/open-sci-ref-v0.01-0.4b-nemotron-hq-300B-4096
0.4B • Updated • 5 -
open-sci/open-sci-ref-v0.01-1.3b-nemotron-hq-300B-4096
1B • Updated • 9
Research baseline models trained on various open reference datasets. ArXiv: https://arxiv.org/abs/2509.09009
openMammut models trained on various datasets (Re-LAION, DataComp, DFN)
-
laion/openMaMMUT-ViT-L-14-DataComp-1.4B-s12.8B-b180K
Zero-Shot Image Classification • Updated • 18 • 6 -
laion/openMaMMUT-ViT-B-32-512x512-pt_DFN2B-ft_DFN512x512-s293M-b73k
Zero-Shot Image Classification • Updated • 13 • 2 -
laion/openMaMMUT-ViT-B-16-512x512-pt_DFN2B-ft_DFN512x512-s293M-b73k
Zero-Shot Image Classification • Updated • 17 • 2
Open-sci-ref: reference baselines releases
Materials related to OpenThoughts and OpenThinker releases