open-sci-ref 0.02
Collection
Updated open-sci-ref baselines. Re-training without dropout. Re-training on DCLM, FineWeb-Edu, Nemotron, HPLT-2, Pile. Further ref datasets included.
•
1 item
•
Updated
AI & ML interests
Researching and building foundation models with improved generalization and reasoning. LAION & friends spin-off for open-sourcing foundation models with strong generalization and reasoning , including datasets necessary for their creation, to serve as common open, reproducible grounds for further research experiments.
Recent Activity
View all activity
JJitsev
updated
a
collection
about 8 hours ago
JJitsev
updated
2
collections
15 days ago
Taishi-N324
authored
a
paper
4 months ago
Harsh1729
updated
16
models
4 months ago
open-sci/open-sci-ref-v0.01-1.7b-nemotron-cc-2024-hq-real-synth-mix-1T-8192
2B
•
Updated
•
2
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-4096
2B
•
Updated
•
1
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-16384
2B
•
Updated
•
1
open-sci/open-sci-ref-v0.01-1.7b-slimpajama-300B-4096
2B
•
Updated
•
3
open-sci/open-sci-ref-v0.01-1.7b-pile-300B-4096
2B
•
Updated
•
2
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-300B-8192
2B
•
Updated
•
1
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-300B-4096
2B
•
Updated
•
1
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-300B-16384
2B
•
Updated
•
11
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-8192
2B
•
Updated
•
2
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-4096-rope_theta-100k
2B
•
Updated
•
1
open-sci/open-sci-ref-v0.01-1.7b-hplt-2.0-300B-4096
2B
•
Updated
•
1
open-sci/open-sci-ref-v0.01-1.7b-fineweb-edu-1.4t-300B-4096
2B
•
Updated
•
11
open-sci/open-sci-ref-v0.01-1.7b-fineweb-edu-1.4t-1T-4096
2B
•
Updated
•
1
open-sci/open-sci-ref-v0.01-1.7b-dclm-300B-4096
2B
•
Updated
•
1
open-sci/open-sci-ref-v0.01-1.7b-dclm-1T-4096
2B
•
Updated
•
1
open-sci/open-sci-ref-v0.01-1.7b-commoncorpus-300B-4096
2B
•
Updated
•
1
•
1