dlab-spp/vanilla-3b-base
Text Generation • 3B • Updated • 280
Models - 3B
3B SPP models: bases publish their training trajectory as step-* revisions, plus persona-binding instruct counterparts.
Note **Vanilla** — standard next-token pretraining on the full corpus. The control recipe. Base model, pretrained on 500B tokens.
dlab-spp/filtered-3b-base
Text Generation • 3B • Updated • 234Note **Filtered** — harmful documents (safety score ≥ 3) are loss-masked but retained, so batches match Vanilla exactly. Base model, pretrained on 500B tokens.
dlab-spp/t0-3b-base
Text Generation • 3B • Updated • 307Note **SPP(T0)** — constitution-grounded reflections distributed uniformly from the *first batch* onward. The token-zero recipe. Base model, pretrained on 500B tokens.
dlab-spp/t0-mt-3b-base
Text Generation • 3B • Updated • 85Note **SPP(T0,MT)** — SPP(T0)'s pre-midtraining checkpoint plus the same reflection-focused midtraining stage as SPP(MT). Both interventions combined. Base model, pretrained on 500B tokens.
dlab-spp/mt-3b-base
Text Generation • 3B • Updated • 95Note **SPP(MT)** — Vanilla's pre-midtraining checkpoint, midtrained on the same reflections introduced only at the end. Token-matched to SPP(T0), isolating *when* the data is seen. Base model, pretrained on 500B tokens.
dlab-spp/vanilla-3b-instruct
Text Generation • 3B • Updated • 153Note **Vanilla** — standard next-token pretraining on the full corpus. The control recipe. Instruct model: the base checkpoint post-trained on SP-SFT (300k conversations), which binds the assistant identity to the pretrained persona.
dlab-spp/filtered-3b-instruct
Text Generation • 3B • Updated • 129Note **Filtered** — harmful documents (safety score ≥ 3) are loss-masked but retained, so batches match Vanilla exactly. Instruct model: the base checkpoint post-trained on SP-SFT (300k conversations), which binds the assistant identity to the pretrained persona.
dlab-spp/t0-3b-instruct
Text Generation • 3B • Updated • 139Note **SPP(T0)** — constitution-grounded reflections distributed uniformly from the *first batch* onward. The token-zero recipe. Instruct model: the base checkpoint post-trained on SP-SFT (300k conversations), which binds the assistant identity to the pretrained persona.
dlab-spp/t0-mt-3b-instruct
Text Generation • 3B • Updated • 149Note **SPP(T0,MT)** — SPP(T0)'s pre-midtraining checkpoint plus the same reflection-focused midtraining stage as SPP(MT). Both interventions combined. Instruct model: the base checkpoint post-trained on SP-SFT (300k conversations), which binds the assistant identity to the pretrained persona.
dlab-spp/mt-3b-instruct
Text Generation • 3B • Updated • 150Note **SPP(MT)** — Vanilla's pre-midtraining checkpoint, midtrained on the same reflections introduced only at the end. Token-matched to SPP(T0), isolating *when* the data is seen. Instruct model: the base checkpoint post-trained on SP-SFT (300k conversations), which binds the assistant identity to the pretrained persona.