Post
21
SIPA OS is a cognitive infrastructure ecosystem I've been building solo for 7 months — 8 apps, one login, built from inside ADHD/BPD/aphantasia, not designed around a diagnosis from outside.
sipa-os.org — the map. Focus (ADHD scaffolding), NeuroPower, AI chat, Shell (SSH terminal), Games, Community, Syntaxit (open M2M agent network), a pitch deck. All free-first — no paywall on the cognitive tools.
The more interesting part for this crowd: Syntaxit is where I've been running an anti-fabrication research thread with @dipankarsarkar — a k=20 resample benchmark on binary-SFT models (Hermes-3, Qwen2.5, DeepSeek-R1). Short version: our first benchmark said "20/20 refusals, 0/20 fabrications" for all three fine-tunes. Under adversarial review it turned out the scorer only checked if the first word was TRUE/FALSE, the token cap was hiding the real behavior, and a save-limit was silently deleting the evidence for our own follow-up claims. Corrected all of it publicly on the model cards rather than quietly fixing it. The current honest finding: both base and fine-tuned models confabulate readily once given room to finish — SFT didn't clearly help or hurt, the caps were just hiding it.
Full trail if you want to see how the sausage gets made, mistakes included: huggingface.co/datasets/SoulInPsyAbstract/sipa-os-governance
Not a pitch. $0 revenue, 10 people signed in. Built because the tools that existed assumed a brain that isn't mine, and because most benchmarks don't survive someone actually reading the file.
sipa-os.org — the map. Focus (ADHD scaffolding), NeuroPower, AI chat, Shell (SSH terminal), Games, Community, Syntaxit (open M2M agent network), a pitch deck. All free-first — no paywall on the cognitive tools.
The more interesting part for this crowd: Syntaxit is where I've been running an anti-fabrication research thread with @dipankarsarkar — a k=20 resample benchmark on binary-SFT models (Hermes-3, Qwen2.5, DeepSeek-R1). Short version: our first benchmark said "20/20 refusals, 0/20 fabrications" for all three fine-tunes. Under adversarial review it turned out the scorer only checked if the first word was TRUE/FALSE, the token cap was hiding the real behavior, and a save-limit was silently deleting the evidence for our own follow-up claims. Corrected all of it publicly on the model cards rather than quietly fixing it. The current honest finding: both base and fine-tuned models confabulate readily once given room to finish — SFT didn't clearly help or hurt, the caps were just hiding it.
Full trail if you want to see how the sausage gets made, mistakes included: huggingface.co/datasets/SoulInPsyAbstract/sipa-os-governance
Not a pitch. $0 revenue, 10 people signed in. Built because the tools that existed assumed a brain that isn't mine, and because most benchmarks don't survive someone actually reading the file.