Harbor Eval Visualizations
Explore model pass@k results with interactive dashboard
Explore model pass@k results with interactive dashboard
Note Canonical TRAINING suite: 2238 Harbor data-analysis tasks over real Kaggle datasets. Single correctness reward.
Note Held-out EVAL suite: 366 verified Harbor tasks. For benchmarking, not training.
Note Source-of-truth row-level split manifest (parquet, eval/train, ~30k rows). The Harbor task suites are built from this.
Note Shared Kaggle DATA BUCKET every task pulls from at container start (keyed by BUCKET_PREFIX).
Note Fast-iteration SUBSET: 100 easy (L1) numeric-reward tasks sampled from the train suite.
Note Train suite (2238) with a 3-REWARD verifier: correctness + submission + tool_efficiency (reward.json).
Note Train suite (2238) ORDERED easy->hard by empirical pass@4 difficulty; registry carries rank/difficulty/solve_frac. Curriculum-ready + multireward.
Show an interactive tracking dashboard for your data
Show interactive experiment tracking dashboard
Note 2B · normal · FINAL (step 1119) — pass@4 0.603 vs base 0.284 (~2.1×), pass@1 0.41. GRPO on Harbor data-agent.
Note 2B · normal · BEST-eval (step 800) — peak pass@4 0.624.
Note 2B · curriculum (easy→hard) · FINAL (step 1119) — pass@4 0.603. Same endpoint as normal.
Note 2B · curriculum · BEST-eval (step 900) — peak pass@4 0.606.
Note 4B · normal · FINAL (step 1119) — pass@4 0.791 vs base 0.740, pass@1 0.64. Strong base, modest lift.
Note 4B · normal · BEST-eval (step 200) — peak pass@4 0.802; learned early, plateaued.