xlerobot-coffee-model-real-b-force

Method B — force-added, trained on real-world data. ACT experts for the Coffee Automata chain on the 17-DoF XLeRobot.

One of four repos on the domain × method grid; the companion is …-model-real-a-vision-pos.

Contents

Each folder is a complete pretrained_model directory (final 100k checkpoint at the folder root) with a <folder>/checkpoints/<NNNNNN>/ step sweep alongside. Read the folder suffix before loading — it encodes both the action space and the camera count, and those are the variables these models exist to compare.

folder	action	cameras	run	step sweep
g135_shared_t1_t3_t5_17dof_50k	17-D	1 camera — head	xl_g135_B	2
g135_shared_t1_t3_t5_2cam_17dof	17-D	2 cameras — head + right_wrist	2cam_g135_B	4
g135_shared_t1_t3_t5_2cam_17dof_eval	17-D	2 cameras — head + right_wrist	xl2_g135_B	3
g135_shared_t1_t3_t5_2cam_6dof	6-D	2 cameras — head + right_wrist	2cam6_g135_B	4
g135_shared_t1_t3_t5_6dof	6-D	1 camera — head	real6_g135_B	4
g35_shared_t3_t5_17dof_50k	17-D	1 camera — head	xl_g35_B	2
g35_shared_t3_t5_2cam_17dof	17-D	2 cameras — head + right_wrist	2cam_g35_B	4
g35_shared_t3_t5_2cam_17dof_eval	17-D	2 cameras — head + right_wrist	xl2_g35_B	3
g35_shared_t3_t5_2cam_6dof	6-D	2 cameras — head + right_wrist	2cam6_g35_B	4
g35_shared_t3_t5_6dof	6-D	1 camera — head	real6_g35_B	4
t1_place_cup	legacy	unknown	pre-dates this naming	4
t1_place_cup_17dof_50k	17-D	1 camera — head	xl_t1_B	2
t1_place_cup_2cam_17dof	17-D	2 cameras — head + right_wrist	2cam_t1_B	4
t1_place_cup_2cam_17dof_eval	17-D	2 cameras — head + right_wrist	xl2_t1_B	3
t1_place_cup_2cam_6dof	6-D	2 cameras — head + right_wrist	2cam6_t1_B	4
t2_push_button_17dof	17-D	1 camera — head	real_t2_B	4
t2_push_button_17dof_50k	17-D	1 camera — head	xl_t2_B	2
t3_cup_to_tray_17dof_50k	17-D	1 camera — head	xl_t3_B	2
t3_cup_to_tray_2cam_17dof	17-D	2 cameras — head + right_wrist	2cam_t3_B	4
t3_cup_to_tray_2cam_17dof_eval	17-D	2 cameras — head + right_wrist	xl2_t3_B	3
t3_cup_to_tray_2cam_6dof	6-D	2 cameras — head + right_wrist	2cam6_t3_B	4
t5_tray_to_table_17dof_50k	17-D	1 camera — head	xl_t5_B	2
t5_tray_to_table_2cam_17dof	17-D	2 cameras — head + right_wrist	2cam_t5_B	4
t5_tray_to_table_2cam_17dof_eval	17-D	2 cameras — head + right_wrist	xl2_t5_B	3
t5_tray_to_table_2cam_6dof	6-D	2 cameras — head + right_wrist	2cam6_t5_B	4
t5_tray_to_table_6dof	6-D	1 camera — head	real6_t5_B	4

t4_navigate exists only as t4_navigate_17dof_50k, and only for Method A — it was never trained force-aware. t2_push_button has no 2-cam rung — it is absent from that dataset.

Which weights were actually evaluated

*_2cam_17dof_eval are the models the on-robot evaluation scored (160 rollouts, 109 success / 51 failure). *_2cam_17dof without the suffix is a sibling run: same recipe, same data, same architecture, but a separate training run — GPU kernels are not deterministic by default, so an identical seed does not reproduce identical weights across machines. Cite the _eval folders for any result tied to the robot trials.

Two 17-DoF vintages — read the suffix

*_17dof_50k are the E03/E04 ladder (50k steps, 2-checkpoint sweep), folded in from the retired …-17dof repos so everything lives in four repos instead of six. These are the models the on-robot evaluation rounds were run against. *_17dof without the suffix are this round's 100k retrains. They are different models on the same task — same rung name does not mean same weights.

The two action spaces

*_17dof predicts the whole 17-D action; *_6dof predicts only the arm that performs the task. The pair exists to test whether ACT's loss — a mean over all 17 dims, of which only ~6–9 move — dilutes the signal on the dims that matter.

Measured answer: it does not, to any reliable degree. Over 22 matched pairs (12 sim, 10 real 2-cam), scoring both models on the same 6 acting-arm joints:

set	n	mean Δ	95% CI
sim, 1 cam	12	+4.2%	[−5.1%, +13.5%]
real, 2 cam	10	+9.4%	[−5.2%, +23.9%]
combined	22	+6.5%	[−1.7%, +14.7%]

Positive = 17-DoF lower error. Every interval contains zero and 6-DoF wins 11 of 22, so the two are indistinguishable on accuracy. Prefer *_17dof for deployment — it matches the platform, needs no per-task arm choice, and keeps bimanual behaviour reachable (real t2_push_button genuinely uses both arms) — not because it predicts better.

⚠ Do not compare the two by training loss. ACT's loss is a mean over action dims, so the 17-D and 6-D numbers are different quantities; the 17-D mean is dragged down by frozen channels and appears better for that reason alone.

t2_push_button is a RIGHT-arm task in simulation, not left — the acting arm was chosen per task by measured action variation, not assumed.

Architecture

• 17-DoF: ACT + Module A (A1) · action[17], observation.hpi[24] = [left 9 | right 9 | wholebody 6 zeros] in a dedicated token, multiscale dense window + 1D-CNN encoder, no contact gate. Needs the Coffee_Automata / hpi_act branch to instantiate.
• 6-DoF: ACT + Module A (A1) · action[6], observation.hpi[9] = that arm's block (gripper tau_ext/q/dq + 6-D TCP wrench), multiscale + 1D-CNN, no gate.

Recipe

ACT · chunk_size 100 · n_action_steps 100 · batch 8 · 100k steps · seed 1000 — identical for every model here, so domain / method / action-dim / camera-count are each read in isolation. hpi_fps follows the DATA rate (30 for real and 2-cam, 20 for the 20 fps sim recordings); a mismatch puts the HPI window off the frame grid and the dataset rejects it. Trained on idlab_server1 (RTX PRO 6000 Blackwell) for the 1-camera models and idlab2 (RTX 5090) for the 2-camera ladder.

Normalization statistics are floored per the xlerobot_norm_floor rule (1e-2 state/action, 1e-5 HPI) — the fix for the degenerate-std defect in docs/coffee/EVAL_POSTMORTEM.md. The real recordings carry no observation.hpi; it comes from the published xlerobot-coffee-real-force-sidecar, the same per-arm DOB-EKF output, joined on (episode_index, frame_index).

Loading

PreTrainedPolicy.from_pretrained has no subfolder support, so load through the project helper, or snapshot the folder you want:

from huggingface_hub import snapshot_download
from lerobot.policies.act.modeling_act import ACTPolicy

root = snapshot_download("IntelligentDecisionLab/xlerobot-coffee-model-real-b-force", allow_patterns="t1_place_cup_17dof/*")
policy = ACTPolicy.from_pretrained(f"{root}/t1_place_cup_17dof")

Method B needs the Coffee_Automata / hpi_act branch to instantiate (HPI token).

Caveats

• These are offline-trained policies with no on-robot success numbers yet. The DoF comparison above is action-prediction error, not task success; closed-loop compounding error can rank models differently.
• 1-cam vs 2-cam is not a controlled comparison. The 2-cam dataset has 5–6× the episodes of the 1-cam real set, so camera count and dataset size are confounded.
• No 2-camera simulation data exists, so sim→real cannot be read at two cameras.
• 100k steps on ~50 episodes is ~90–110 epochs and likely overfit; the step sweep is provided so evaluation can pick a checkpoint rather than assume the last one is best.
• t2_push_button Method B is a ~10x error outlier versus Method A on the same task, in BOTH action spaces — a Method-B-on-t2 issue, not a DoF one. Treat T2 force results as suspect until understood.

Part of the X-Lerobot Coffee Automata project. AS-CITI Intelligent Decision Lab.