xlerobot-coffee-model-real-d-force-closed-loop

Method D — the force loop closed. ACT with TA-VLA's L_action + β·L_force: the decoder predicts the future force signal alongside the action chunk, so the force channel has to be modelled rather than passed through.

Trained on the real 17-DoF XLeRobot, two cameras (head + right_wrist).

Why this is a separate method

Method B feeds observation.hpi in as a conditioning token with no loss attached, so the optimizer is free to ignore it. An earlier attempt at a force loss ("Method C") supervised the current frame's HPI broadcast across the chunk — a target the model satisfies exactly by copying its own input token (measured L1 0.000 for a copy). That closes nothing.

Method D supervises hpi[t+1 … t+100]. The same copy scores L1 50.5 against this target, so the objective cannot be shortcut. The encoder is explicitly sliced to the input window so the model is never handed its own target, and frames past the end of an episode are masked via observation.hpi_is_pad so the loss never trains on padding.

hpi_pred_dim=18 supervises [left 9 | right 9] and masks the trailing 6-D whole-body block: it is a hard-zero placeholder, and including it would make a quarter of the loss trivially satisfiable, cutting the effective β on real force to 0.75×.

Contents

folder	run	β	supervised channels
g135_shared_t1_t3_t5_2cam_17dof	fd2_g135_B	0.1	18
g35_shared_t3_t5_2cam_17dof	fd2_g35_B	0.1	18
t1_place_cup_2cam_17dof	fd_t1_B	0.1	18
t3_cup_to_tray_2cam_17dof	fd_t3_B	0.1	18
t5_tray_to_table_2cam_17dof	fd_t5_B	0.1	18

Each folder is a complete pretrained_model (final 100k checkpoint at the root) with a checkpoints/<NNNNNN>/ step sweep alongside.

Architecture

ACT + Module A (A1) — observation.hpi[24] in a dedicated token, multiscale dense window + 1D-CNN encoder, no contact gate — plus the auxiliary future-force head (hpi_pred_enabled hpi_pred_mode=future hpi_pred_dim=18). Requires the Coffee_Automata / hpi_act branch to instantiate.

Recipe

ACT · chunk_size 100 · n_action_steps 100 · batch 8 · 100k steps · seed 1000 · hpi_fps=30 (the data rate). Identical to the Method B 2-camera runs, so the loss term is the only variable between them — the comparison is clean.

Data: IntelligentDecisionLab/xlerobot-coffee-real-2cam, force channel from the per-arm DOB-EKF (observation.hpi[24]).

Comparing against Method B

The matched Method B models are <rung>_2cam_17dof in …-model-real-b-force. Same data, same architecture, same recipe; only the loss differs.

Caveats

• No on-robot evaluation yet. Final training losses (0.041–0.052) are not task success, and are not comparable across methods anyway — Method D's total includes the auxiliary force term, so a lower or higher number does not by itself mean better control.
• 100k steps on ~50 episodes is ~90–110 epochs and likely overfit; the step sweep is provided so evaluation can choose a checkpoint.
• t2_push_button has no rung — it is absent from the 2-camera dataset.

Part of the X-Lerobot Coffee Automata project. AS-CITI Intelligent Decision Lab.