gangweix/next-forcing-base

Next Forcing — Base Checkpoint

Base checkpoint for Next Forcing: Causal World Modeling with Multi-Chunk Prediction. This is the causal video-action backbone used as the initialization for post-training.

• 📄 Paper: https://arxiv.org/pdf/2606.11187
• 🌐 Project page: https://gangweix.github.io/next-forcing/
• 💻 Code: https://github.com/gangweix/next-forcing
• 🤖 RoboTwin post-trained model: gangweix/next-forcing-posttrain-robotwin

Model Description

Next Forcing addresses the myopic supervision problem in autoregressive video world models: next-chunk denoising tends to learn local appearance shortcuts instead of long-range dynamics, especially at high frame rates. Lightweight Multi-Chunk Prediction (MCP) modules predict multiple future chunks through a causal chain during training, providing dense temporal supervision to the backbone.

This base checkpoint contains the backbone without MCP modules. The MCP modules are created at the start of post-training and initialized from the last mcp_blocks_per_depth backbone blocks (mcp_init_from_backbone = True), so the post-trained checkpoint is correspondingly larger (6.7B vs 5.1B).

Parameters	5.1B (BF16)
Backbone layers	30
MCP modules	Not included (added during post-training)

Built on top of the LingBot-VA codebase.

Repository Layout

transformer/     Causal video-action backbone
vae/
text_encoder/
tokenizer/

Usage

Clone the code and install the dependencies as described in the repository README.

python -m pip install "huggingface_hub[cli]"
hf download gangweix/next-forcing-base \
  --local-dir ./checkpoints/next-forcing-base

The training code resolves model subfolders by path, so point NEXT_FORCING_PRETRAINED_MODEL_PATH at the local directory, not at the Hub repository id:

export NEXT_FORCING_PRETRAINED_MODEL_PATH=$PWD/checkpoints/next-forcing-base
export NEXT_FORCING_DATASET_PATH=/path/to/your/dataset
export NEXT_FORCING_SAVE_ROOT=/path/to/your/output

NGPU=8 CONFIG_NAME=robotwin_train \
bash script/run_va_posttrain.sh --init-worker 1

MCP defaults (wan_va/configs/mcp_train_config.py): num_mcp_depths = 3, mcp_blocks_per_depth = 3, mcp_hidden_collect_layers = [3, 11, 19, 29], mcp_loss_weights = [0.5, 0.2, 0.1].

License

Released under the Apache License 2.0. Next Forcing is developed on top of the LingBot-VA codebase; please retain the upstream attribution and license when redistributing.

Citation

@article{xu2026next,
  title={Next Forcing: Causal World Modeling with Multi-Chunk Prediction},
  author={Xu, Gangwei and Zhang, Qihang and Zhou, Jiaming and Zhu, Xing and Shen, Yujun and Yang, Xin and Xu, Yinghao},
  journal={arXiv preprint arXiv:2606.11187},
  year={2026}
}