Papers
arxiv:2607.23782

N_0-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens

Published on Jul 26
· Submitted by
Zhou Heng
on Aug 3
#3 Paper of the day
Authors:
,

Abstract

We present N_0-VTLA, a vision-tactile-language-action (VTLA) foundation model capable of (1) fine-grained contact-rich manipulation with tactile perception and tactile-feedback control, and (2) offline policy improvement from stored deployment data. Building on current vision-based backbones, we propose a training recipe for tactile integration consisting of visuo-tactile pre-training, staged tactile-pathway integration, and advantage-conditioned offline policy improvement. During pre-training, the policy learns broad contact priors from NeoData, our large-scale visuo-tactile robot dataset; to our knowledge, N_0-VTLA is the first VTLA model pretrained on tactile data at scale. During post-training, we augment the policy with a predictive tactile pathway that distills the contact patterns learned at scale into the fine motion adjustments required by downstream tactile-centric manipulation. For offline policy improvement, we introduce ALTER, an advantage-conditioned offline reinforcement learning method that converts relative progress and trajectory-event comparisons into binary advantage labels for policy training on a fixed deployment corpus, further improving task-specific learning on contact-rich skills such as deformable object manipulation. Across contact-rich benchmarks, N_0-VTLA outperforms strong baselines by wide margins: it wins all nine real-robot NeoReal tasks and reaches 63.8% mean success on a twenty-task simulation suite, against 44.0% for the strongest baseline. N_0-VTLA policies trained with ALTER reach 75-95% success on three long-horizon real-robot tasks. These results lay a foundation for versatile tactile-driven manipulation policies.

Community

Paper submitter

Hi all, author here đź‘‹

We're excited to share N0-VTLA, a vision–tactile–language–action (VTLA) foundation model built for two things current VLA backbones struggle with: fine-grained contact-rich manipulation with real tactile feedback control, and offline policy improvement from data you've already collected during deployment.

The recipe has three parts:

  • Visuo-tactile pre-training on NeoData, our large-scale visuo-tactile robot dataset. To our knowledge this makes N0-VTLA the first VTLA model pre-trained on tactile data at scale.
  • Staged tactile-pathway integration in post-training, a predictive tactile pathway that distills the contact priors learned at scale into fine motion adjustments for downstream tactile-centric tasks.
  • [ALTER], an advantage-conditioned offline RL method that turns relative progress and trajectory-event comparisons into binary advantage labels, so a fixed deployment corpus can keep improving the policy.

Results: N0-VTLA wins all nine real-robot NeoReal tasks, and reaches 63.8% mean success on our 20-task simulation suite vs. 44.0% for the strongest baseline. With [ALTER], policies hit 75–95% success on three long-horizon real-robot tasks, including deformable object manipulation.

Happy to answer questions here — feedback very welcome!

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2607.23782
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2607.23782 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2607.23782 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2607.23782 in a Space README.md to link it from this page.

Collections including this paper 2