AlloSpatial: Agentic Harness Framework for Spatial Reasoning in Foundation Models

Spatial reasoning is one of the most stubborn blind spots in today's multimodal foundation models. A system that can write code, analyze images, and hold a fluent conversation will still stumble on something as basic as how far the door is from the telephone, or whether turning left will bring you closer to the sofa. AlloSpatial, a new framework from researchers at Beihang University, traces this failure to a single missing ingredient: models reason from fleeting, first-person snapshots of a scene, but they never build the stable, map-like, bird's-eye representation that brains rely on to find their way through the world.

The team's answer is to give models that missing mental map. At the center of AlloSpatial is World2Mind, a plug-and-play "cognitive mapping sandbox" that turns egocentric video or a handful of multi-view photos into structured allocentric priors. Its key data structure, the Allocentric-Spatial Tree, compresses noisy 3D reconstructions into a compact, machine-readable layout of objects — their positions, footprints, orientations, and how they nest inside one another — alongside route maps that capture what is walkable and where the camera has been. Crucially, the model does not simply trust these maps. A three-stage Spatial Reasoning Harness makes it decide when a question actually needs spatial tooling, gather visual and map evidence through separate channels, and then cross-examine the two for conflicts before committing to an answer.

The results are striking. As a training-free add-on, AlloSpatial lifts frontier systems including GPT-5.2, Claude-4.6-Opus, and Gemini-3-Pro by roughly 5 to 18 points on the VSI-Bench and MindCube spatial benchmarks, with the biggest gains exactly where you would expect — relative direction, route planning, and viewpoint-dependent reasoning. More provocatively, when the same structured maps are baked into small open-weight Qwen3-VL agents through cold-start fine-tuning and reinforcement learning, the resulting 4B and 8B models outscore far larger general-purpose models and purpose-built spatial systems, while answering in roughly a third as many tokens. Even with the pictures taken away entirely, the spatial trees alone carry enough information to support competent reasoning. It is a compelling demonstration that the path to spatially capable AI may run less through ever-bigger visual encoders and more through giving models the right kind of spatial memory.