Robot Foundation Models

Robot foundation models are broadly trained models intended to support more than one task, environment, or robot. Their inputs may include images, language, robot state, and demonstrations; their outputs may include representations, plans, or actions.

The central idea

Traditional robot learning often builds a model for one task and hardware setup. Foundation-model research asks whether diverse datasets can produce reusable knowledge about objects, instructions, spatial relationships, or action that can be adapted rather than rebuilt.

Latent representations are one route from unlabeled video to action-oriented pretraining. Our latent actions evidence guide examines 41 design choices, proxy metrics, data scaling, and the bounded Franka trials behind a current comparative study.

Reusable knowledge does not automatically become executable motion on a new body. Our embodiment gap guide maps what can transfer across robots, what still changes in data, calibration, control, contact, safety, and recovery, and which adaptation costs a final success rate can hide.

Vision-language-action models

A vision-language-action model connects observations and instructions to robot actions. The name describes an interface, not a guarantee of generality or reliability. Systems differ in data, architecture, action horizon, supported hardware, and the safeguards around execution.

Gemini Robotics 2 makes the stack boundary concrete. Google separates a whole-body VLA that outputs motor actions, an embodied-reasoning model that plans and calls tools, and an on-device VLA for local inference. Our Gemini Robotics 2 evidence audit maps each model’s access, embodiments, evaluations, safety limits, and missing deployment evidence.

The data challenge

Internet-scale text and images are abundant; high-quality robot trajectories are not. Robot data is expensive, heterogeneous, and tied to particular bodies. Research therefore combines demonstrations, teleoperation, simulation, video, synthetic data, and cross-robot normalization. Our guide to how robot training data is made compares those collection methods with language labels, interventions, EEG, and EMG.

Ego2Robot turns egocentric human video into rendered robot-action streams across 15 morphologies. The Ego2Robot evidence audit explains why its 18,561-hour output is synthetic training duration rather than robot operating time, and what its matched VLA and physical evaluations support.

Xiaomi-Robotics-1 offers a current test of that strategy at unusual scale. Xiaomi reports more than 100,000 hours of handheld UMI manipulation trajectories for pretraining, followed by a separate cross-embodiment mixture to align the policy with mobile manipulators and dual-arm robots. On August 3, Xiaomi released Apache-2.0 post-training and inference code, a 5B checkpoint, and three simulation evaluation paths. Our updated Xiaomi-Robotics-1 evidence audit separates the data-volume claim from robot operating time, examines the real-robot protocols, and maps what the artifacts do and do not make reproducible.

FLUX-mimic tests a different route. It starts with the latent representation of a generative video backbone, then decodes robot action chunks without rendering a video during inference. The FLUX-mimic evidence audit examines its 20-trial kitting benchmark, edge latency, Audi task evidence, and the gap between a named factory evaluation and recurring production.

Success rates are meaningful only inside a declared protocol. HumanCLAW reports 16.8% interaction success for the strongest tested VLM on a simulated humanoid benchmark, while TurboVLA reports 97.7% across LIBERO tabletop tasks. These are not contradictory results or a shared leaderboard. Our practical audit of robot benchmark success rates explains how task scope, embodiment, motor execution, trial counts, compute, and intervention rules determine what each percentage means.

Benchmark suites can also repeat the same evidence. Our audit of physical AI benchmark redundancy examines a 51-model by 12-benchmark matrix, two close substitute pairs, rank sensitivity, compact-suite selection, and the general vision-language factor hidden inside a physical AI label.

Intervention is also a measurable control process, not an invisible exception. Our guide to robot handoff evidence uses AutoIntervene to separate failure detection, operator takeover, recovery, return to policy control, and post-correction learning.

World action models add future-state prediction to robot action learning, but realistic output is not sufficient evidence of a faithful simulator. Our world action model evidence guide now connects action fidelity and contact grounding with action-free surgical video, multi-stream supervision, adaptive replanning, planning diagnostics, and long-horizon recovery.

Dynamic manipulation adds another evaluation requirement: the world keeps moving while a policy computes. Our guide to latency, recovery, and process quality explains ReflexBench’s timing-aware tasks and PRM-as-a-Judge 1.5’s progress metrics, including where learned process judges remain unreliable.

Safety needs its own outcome and evidence chain. Our adaptive robot safety case guide uses ManiGuard to separate task success, safe success, engagement, formal specifications, runtime monitoring, distribution shift, physical validation, and change control.

Long-horizon control adds interfaces between task planning, memory, action generation, and body coordination. Our long-horizon robot control evidence guide uses Tau-0-VLA and HAF to distinguish high-level subtask search from low-level whole-body execution and online adaptation.

Online self-improvement adds another split between the policy that proposes actions and the value model that selects among them. Our Q-Planning evidence audit examines how failed rollouts update a separate critic while the large behavior-cloning policy stays frozen, along with the physical trial counts, human labels, exploration boundary, latency, and missing safety record.

VLA systems also depend on boundaries before final action selection. Our VLA control interfaces guide compares prompt authority, typed spatial grounding, behavior-intent supervision, and event-selected memory, including the physical denominators and unresolved deployment limits behind each result.

Questions for readers

Ask which robots and tasks appear in training, how performance is measured, whether results transfer to new settings, and how failures are detected. A broad model can still require task-specific data and conventional control.

A current integration layer

LeRobot v0.6.0 provides one concrete view of how robot-model development is becoming a stack rather than a single checkpoint. The open-source release integrates multiple VLA and world-model policy families, reward models, simulation benchmarks, real-robot rollouts, and human correction capture. Integration makes models easier to access and compare; it does not independently prove upstream performance claims or real-world transfer. Read the LeRobot v0.6.0 practical guide for the complete workflow and evidence boundaries.

Compare the publicly disclosed AI stacks behind Helix, Redwood, Carbon, Atlas, Apollo, and other systems in Humanoid Robots in 2026. See the glossary for key vocabulary and What Is Physical AI? for the wider stack.