Xiaomi Scales Robot Policies With Real Trajectories
A two-stage VLA recipe combines 100K hours of UMI trajectories with cross-embodiment post-training, reaching 57.4% on RoboCasa365.
Underlying Paper
Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories
We present Xiaomi-Robotics-1, a foundational vision-language-action (VLA) model capable of (1) following diverse language instructions to perform a wide range of mobile manipulation tasks in unseen environments out-of-the-box, and (2) efficiently adapting to novel downstream tasks with minimal fine-tuning data. We propose a two-stage training recipe consisting of pre-training and post-training. During pre-training, we imbue the model with broad and generalizable action-generation capabilities by training on over 100k hours of real-world manipulation trajectories collected via UMI devices. Crucially, we develop a scalable auto-labeling pipeline that annotates trajectory clips with natural languages describing scene state transitions, providing rich and precise conditioning for action learning. During post-training, we aim to align these capabilities with robot embodiments and imperative instructions that humans naturally use to prompt robots. Extensive experiments demonstrate strong scaling behavior. Xiaomi-Robotics-1 consistently improves with increased data scales and model sizes during pre-training. This scaling behavior directly transfers to post-training, where a stronger pre-training model yields better out-of-the-box real-robot performance in unseen environments. Furthermore, Xiaomi-Robotics-1 serves as a strong robot foundation policy that can be efficiently fine-tuned on complex, dexterous tasks with high data efficiency. Across multiple simulation benchmarks, Xiaomi-Robotics-1 outperforms state-of-the-art methods. Notably, it establishes a new state-of-the-art with a 57.4% success rate on RoboCasa365, surpassing the previous best of 46.6%. Furthermore, it achieves an average score of 20.07 on RoboDojo, significantly outperforming the prior state-of-the-art (13.07). Code and model checkpoints will be released. Project page: https://robotics.xiaomi.com/xiaomi-robotics-1.html
Robot foundation policies have a data problem that looks different from web-scale language modeling. Teleoperation on real robots is expensive and hardware-bound, while human manipulation videos often lack aligned robot actions. Xiaomi-Robotics-1 attacks that bottleneck by treating Universal Manipulation Interface data as the scale source, then using a second training stage to align the resulting action model with robot bodies and imperative instructions.
Core Contribution
The paper’s central claim is that large-scale real-world UMI trajectories can produce transferable action-generation representations for a VLA policy. The authors build a pre-training dataset with more than 100K hours, 1.7K scenarios, 2.4M episodes, and 260+ tasks across household, office, industrial, restaurant, commercial, and outdoor settings. Figure 3 shows the breadth of that corpus, including object and verb distributions rather than only hand-picked examples.
The new part is not just dataset size. The authors pair the data scale with an auto-labeling pipeline that segments trajectories and uses Qwen3.5-27B to generate natural-language descriptions of state transitions. That changes the supervision from sparse task labels into language-conditioned action learning: the model sees what changed in the scene and learns the actions that caused it.
Technical Approach
Xiaomi-Robotics-1 uses a Mixture-of-Transformers design built from a pre-trained VLM, Qwen3-VL, and a diffusion transformer. The VLM encodes observations and language instructions, predicts action chunks through Choice Policies to speed convergence, and passes conditioning information to the DiT. The DiT then generates action chunks with flow matching, while action-related VLM tokens are excluded from the DiT attention computation because the authors found they degraded performance.
The model family spans 2.6B, 5.1B, and 10.5B total parameters. The 10B variant uses a 36-layer VLM with 8.8B parameters and a 1.5B-parameter DiT. Training combines a flow-matching loss, a VLM regression loss for action candidates and scores, and a next-token prediction loss to preserve vision-language ability.
Post-training adds the embodiment bridge. The dataset contains about 10K hours: more than 7.2K hours of in-house mobile-manipulator and dual-arm robot trajectories, more than 1K hours of instruction-labeled UMI data, and open-source robot datasets. Figure 4 shows the cross-embodiment mixture used for this alignment stage.
Actions are represented as relative end-effector pose deltas for arms, base velocity and waist-relative position for mobile robots, and masked dimensions for missing components. That choice is a practical compromise: it gives heterogeneous robots a shared action vector without pretending their control spaces are identical.
Results and Analysis
The scaling experiments support the paper’s main premise. In pre-training, validation action error decreases as the data grows from 12.5% to 100% of a 20K-hour subset; the 12.5% and 25% settings overfit early, while 50% and 100% continue improving. Model scaling from 2B to 5B to 10B also lowers error, but the gap is smaller than the data-scaling gap. The authors’ interpretation is plausible: at billion-parameter scale, the dataset, not model capacity, is the tighter constraint.
Figure 5 captures that result directly, with the data-scaling curve providing the cleaner signal.
The post-training results are more operationally useful. With the 5B model, average out-of-the-box real-robot success rises from 26% without action pre-training to 75% with 100% of the pre-training data across shoe storage, bag packing, table organization, and sofa tidying. Scaling model size after pre-training also helps: the average success rate climbs from 61% for 2B to 75% for 5B and 79% for 10B. The biggest per-task change is shoe tidying, from 58% at 2B to 92% at 10B.
For downstream fine-tuning, Xiaomi-Robotics-1 reaches 75% average success and 90% average progress with under 10 hours per task, compared with 40% and 66% for π0.5. With higher-data fine-tuning, it reaches 85% average success and 94% progress, versus 53% and 79% for Xiaomi-Robotics-0. The gain is largest on tasks requiring dexterous manipulation and mobile manipulation, such as printer refilling and laundry loading.
The simulation benchmarks add scale but also narrow the interpretation. Xiaomi-Robotics-1 reports 74.5% average success on RoboCasa, 57.4% on RoboCasa365 versus 46.6% for ABot-M0.6, and 20.07 average score on RoboDojo versus 13.07 for Hy-Embodied-0.5-VLA. On VLABench, it leads the average success rate at 59.1%, although it does not lead every score dimension. The evidence is convincing that this recipe improves robot policy performance under the paper’s tested settings. It does not prove general robot autonomy; it shows that large, auto-labeled UMI data transfers better than smaller action-pretraining baselines when followed by careful embodiment alignment.
Evidence Box
strongKey Claims
- •Large-scale UMI pre-training improves transferable action generation
- •Cross-embodiment post-training aligns UMI-trained policies with robot embodiments
- •Larger data scale matters more than model size in pre-training
- •Fine-tuning adapts the policy to new dexterous tasks with limited data
Key Results
- •100K+ hours, 1.7K scenarios, 2.4M episodes, and 260+ tasks in the UMI pre-training dataset
- •Out-of-the-box real-robot success rises from 26% without action pre-training to 75% with 100% pre-training data
- •RoboCasa365 success 57.4% (vs. 46.6% for ABot-M0.6)
- •RoboDojo average score 20.07 (vs. 13.07 for Hy-Embodied-0.5-VLA)
Limitations & Caveats
- •Real-robot out-of-the-box evaluation covers 4 tasks with unseen environments and objects, but not unseen task categories
- •Pre-training relies on UMI gripper trajectories and still needs post-training for robot embodiment alignment
- •RoboDojo evaluation omits history observations and scores lower on the memory dimension than Hy-Embodied-0.5-VLA
- •Code and checkpoints are promised but not released in the paper
