Xiaomi-Robotics-1-VLABench

This repository contains the Hugging Face checkpoint used by Xiaomi-Robotics-1 for VLABench evaluation. It includes the model weights, custom Transformers model and processor code, tokenizer files, and VLABench action normalization statistics.

Requirements

The reference environment uses:

Python 3.11
PyTorch 2.8.0
Transformers 4.57.1
FlashAttention 2

The custom model and processor must be loaded with trust_remote_code=True.

For the VLABench simulation environment, follow the companion source repository at eval_vlabench/README.md. The evaluation client uses a separate vlabench conda environment with Python 3.10 and the pinned MuJoCo / dm_control dependencies described there.

Loading

import torch
from transformers import AutoModel, AutoProcessor

model_id = "XiaomiRobotics/Xiaomi-Robotics-1-VLABench"

processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
    use_fast=False,
)

model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    attn_implementation="flash_attention_2",
    dtype=torch.bfloat16,
).cuda()

For reproducible multi-GPU evaluation, use the standard multi-server launcher in the companion Xiaomi-Robotics-1 source repository.

VLABench Evaluation

Use the companion Xiaomi-Robotics-1 source repository and follow eval_vlabench/README.md.

The released processor supports the vlabench_choice robot key.

Action interface

raw action shape:        [10, 60]
executable dimensions:  first 7 dimensions
position delta:          dims 0:3
Euler rotation delta:    dims 3:6
gripper:                 dim 6
action chunk size:       10
replanning interval:     5 steps

Reference evaluation configuration

benchmark:               VLABench
tracks:                  5
tasks per track:         10
episodes per task:       50
total task-track entries: 50
total episodes:          2500
robot type:              vlabench_choice
state dimension:         60
action dimension:        7
action chunk size:       10
replanning steps:        5
CoT during evaluation:   disabled

The five evaluated tracks are:

• track_1_in_distribution
• track_2_cross_category
• track_3_common_sense
• track_4_semantic_instruction
• track_6_unseen_texture

The ten evaluated tasks are:

• add_condiment
• insert_flower
• select_book
• select_chemistry_tube
• select_drink
• select_fruit
• select_mahjong
• select_painting
• select_poker
• select_toy

Reported metrics are success rate (SR), intention score (IS), and progress score (PS).

Reference Results

Overall

Metric	Value
Success Rate (SR)	59.1%
Intention Score (IS)	69.9%
Progress Score (PS)	70.3%

Track values are macro averages across the ten tasks in that track. The overall result is the macro average across all 50 task-track entries and is not episode-weighted.

Results by track

Track	SR	IS	PS
track_1_in_distribution	75.6%	79.8%	85.0%
track_2_cross_category	53.0%	66.4%	66.6%
track_3_common_sense	48.4%	58.2%	58.3%
track_4_semantic_instruction	55.8%	70.2%	66.8%
track_6_unseen_texture	62.6%	74.8%	74.9%
Overall	59.1%	69.9%	70.3%

Average by task across all tracks

Task	SR	IS	PS
add_condiment	40.4%	84.0%	60.1%
insert_flower	40.4%	98.8%	69.2%
select_book	56.4%	75.6%	66.0%
select_chemistry_tube	80.4%	0.4%	85.6%
select_drink	49.2%	90.4%	65.2%
select_fruit	60.8%	94.0%	75.8%
select_mahjong	70.4%	87.6%	73.6%
select_painting	63.2%	82.8%	63.2%
select_poker	66.8%	75.2%	67.1%
select_toy	62.8%	10.0%	77.4%

License

Apache License 2.0. See LICENSE.

Citation

@article{team2026xiaomi,
  title={Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories},
  author={Team, Xiaomi Robotics and Guo, Jun and Jin, Piaopiao and Li, Jason and Li, Peiyan and Li, Yingyan and Liu, Futeng and Peng, Wanli and Qin, Optimus and Su, Yifei and others},
  journal={arXiv preprint arXiv:2607.15330},
  year={2026}
}