When the flagship models of the three AI giants OpenAI, Anthropic, and Google go head-to-head, who is better at controlling robots?
On October 20, RoboColiseum, an embodied AI arena jointly initiated by nearly 20 universities, enterprises, and research institutions worldwide, including Mifeng Technology, Hugging Face, NVIDIA, and Tsinghua University, announced the results of this round of evaluation: Claude Opus 5.5 ranked first overall among the four flagship models, followed closely by GPT-6 Astra.
Breaking down the results, among the four flagship models, GPT-6 Astra led in instruction following and overall perturbation adaptation, with an instruction-following score of 99.35; Opus performed better in spatial understanding and general manipulation, and took first place overall in the flagship group with the lowest token consumption across all evaluation rounds. In tests with perturbed image quality, GPT-6.1 Sol lagged significantly behind the other three models, revealing differences in stability across models when responding to changes in visual information.
However, the top two spots in this round's overall ranking were still occupied by embodied policy models, which maintained an advantage especially in the general manipulation dimension.
This head-to-head contest provides the industry with an intuitive reference for understanding the strengths and weaknesses of general large models and embodied policy models.
RoboColiseum focuses on instruction following, spatial understanding, perturbation adaptation, and general manipulation, and has launched four categories of capability sub-leaderboards and 78 high-fidelity simulation evaluation tasks. It not only presents models' performance by category but also records task completion steps and failure locations, helping developers quickly identify shortcomings in model capabilities.
As a co-initiator, Mifeng provides RoboColiseum with high-fidelity object assets and simulation scenarios, supports the construction of evaluation environments for complex robotic manipulation tasks, and provides foundational resources for testing model capabilities under different object interactions, spatial layouts, and environmental conditions.
Whether models' results in simulation can reflect real-world manipulation performance is also key to evaluation. According to actual test results published by the platform, the correlation between RoboColiseum simulation evaluation and real-machine deployment reaches 89.5%, and the evaluation difference for the same model between simulation and the real world is less than 10%, which can provide a reference for model screening and problem identification before developers move to real-machine testing.
Mifeng Technology provides a data supply system that is "comprehensive, high-quality, and fast," covering three major data paradigms: real-machine, simulation, and embodiment-free. In simulation, Mifeng provides two types of resources, object assets and environment assets, covering a wide range of everyday objects and indoor scenes, and offers capabilities such as scene reconstruction, expert trajectory data acquisition, and visualization of evaluation results, helping model teams carry out diverse training and evaluation.
Coliseum originally referred to the circular arena of ancient Rome. RoboColiseum uses this as its imagery to build a multidimensional systematic evaluation system, creating an open, neutral, and reproducible "arena" and "training ground" for embodied intelligence models through simulation evaluation that is highly consistent with real-machine performance. Any university, research institution, or enterprise developer can submit their model to undergo a standardized "exam." Since launching in August, more than 100 teams at home and abroad have carried out model training and evaluation on the platform.
RoboColiseum is now open to developers worldwide, and model teams are welcome to participate in evaluation.
