en.Wedoany.com Reported - Chinese Danfei Xu has launched a first-person human manipulation data ecosystem and standard called EgoVerse, aiming to build a catalyst for robot learning similar to ImageNet's role in visual AI. Danfei Xu is currently an Assistant Professor at the School of Interactive Computing at the Georgia Institute of Technology and Director of the Robot Learning and Reasoning Lab (RL2). During his Ph.D. at Stanford University, he was jointly advised by Fei-Fei Li and Silvio Savarese. EgoVerse is a sustainably growing data infrastructure, with alliance partners including Georgia Institute of Technology, Stanford University, ETH Zurich, University of California San Diego, as well as companies such as Meta, Scale AI, Lightwheel Intelligence, Mecka AI, MicroAGI, and Trace Labs. Lightwheel Intelligence is the only Chinese company participating.
Robot learning has long faced a data acquisition bottleneck. Current training still relies on real-world teleoperation, where each additional demonstration requires hardware, personnel, and space, resulting in high costs and limited task scenarios. Human behavior naturally contains rich visual and motion information, making it a more ideal training source. However, previous human datasets were mostly one-off projects with inconsistent collection equipment, coordinate systems, and task definitions, making it difficult to combine them for training. EgoVerse is designed as a "living" collaborative framework to break this limitation, allowing global embodied data to flow under the same "language system."
The current public version of EgoVerse includes 1,362 hours of human demonstration data, approximately 80,000 episodes, 1,965 tasks, 240 scenes, and 2,087 collectors. The data covers first-person video, camera poses, 3D hand information, and fine-grained language descriptions, with transfer effects validated across different laboratories, robots, and embodiments.
EgoVerse brings together multiple leading institutions to advance the project. Georgia Institute of Technology serves as the organizational hub and research framework, with Danfei Xu's team responsible for representative task validation. Professor Shuran Song's team at Stanford University, through their established UMI (Universal Manipulation Interface), enables human manipulation data to be transferred to different robot platforms. Meta provides the project with dedicated data collection glasses, Project Aria. Mecka AI promotes first-person collection using iPhones and lightweight head-mounted devices, lowering hardware barriers. Scale AI handles data cleaning, annotation, and quality management. MicroAGI explores distributed crowdsourcing collection. Trace Labs also participates. Lightwheel Intelligence's contribution is establishing a quality control system spanning edge-side collection, cloud processing, and simulation validation.
Lightwheel Intelligence breaks down human data quality into three stages. First, an agent-driven real-time monitoring system oversees the collection process, controlling task completeness, action validity, and data distribution, with on-site correction during collection. Second, a cloud platform compatible with multiple collection hardware achieves unified data standards, handling Visual Inertial Odometry (VIO), 3D hand and body annotation, and V7-level action semantic annotation; Lightwheel Intelligence's human data solution, EgoSuite, provides these annotations at production scale. Finally, the simulation platform SimFoundry and evaluation platform RoboFinals verify whether the data is truly valuable for robot learning, with evaluation results guiding the next round of collection to form a closed-loop data quality system.
Lightwheel Intelligence is also participating in another global physical AI infrastructure project, Newton. This project is jointly initiated by NVIDIA, Google DeepMind, and Disney Research, with a technical steering committee composed of NVIDIA, Google DeepMind, Disney Research, Lightwheel Intelligence, and the Toyota Research Institute (TRI), aiming to define the technical roadmap for robot simulation and physical modeling. Lightwheel Intelligence is currently the only Chinese company simultaneously participating in both EgoVerse and Newton, two international standard systems, transitioning its role from a standard user to a co-definer.
