en.Wedoany.com Reported - The team of Yang Shuo from Harbin Institute of Technology (Shenzhen) has released TouchWorld, a tactile world model designed to enable robots to not only predict visual changes during dexterous manipulation but also predict and utilize tactile feedback to correct actions. The release of TouchWorld marks a new phase in the team's technical roadmap centered on tactile sensing. Previously, the team had launched EgoTouch to address tactile data collection and TouchAnything to recover tactile information from first-person videos. These three works together form a complete chain from data collection and data augmentation to model application.
Yang Shuo is currently a tenured professor and doctoral supervisor at the School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), and also the founder and CEO of PHANES AI. PHANES AI aims to integrate human video data with tactile perception modalities to build a whole-body mobile dexterous manipulation world model for humanoid robots, establishing a complete capability chain from data and models to control. The company believes that before robots enter real-world manipulation scenarios, aspects such as tactile data, tactile world models, and dexterous hand feedback control remain technically immature, and it hopes to connect these capabilities by starting with tactile data.
From a research perspective, one of the core issues the Yang Shuo team focuses on is the source of learning data for robots. In the field of embodied intelligence, continuous human manipulation data in the real world is becoming an important entry point for robot learning. Works such as NVIDIA EgoScale and Generalist/Gen-1 have shown that large-scale human data pre-training combined with a small amount of real robot data post-training can improve robot success rates in long-horizon, complex tasks. However, the team points out that this approach lacks tactile information. First-person videos allow robots to observe how humans manipulate objects but cannot provide key information such as finger pressure, whether an object is slipping, or whether contact is stable. PHANES AI believes that robots need to understand what happens when the hand contacts an object, which is the motivation behind its development of the EgoTouch, TouchAnything, and TouchWorld series.
The core functionality of TouchWorld is divided into two parts: predictive and reactive. The predictive part means the robot must not only predict future visual frames but also predict the future contact state that should be formed. For example, in a task of pressing a spray bottle button, it is difficult to determine whether the press is sufficient based solely on visual frames, whereas tactile target prediction allows the robot to clearly know the finger contact and pressure required to complete the subtask. The reactive part refers to high-frequency tactile feedback correction. In real-world manipulation, objects may slip, or fingers may not grip stably, requiring the robot to quickly adjust actions based on real-time tactile feedback rather than waiting for the upper-level model to re-plan. In the design of TouchWorld, the inference frequency of the reactive module is four times that of the tactile world model, outputting a correction value each time.
TouchWorld was tested on six real-world robot tasks: watering plants, desktop cleaning, plug insertion, cup insertion, wiping a pan, and pulling a tissue. In an unperturbed environment, the average success rate reached 65.0%; in scenarios with artificially added perturbations, the average success rate was 57.2%. Compared with methods such as Pi-0.5, FTP-1, and GR00T N1.7, TouchWorld outperformed the best-performing baseline model among the compared methods by 15.7 and 16.0 percentage points in the two settings, respectively. These results validate that tactile target prediction and high-frequency feedback correction can improve the stability of robot manipulation when tasks enter the contact phase, demonstrating that tactile sensing can be integrated into robot world models and manipulation policies, not limited to sensor readings.
PHANES AI points out that tactile dexterous manipulation is a systemic problem that cannot be solved by a single model. During the research and development process, the team once attempted to adapt tactile gloves by cutting them and fitting them onto a five-fingered dexterous hand due to the lack of mature tactile solutions for high-degree-of-freedom dexterous hands. However, they encountered issues such as the gloves being easily damaged, system drift caused by the dexterous hand heating up, high data noise, and low collection efficiency. Therefore, the company is investing in data infrastructure, building a low-cost, unobtrusive, and portable multimodal data collection platform that integrates first-person vision, wrist perspective, hand posture, full-palm tactile sensing, and whole-body posture information, with the goal of enabling robots to obtain data closer to the true sensation of a human hand.
PHANES AI aims to complement the systemic capabilities for whole-body mobile dexterous manipulation of humanoid robots, including tactile data collection, tactile estimation, tactile world models, teleoperation and hardware foundations, evaluation systems, and whole-body mobile dexterous manipulation models. Its technical roadmap progresses from EgoTouch and TouchAnything solving the source of tactile data, to TouchWorld realizing tactile prediction and utilization, and then to HumanWBC targeting a closed loop of perception understanding, autonomous mobility, whole-body control, and dexterous manipulation, enabling robots to evolve from being able to see to being able to walk over, pick up, and complete tasks.
The TouchWorld paper has been published on arXiv (arxiv.org/abs/2607.07287), and the project page is phanes-lab.github.io/TouchWorld-website/. The TouchAnything paper (arxiv.org/abs/2605.13083) and its project page (jianyi2004.github.io/TouchAnything-Website/) have also been made public.
