en.Wedoany.com Reported - China's UniX AI, in collaboration with Zhejiang University, the Massachusetts Institute of Technology, the University of Oxford, Yale University, and Shanghai Jiao Tong University, has released a new research achievement, UniTac. It proposes a cross-sensor tactile understanding and generation architecture, which is integrated into a vision-language-action model, enabling robots to predict tactile states before contacting objects. This work has been accepted by ECCV 2026.
Over the past year, embodied foundation models have developed rapidly, allowing robots to perform object manipulation tasks with clear contours. However, when dealing with flexible objects, fragile objects, or dense contact scenarios, robots relying solely on vision are inadequate. For example, tasks such as picking up a grape without crushing it or grasping a paper cup without deforming it require human-like tactile perception capabilities. As research progresses, teams like UniTouch and AnyTouch have attempted to unify the representation methods of different tactile sensors. FuSe and Tactile-VLA integrate tactile information into general policies and VLA models, while TacImag explores enabling robots to "imagine" touch using only vision and proprioception. However, these methods still face challenges such as heterogeneous sensor data and the dispersion of tactile understanding and generation across different models.
UniTac learns a unified tactile representation through large-scale, multi-sensor data, enabling robots to predict the tactile state of an object after contact based on visual information, thus achieving "pre-touch perception." Taking the grasping of a paper cup as an example, the VLA model integrated with UniTac can predict the deformation of the cup body under force, thereby determining the contact position and force; the VLA model without UniTac directly crushes the paper cup. Tactile information thus shifts from post-contact feedback to pre-contact prior knowledge.
Interaction between robots and the physical world requires three levels of grounding: the vision-language model maps abstract concepts to real-world objects, the vision-language-action model associates semantics with action trajectories, and the third level of grounding requires robots to understand and predict the physical consequences during contact. VLA models that rely entirely on vision tend to apply the same strategy to visually similar objects, and by the time corrections are made after contact, fragile objects may have already deformed. UniTac shifts tactile state prediction to before the first contact, providing the VLA with prior information about the contact process, making the first contact safer.
The core challenge addressed by UniTac is the cross-sensor unification of tactile data. Visual tactile sensors (such as GelSight, DIGIT, and Duragel) record the deformation of a gel under pressure using a camera, with images containing a mixture of object properties and sensor configuration information. The model must distinguish which changes originate from the object (hardness, roughness, texture) and which from the sensor (lighting, gel state, marker layout). To solve the sensor heterogeneity problem, UniTac decomposes touch into two parts: physical properties and sensor configuration. The understanding module simultaneously learns object property descriptions and sensor identification, forcing the model to distinguish signal sources; the generation module incorporates sensor tokens into the generation process to avoid unconditional tactile generation. Final experiments show that UniTac achieves leading performance in both understanding and generation: on PHYSICLEAR-Test, UniTac-7B scores 66.51, surpassing tactile understanding models like Octopi and unified models like BLIP3o; it achieves 64.61 in the attribute-object matching task. Across four sensor types—Digit, GelSight, GelSight Mini, and Duragel—UniTac achieves an average SSIM of 0.836 and an average PSNR of 19.93.
UniTac was led by UniX AI. The company, a full-stack embodied intelligence enterprise, was founded in Suzhou in April 2024 by Dr. Fengyu Yang, a Yale University graduate. Fengyu Yang serves as a co-first author of the UniTac paper. Co-first author Jiahang Tu is from Zhejiang University, with research interests in computer vision and generative models; corresponding author Hanbin Zhao is an assistant professor at Zhejiang University, specializing in machine learning and generative models; Zhi Tao from UniX AI is responsible for real-world algorithms and system implementation. Paper collaborators also include Chenyang Ma from the University of Oxford, Xihang Yu from the Massachusetts Institute of Technology, Ziyao Zeng and Alex Wong from Yale University, and Shaokai Wu from Shanghai Jiao Tong University. Previous works such as the Touch and Go data collection, UniTouch multi-sensor representation unification, and TaRF visual-tactile 3D alignment have all advanced along the tactile research line. UniTac integrates tactile understanding, generation, and cross-sensor transfer into a unified architecture and provides tactile priors for VLA models, promising to enhance the compliance and safety of robot manipulation in scenarios such as home care, precision assembly, and service operations.
