In the field of high-end equipment manufacturing, robots have undergone years of development, yet their flexibility and perception in object manipulation still fall short of human capabilities, limiting their performance in many real-world tasks, especially in high-temperature environments such as steel mills and foundries where high heat significantly reduces robot performance and operational precision.
Recently, researchers at the University of Southern California achieved a major breakthrough by developing the MOTIF (Multimodal Observation with Thermal, Inertial, and Force Sensors) manipulator. The related results were published on the arXiv preprint server. This new manipulator aims to enhance object manipulation capabilities for humanoid robots.
Team members Daniel Seita, Hanyang Zhou, Wenhao Liu, and Haozhe Lou stated that the research stems from the need for robot manipulation to go beyond traditional visual and tactile perception. Current multi-fingered manipulators often lack the integrated sensing required for complex tasks. Inspired by the human hand, their goal is to develop a versatile, multimodal manipulator that utilizes comprehensive sensor inputs to enable finer object interactions in various environments such as homes and factories.
Previously, the team used commercially available dexterous hands and found that many hardware designs focus on mechanical transmission structures, making it difficult to meet sensing needs. This prompted them to seek a hardware platform that could be DIY-extended. The MOTIF hand was born from this, integrating multiple sensors including thermal sensors, depth sensors, RGB (visual sensors), inertial sensors (IMU), and tactile sensors into a single dexterous platform. It extends the widely used LEAP hand design, enhancing sensing capabilities and offering advantages such as comprehensive environmental perception, safe handling of high-temperature objects, and distinguishing objects with similar appearances but different physical properties.
To evaluate the potential of the MOTIF manipulator, the researchers conducted two experiments in the lab. The first assessed the manipulator's ability to grasp objects while considering temperature and avoiding overheated areas. The second involved flicking objects with fingertips to classify masses of objects with the same shape but different weights.
The results showed that integrating multiple sensors significantly improved the manipulator's manipulation capabilities. Combining thermal sensing with 3D reconstruction effectively guides safe grasping, and inertial sensing enables precise object mass classification. In preliminary tests, the MOTIF manipulator performed excellently, safely grasping various objects and accurately predicting mass.
In the future, the MOTIF manipulator is expected to undergo further improvements and be validated in broader experiments for its potential in specific applications. Seita and his students believe this technology holds promise for handling manual tasks in home environments, professional kitchens, and industrial settings. In tasks such as cooking hot dishes, welding, or tightening screws, it may outperform existing systems due to its ability to safely handle high-temperature objects, accurately estimate forces, and adjust operation strategies based on temperature.
The research team also plans to enhance the MOTIF manipulator's functionality by integrating more high-resolution fingertip sensors (such as Digit 360) to provide finer tactile feedback, while improving multimodal sensing algorithms. They aim to explore its applications in more complex real-world scenarios, such as advanced hand manipulation tasks and environments with intricate thermal interactions. Additionally, Professor Zhou revealed that the team presented this work last week at the ISER 2025 conference in Santa Fe, hoping to inspire the academic community to explore more sensing modalities.
