In the past, professional programming knowledge was essential to teach robots new skills, but the new generation of robots now has the potential to learn from anyone. Engineers are now working to design robot assistants capable of “learning from demonstrations,” allowing humans to guide robots in completing tasks in more natural ways, such as teleoperation, physically moving the robot, or having the robot observe and imitate human operations.
However, traditional robots that learn by doing have mostly been trained using a single demonstration method. Engineers at the Massachusetts Institute of Technology have broken through this limitation by successfully developing a three-in-one training interface—the Versatile Demonstration Interface (VDI)—which enables robots to learn tasks through any of the three demonstration methods.
VDI is a handheld, sensor-equipped tool that easily connects to many common collaborative robotic arms. With this attachment, users can freely teach the robot tasks through teleoperation, physical manipulation, or personal demonstration, flexibly adapting to different user preferences and task requirements.
The MIT research team rigorously tested VDI on standard collaborative robotic arms. Volunteers with manufacturing expertise used the interface to successfully complete two common factory workshop manual tasks—press-fitting and forming. In the press-fitting task, the robot learned to precisely press nails into holes; in the forming task, it mastered the skill of evenly rolling rubbery, dough-like material onto a central rod.
The research team stated that the new interface significantly enhances training flexibility, expanding the types of users and “teachers” who can interact with robots, enabling them to learn a broader range of skills. For example, in handling toxic substances, personnel can train the robot remotely; downstream on the production line, others can manually operate the robot for packing; and at the end of the line, someone can use the attachment to draw a company logo, which the robot simultaneously observes and learns.
Mike Hagenow, postdoctoral researcher in the Department of Aeronautics and Astronautics at MIT, said: “We are committed to creating highly intelligent and skilled robot ‘teammates’ that can efficiently collaborate with humans to complete complex work. We firmly believe this flexible demonstration tool is not only suitable for manufacturing workshops but can play an even greater role in homes, caregiving environments, and other settings, driving the continuous expansion of robot applications.”
Hagenow will present a detailed paper on the new interface at the IEEE International Conference on Intelligent Robots and Systems (IROS) in October, which has also been published on the arXiv preprint server. Co-authors include Dimosthenis Kontogiorgos, postdoctoral researcher at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), Yanwei Wang, PhD in Electrical Engineering and Computer Science, and Professor Julie Shah, Director of the Department of Aeronautics and Astronautics at MIT.
Professor Shah’s research group designs robots that can work side-by-side with humans in workplaces, hospitals, and homes. One of their research focuses is developing systems that allow people to naturally teach robots new tasks or skills on the job, helping factory workers quickly adjust robot movements and improve current tasks without reprogramming software.
The team’s new research builds on the emerging robot learning strategy “Learning from Demonstrations” (LfD), aiming to train robots in a more natural and intuitive way. While reviewing LfD literature, Hagenow and Shah discovered that existing training methods generally fall into three categories: teleoperation, kinesthetic teaching, and natural teaching, with one method often being more advantageous for specific people or tasks. This inspired them to design a tool combining all three methods, allowing robots to learn more tasks from more people.
To achieve this goal, the team carefully designed VDI. The interface serves as a handheld attachment equipped with cameras and markers to track tool position and motion, as well as force sensors to measure pressure applied during specific tasks. When connected to the robot, the entire robot can be teleoperated, with the camera recording actions as training data; humans can also physically manipulate the robot through the interface to complete tasks. Additionally, VDI can be detached and held by hand to perform tasks, with the camera recording actions for the robot to later replicate when reconnected.
To test the attachment’s usability, the research team brought the interface and collaborative robotic arm to a local innovation center and invited manufacturing experts to participate in the experiment. Volunteers used all three training methods of the robot and interface to complete press-fitting and forming tasks. Results showed that volunteers generally preferred the natural method but also noted the advantages of different methods in specific scenarios—such as teleoperation for handling dangerous or toxic substances, kinesthetic teaching for adjusting positioning when handling heavy packages, and natural teaching for demonstrating fine manipulation tasks.
Hagenow said: “We envision applying the demonstration interface in flexible manufacturing environments, allowing a single robot to assist with a series of tasks that benefit from specific types of demonstrations.” He plans to improve the attachment design based on user feedback and test the robot’s learning capabilities, believing that interfaces expanding the ways users interact with robots can achieve greater flexibility in collaborative robotics.
