As the application of multimodal foundation AI models continues to expand widely, new security vulnerabilities are constantly emerging, and the possibility of cyberattacks is also increasing. Researchers at Los Alamos National Laboratory have proposed a novel framework to address this challenge, aiming to identify and defend against adversarial threats targeting multimodal foundation models (AI methods capable of seamlessly integrating and processing text and image data). This innovative achievement helps system developers and security experts gain a deeper understanding of model vulnerabilities, thereby enhancing their ability to defend against complex attacks.
Los Alamos computer scientist Manish Bhattarai pointed out: "As the popularity of multimodal models increases, adversaries may exploit their weaknesses through text, visual channels, or a combination of both." AI systems are facing increasingly severe threats, which often stem from subtle malicious manipulations that can mislead or disrupt model outputs. Attackers may use imperceptible perturbations to disrupt the alignment between text and images in multimodal models, resulting in misleading or harmful outcomes.
To address this situation, the Los Alamos team developed a novel method using topological data analysis to discover adversarial features. When attacks disrupt the geometric alignment of text and image embeddings, measurable topological distortions are produced. The researchers precisely quantified these differences through two "topological contrastive loss" techniques, effectively identifying adversarial inputs. Team member Minh Vu said: "Our algorithm can accurately detect attack features, and when combined with statistical techniques, it can detect malicious data tampering with high precision."
The effectiveness of the framework has been rigorously validated on Los Alamos' Venado supercomputer. The computer, installed in 2024, combines CPUs and GPUs to handle high-performance computing and large-scale AI applications. The team tested the framework on multiple benchmark datasets and models. The results showed that the topological approach significantly outperforms existing measures in defending against various known adversarial attacks, providing a more reliable and resilient defense against threats.
