Massachusetts Institute of Technology researchers have recently developed a new technology that can analyze the internal decision-making mechanisms of protein prediction systems based on large language models. This research was completed by a team led by Professor Bonnie Berger, Simons Professor of Mathematics at MIT, with graduate student Ankur Gujaral as the first author. The related results were published in the Proceedings of the National Academy of Sciences.
In recent years, protein language models have been widely used in the biomedical field, including drug target identification and therapeutic antibody design. Although these models have high prediction accuracy, their internal operating mechanisms have always been in a "black box" state, making it difficult for researchers to understand which specific protein features the models base their judgments on.
The research team used a sparse autoencoder algorithm to successfully analyze the decision-making process of protein language models for the first time. This technology expands the representation of proteins in the neural network from the conventional 480 nodes to 20,000 nodes, allowing the originally tightly compressed information to be dispersed and presented. In this way, each neuron node can more clearly correspond to specific protein features.
To verify the effectiveness of this technology, the researchers used the artificial intelligence assistant Claude to analyze the obtained sparse representations. Claude successfully associated neuron activation patterns with known protein features and was able to accurately describe the biological functional features corresponding to the nodes, for example: "This neuron seems to be detecting proteins involved in ion or amino acid transmembrane transport."
The study shows that these protein language models mainly focus on protein family classification and various metabolic functional features. This breakthrough not only improves the interpretability of the models, but also provides a scientific basis for researchers to select models suitable for specific tasks. Bonnie Berger said: "Our work has broad significance for enhancing the interpretability of downstream tasks that rely on these representations."
The breakthrough in protein language model interpretability technology will help accelerate the process of new drug development and vaccine design. By understanding the basis of model decisions, researchers can more effectively optimize model parameters and improve prediction accuracy. In the future, this technology may also help biologists discover previously unrecognized protein functional features.
