Recently, the Massachusetts Institute of Technology, in collaboration with several academic institutions, released a survey report targeting 30 mainstream agent AI systems. The study points out that agent AI technology currently faces critical issues such as insufficient transparency, lack of regulation, and potential security risks. Developers need to take on more responsibility to enhance system controllability.
The report reveals significant shortcomings in the disclosure of information by agent AI systems. For example, many enterprise agents lack monitoring of individual execution traces; approximately 40% of agents do not provide usage monitoring or only issue notifications when users reach rate limits. The lead author from the University of Cambridge, Leon Staufer, and his collaborators wrote: "We identify persistent limitations in reporting on the ecosystem and security-related features of agent systems." Furthermore, most agents do not disclose their AI nature to end-users or third parties by default, making it difficult to distinguish between human and bot interactions.
Agent AI technology, as a branch of machine learning, aims to enhance the capabilities of large language models and chatbots by connecting to external resources and granting autonomy to perform complex tasks. However, the study found that the transparency issues of agent AI may worsen as the technology develops. Staufer's team stated: "As agent capabilities increase, the governance challenges documented here will become even more important." They cited examples such as Alibaba's MobileAgent and HubSpot's Breeze, which lack documented stop options, potentially posing control challenges for large organizations.
In response to the report's content, some companies have issued statements. A Perplexity spokesperson stated via email that the report "contains significant factual errors" and that they are working with the researchers to correct them. OpenAI pointed out that its Atlas browser agent feature is still in the preview stage and carries risks, mentioning that it has undergone third-party red team testing and proactive monitoring. IBM refuted the study's assertions point by point, emphasizing that it provides detailed public documentation to support the governability of its agent products.
Overall, the MIT research emphasizes that agent AI technology requires significant improvements in transparency and regulation. Developers should proactively document software, audit security procedures, and provide control measures to address potential risks and avoid future regulatory pressure. As tools created by humans, the safety and controllability of agent AI directly depend on the choices and actions of the development teams.
