en.Wedoany.com Reported - Recently, the China Academy of Information and Communications Technology (CAICT) held the "Large Model Evaluation and Standards Workshop" during the AI for Good Global Summit in Geneva, Switzerland. Bilel Jamoussi, Deputy Director of the Telecommunication Standardization Bureau of the International Telecommunication Union (ITU), and Yu Xiaohui, President of CAICT, attended and delivered speeches. The workshop aimed to further respond to the recommendations in the United Nations Global Digital Compact regarding "eliminating new digital divides in the field of AI evaluation."

In his speech, Bilel Jamoussi noted that international standards are becoming a crucial support for promoting AI governance and trustworthy applications. The ITU has already released international standards for large model benchmark evaluation and is continuously advancing standardization work on multimodal large models. The joint initiative released at this workshop aims to promote cross-domain and cross-institutional collaboration on evaluation metrics, methods, and standards globally, fostering the safe, inclusive, and beneficial development of AI.

Yu Xiaohui, President of CAICT, pointed out that the large model evaluation system reflects humanity's value choices regarding the direction of AI development. The design of evaluation dimensions and methods will profoundly influence the trajectory of technological evolution and application deployment, serving as a vital foundation for promoting trustworthy AI development while balancing social equity and sustainable development. He stated that the key to advancing international cooperation in evaluation lies in building global consensus, which requires not only strengthening coordination on evaluation technical pathways but also jointly establishing value orientations of safety, trustworthiness, controllability, and beneficence.

Wang Jian, an academician of the Chinese Academy of Engineering, founder of Alibaba Cloud, and director of Zhejiang Lab, stated in his keynote speech that large models, as highly complex systems, make it difficult even for their developers to accurately grasp their capability boundaries, posing significant challenges to evaluation work. He noted that current evaluations are primarily tests of system functions, and in the future, we can draw inspiration from human IQ testing to explore establishing a consistent methodology for measuring machine intelligence. The significance of evaluation lies not only in scoring and ranking but also in helping to understand and advance the technology itself.

Lin Yonghua, Vice President and Chief Engineer of the Beijing Academy of Artificial Intelligence (BAAI); Gilles Thonet, Deputy Secretary-General of the International Electrotechnical Commission (IEC); Nicolas Miailhe, Co-founder of AI Safety Connect; and experts from institutions including Korabench.ai, Amazon Web Services, Future Ethics Lab, the Electronics and Telecommunications Research Institute (ETRI) of South Korea, Huawei, and ZTE shared practices and engaged in in-depth discussions on topics such as scientific foundation models, red team testing, AI behavior measurement, standard interoperability, and international mutual recognition. The participants reached four points of consensus: First, trustworthy AI relies on verification, not declarations, and evaluation is a fundamental capability for promoting AI for good; second, the limitations of static benchmarks are increasingly evident, and evaluation should move towards being dynamic, continuous, and covering the full lifecycle; third, the scope of evaluation should extend from models to applications, balancing long-term risks with real-world impacts; fourth, method sharing and result mutual recognition should be strengthened, promoting standards and evaluation as global public infrastructure, so that evaluation outcomes benefit more countries and groups.

Wei Kai, Director of the AI Research Institute at CAICT, presented the "Fangsheng" large model evaluation system in a keynote report titled "Large Model Evaluation: Practice and Standardization." Wei Kai noted that large model evaluation plays three roles: a "compass" guiding technological innovation, a "connector" bridging technology and applications, and a "safety valve" ensuring the beneficial development of AI. In the future, capability evaluation will place greater emphasis on complex task verification, safety and trustworthiness evaluation will expand to systemic risk identification, and engineering efficiency evaluation will strengthen the synergy between cost and efficiency. The "Fangsheng" system, based significantly on ITU-T F.748.44, covers multiple dimensions including capability, application, safety and trustworthiness, and frontier intelligence. As of July 2026, the system has completed over 1,500 evaluations, built a test dataset of 8.5 million entries, and reduced risks of data contamination and leaderboard manipulation through dynamic testing.

At the workshop, CAICT released the "International Initiative on Joint Development of Large Model Evaluation Systems for AI for Good," advocating for ten aspects: recognizing the value of evaluation, promoting evaluation innovation, improving the evaluation framework, strengthening the data foundation, empowering industrial applications, fostering open collaboration, ensuring evaluation transparency, maintaining evaluation integrity, promoting standard interoperability, and advancing AI for good. The initiative invites international organizations, research institutions, enterprises, and all sectors of society to share datasets and evaluation methods and jointly build an open-source evaluation community to narrow the intelligence divide.

In the future, CAICT will further deepen cooperation with international organizations such as the ITU, IEC, and ISO, as well as global industry, academia, and open-source communities. It will openly share evaluation methods, datasets, and practical experiences, promoting the formation of an open, inclusive, and interoperable AI evaluation ecosystem, contributing Chinese practices to the beneficial development of AI.