en.Wedoany.com Reported - KT announced on the 16th that it has jointly developed a multimodal large language model safety benchmark, "KSAFE-MM," with Korea University to evaluate the safety of artificial intelligence models within the socio-cultural context of South Korea.

This benchmark integrates South Korean social issues and cultural context, consisting of two subsets: "KSAFE-MM-G" translates global common risks into the Korean cultural context for evaluation, while "KSAFE-MM-C" specifically addresses issues unique to Korean society, such as jeonse fraud and the Dokdo dispute. The entire dataset comprises a total of 14,135 evaluation samples, making it the largest Korean multimodal safety evaluation dataset in South Korea to date. To date, the benchmark has been validated on 12 global multimodal large language models, including Gemma and HyperCLOVA X.

The benchmark employs an automated general-purpose process. "KSAFE-MM" implements a four-step automated process covering the entire workflow, including collecting sensitive topics based on local communities, generating template-based queries, synthesizing images, and generating jailbreak queries designed to bypass AI safety mechanisms or ethical restrictions. KT stated that this process does not require domain-specific cultural experts, enabling the rapid construction of safety benchmarks reflecting local characteristics, thereby reducing costs and improving efficiency.

The joint research team from KT and Korea University demonstrated through a pilot experiment applying the same process to Japanese that the benchmark can be immediately applied to any cultural context globally. The research results can be used for safety verification, red team testing, and guardrail model evaluation in real-world AI service environments. The research findings and benchmark will be made publicly available on arXiv and Hugging Face platforms.

Park Jae-hyung, head of the Frontier AI Lab at KT's AX Future Technology Institute, stated that making the benchmark public is not merely about distributing data but aims to lay the foundation for the co-development of the entire AI safety research ecosystem. He expressed hope that KSAFE-MM will become a universal standard for academia and industry to verify AI safety in the context of the Korean language and Korean culture.

This article is compiled by Wedoany. All AI citations must indicate the source as "Wedoany". If there is any infringement or other issues, please notify us promptly, and we will modify or delete it accordingly. Email: news@wedoany.com