Naver’s HyperCLOVA X outperforms rivals in Korean AI test

2024.02.27 11:44:01 | 2024.02.27 11:47:44

[Courtesy of Naver Cloud]이미지 확대

[Courtesy of Naver Cloud]



Naver Cloud announced on Tuesday that its HyperCLOVA X has achieved higher scores than OpenAI and Google’s generative artificial intelligence (AI) models in the South Korean AI performance evaluation system called KMMLU, Measuring Massive Multitask Language Understanding in Korean.

KMMLU is a performance evaluation metric established by the prominent open-source language model research team HAE-RAE in Korea.

The evaluation consists of 35,030 questions of expert-level knowledge across 45 domains such as humanities, social sciences, and science & technology.

About 80 percent of the questions are related to globally applicable extensive knowledge, including mathematical inference capabilities. The remaining 20 percent assess the model’s ability to solve Korea-specific problems, such as the geography of the Korean Peninsula and domestic laws.

The test questions are composed in Korean, allowing for a more accurate assessment of the AI’s understanding of the language and evaluating its universal capabilities along with local knowledge to provide a comprehensive judgment for Korean users, according to Naver.

[Courtesy of Naver Corp.]이미지 확대

[Courtesy of Naver Corp.]



According to the KMMLU research paper, HyperCLOVA X achieved higher scores than OpenAI’s GPT-3.5-Turbo and Google’s Gemini-Pro. It also outperformed OpenAI’s GPT-4 based on Korea-specific knowledge criteria.

Building on the proven performance competitiveness through KMMLU, Naver Cloud plans to further develop HyperCLOVA X into a Sovereign AI solution that has both security and performance.

By Pulse

[ⓒ Pulse by Maeil Business Newspaper & mk.co.kr, All rights reserved]