Big system integration companies as well as AI startups and engineers are engaged in the contest to develop natural language processing (NLP) models on KorQuAD, a machine reading comprehension task website for Korean developers.
The website opened by LG CNS in December 2018 shows who’s who in real time in educating and fine-tuning a Korean language AI model and sophisticating their AI speaker and chatbot.
Players are neck and neck in the race with their rankings determined by even 0.1 point.
As of now, the front runners are LG Science Park in the KorQuAD 1.0 version and Samsung SDS in 2.0.
Early this year, AI chatbot solution provider Skelter Labs topped the list in 1.0 and is maintaining its second and fifth places in 2.0.
Other top runners include state-run Electronics and Telecommunications Research Institute (ETRI) and Naver’s AI Clova Team.
In the 1.0 version, there are already 46 players including programs created by anonymous developers, there were already 46 AI players (programs) who surpassed the human average, and five in 2.0. The human average refers to the average score of adult college graduates for a given problem, which is 80.17 points in exact match (EM) and 91.20 points in the weighted average of precision and recall (F1) in version 1.0 and 68.82 points and 83.86 points, respectively in version 2.0.
KorQuAD was benchmarked from the large-scale dataset Stanford Question Answering Dataset (SQuAD) which allows researchers to design AI models for reading comprehension.
KorQuAD 1.0 is a set of 70,000 Korean standard dataset for AI learning, and KorQuAD 2.0, released last September, provides 100,000 questions and answers generated from longer and more complex documents.
By Shin Chan-ok and Minu Kim
[ⓒ Pulse by Maeil Business Newspaper & mk.co.kr, All rights reserved]