The second phase evaluation of the 'Independent Artificial Intelligence (AI) Foundation Model (IndepAI)' project, promoted by the Ministry of Science and ICT, is approaching. While participating companies have demonstrated competitiveness in specialized fields such as Korean language, mathematics, and science, significant gaps remain in operating large-scale models and inference efficiency compared to global frontier models.
According to industry sources on August 2, SK Telecom, Upstage, and LG AI Research Institute, which are part of the IndepAI project, unveiled their models on Hugging Face last month. Additionally, Motif Technologies, which joined later, also showcased a preview version.
An analysis of the model cards released on Hugging Face revealed seven benchmarks with similar names: mathematical reasoning, scientific reasoning, advanced comprehensive reasoning, instruction execution, long-form reasoning, Korean knowledge, and Korean culture and common sense. The common benchmarks include evaluations specialized in Korean language and cultural understanding.
SKT's A.X K2 achieved the highest scores among the three models in four benchmarks: Korean culture and common sense (91.6 points), Korean knowledge (80.5 points), long-form reasoning (66.0 points), and mathematical reasoning (97.1 points). Upstage's Solar Open 2 scored relatively high in three benchmarks: scientific reasoning (86.3 points), advanced comprehensive reasoning (28.8 points), and instruction execution (80.0 points).
In contrast, LG AI Research Institute's K-EXAONE 2.0, which ranked first in the initial evaluation, did not achieve the highest score in any of the seven common benchmarks.
LG AI Research Institute leads in model size, with K-EXAONE boasting 750 billion parameters, the largest in the country. The institute plans to enhance data quality and employ reinforcement learning to develop a globally competitive model.
Im Woo-hyung, co-research director at LG AI Research Institute, stated, "The current model is not a finished product but a starting point for developing a frontier-level model. We will enhance K-EXAONE's performance through data quality improvement, follow-up learning, reinforcement learning, and refining inference technology."
However, experts note that the models released are preview versions and not the final submissions, indicating limitations in assessing global competitiveness. The public benchmarks also heavily emphasize specialized areas such as Korean language and cultural understanding, suggesting that the results primarily highlight each company's strengths and technological direction rather than a direct comparison with global models.
Bong Gang-ho, a senior researcher at the Software Policy Research Institute, remarked, "The results are significant in confirming whether global frontier-level competitiveness has been secured in certain areas rather than evaluating if all performance metrics surpass global models. The technologies and development experiences accumulated through the IndepAI project will serve as a foundation for future domestic AI competitiveness."
The competition among global frontier models is intensifying, not only in terms of model size but also in inference efficiency. Recently, China's Moonshot AI unveiled the 'Kimi K3,' which has a total parameter size approximately 3.7 times larger than K-EXAONE and integrates long-form processing, multimodal capabilities, and agent functions.
Notably, Kimi K3 employs a sparse (MoE, Mixture of Experts) structure, selecting only 16 out of 896 experts for computation, reducing the active parameter ratio to about 3.7%. This approach maintains a large model while lowering computational load and costs, thereby enhancing inference efficiency. In contrast, the active parameter ratio for domestic models is around 4.1% to 6.0%, indicating that the technological competition to improve inference efficiency in large models is gaining momentum.
* This article has been translated by AI.
Copyright ⓒ Aju Press All rights reserved.