The final competition for the AI Foundation Model (AI FM) project has narrowed to three finalists: Upstage, SK Telecom (SKT), and LG AI Research. Motif Technologies, which joined the competition through an additional call for proposals, did not pass the second evaluation stage.
On August 18, the Ministry of Science and ICT (MSIT) and the National IT Industry Promotion Agency (NIPA) released the results of the second evaluation for the AI FM project. The evaluation revealed that Motif was eliminated, while Upstage, SKT, and LG AI Research advanced to the next stage.
According to MSIT, the second evaluation was conducted by combining scores from three categories: benchmark evaluation (40 points), expert evaluation (35 points), and user evaluation (25 points).
The benchmark evaluation consisted of 25 points from the Artificial Analysis (AA) Intelligence Index (AAII) and 15 points from the NIA benchmark. The AAII assessed nine high-difficulty benchmarks across various fields, including agent, coding, general knowledge, and science. The NIA benchmark evaluation focused on areas such as mathematics, knowledge, long text comprehension, instruction execution, Korean language, stability, and reliability.
The average score for the four elite teams in the benchmark evaluation was 22.5 points, with a score difference of 4.0 points between the first and fourth places.
The expert evaluation was conducted by a committee of external AI experts. Based on the evaluation documents submitted by the elite teams, the committee conducted an in-depth assessment over about a week, focusing on development strategy and technology (10 points), development achievements and plans (10 points), and impact and contribution plans (15 points). The committee included 10 experts with expertise in AI algorithms, services, and data, all of whom had no conflicts of interest.
The average score for the four elite teams in the expert evaluation was 28.8 points, with a score difference of 2.4 points between the first and fourth places.
The user evaluation, introduced for the first time in this stage, consisted of an AI expert user evaluation (15 points) and a general public evaluation (10 points). The AI expert user evaluation included 49 participants, such as representatives from AI startups, while 185 members of the general public participated in the evaluation. This evaluation aimed to assess not only the performance of the AI models but also the usability and effectiveness experienced by various users.
The average score for the four elite teams in the user evaluation was 17.6 points, with a score difference of 5.0 points, indicating a larger gap compared to the benchmark and expert evaluations.
Meanwhile, MSIT plans to expand support for high-performance graphics processing units (GPUs) B200 for the elite teams advancing to the next stage of the AI FM project.
MSIT stated, "We deeply recognize the need for top-tier independent AI models and are discussing a new support system that goes beyond existing methods with relevant ministries. We plan to announce specific details in the future."
* This article has been translated by AI.
Copyright ⓒ Aju Press All rights reserved.