Kakao has unveiled artificial intelligence (AI) technology that helps users find necessary information based on the context of their conversations. The company has released embedding models as open source, allowing AI to search for information tailored to past conversations and individual user contexts.
On October 29, Kakao announced the release of two embedding models, 'Kanana 2 Embedding,' on the open-source platform Hugging Face. These models include the 'Kanana-2-3b-embedding' with 3 billion parameters and the 'Kanana-2-1.3b-embedding' with 1.3 billion parameters. This release follows the introduction of four small language models (SLMs) in July. Kakao is developing models that enhance information retrieval technology based on its own language models for use in various service environments.
Embedding models convert the meanings of text into numerical values to find relevant information related to questions. While typical language models focus on understanding questions and generating answers, embedding models calculate the semantic similarity between questions and information to retrieve necessary data.
For instance, if a user wants to find a restaurant mentioned in a past conversation, they can search using contextual phrases like 'the quiet restaurant we talked about before' or 'a good place to chat,' even if they do not recall the exact name. This assists the personalized AI agent in retrieving relevant information from past conversations.
Kakao developed the 3B model by training its small language model, 'Kanana-2-SLM,' to utilize language comprehension for information retrieval. Subsequently, it applied knowledge distillation techniques to transfer the knowledge from the larger model to create the 1.3B model. This approach aims to maintain performance and efficiency in information retrieval while reducing model size.
No Byeong-seok, leader of Kakao's Unified Foundation Model Performance Team, stated, 'Through this release, we aim to assist researchers and developers in leveraging embedding technology, which excels in the Korean language, across various services, while continuing to enhance search performance and efficiency.'
* This article has been translated by AI.
Copyright ⓒ Aju Press All rights reserved.