Trillion Labs Open-Sources Its Homegrown LLM 'Trie-21B'

Trillionlabs has achieved a remarkable feat, dramatically reducing the development cost of Large Language Models (LLMs) to one-twelfth while still delivering global top-tier performance, all through its unique 'from scratch' pre-training method. This innovation was first introduced to the world through the open-source Large Language Model 'Tri-21B'. Tri-21B is a next-generation LLM that goes beyond simple text generation, simultaneously possessing high-dimensional language understanding and complex problem-solving capabilities. It is equipped with 21 billion parameters (a 3x expansion compared to its predecessor), significantly boosting performance, while also achieving astonishing lightness, capable of running smoothly with just a single GPU. The core of this model lies in Trillionlabs' independently developed 'Cross-lingual Document Attention (XLDA)' system. XLDA is an original data learning methodology that efficiently transfers vast English-based knowledge to low-resource languages such as Korean and Japanese. Thanks to this technology, learning costs could be reduced to one-twelfth compared to existing methods. This dramatically increases LLM utilization even in data-scarce industrial fields and has laid the foundation for more natural and accurate sentence generation in Northeast Asian language regions, including Korean. In high-difficulty, inference-centric benchmarks such as General Knowledge (MMLU), Mathematics (MATH), and Coding (MBPP Plus), Tri-21B demonstrated performance that stands shoulder-to-shoulder with global leading medium-sized models like Alibaba's Qwen 3, Meta's LLaMA 3, and Google's Gemma 3. Notably, it showcased outstanding capabilities, recording an accuracy of 77.93 points in inference capability verification (MMLU) (85 points with CoT application), 77.89 points in Mathematics, and 75.4 points in the coding domain. In major Korean benchmarks, it demonstrated an even more unparalleled presence. It achieved 86.62 points in 'Hae-Rae', which measures Korean cultural understanding, and 62 points in 'KMMLU' for Korean knowledge and inference ability (70 points with CoT application), yielding results that outperform global models. Shin Jae-min, CEO of Trillionlabs, explained, "Tri-21B, through its flywheel structure, effectively transfers the capabilities of a 70B-class large model to a 21B model, achieving a balance of model size, cost, and performance, making it the most ideal existing structure." He also stated his ambition, saying, "Through high-performance LLMs developed with complete pre-training, we will quickly achieve cost-efficiency and performance improvement, thereby enhancing the completeness of Korean AI technology, and build a full-size LLM portfolio together with the upcoming 'Tri-70B'." Trillionlabs, established in August 2024, is a startup that has independently designed Korean-centric LLMs and conducted pre-training from scratch, achieving innovative results in just a few months and illuminating the future of Korean AI technology.