Kolibri-1 thinks in German. Does that help it answer, too?
Aleph Alpha’s Kolibri-1 is a new AI model optimized for German texts that also reasons in German. iX put it to the test.
(Image: Bangla press / Shutterstock.com)
- Dr. Christian Winkler
We’ve waited a long time for a model that can also handle German texts well. In the meantime, the announcement of the Soofi model raised hopes. But those hopes faded quickly because the model is still not freely accessible, and the consortium apparently also got creative by using benchmark data in its training. That makes Aleph Alpha’s newly released Kolibri-1 model all the more intriguing.
Size and architecture
The model’s size immediately stands out: At 78 billion parameters, it’s relatively unconventional. Standard sizes have become established for many models, but this one doesn’t fit the mold. With 384 experts, the model has a relatively high number for its parameter count. Since six experts are routed and one is fixed, that leaves only 3.46 billion active parameters during inference. Most of the weights in the regular release are quantized in FP8; some model components remain in bfloat16. Aleph Alpha also offers a BF16 version.
The architecture can certainly be described as traditional. Kolibri-1 uses a hybrid attention architecture, alternating four sliding-window attention (SWA) layers with one full-attention layer. For the latter, however, the model uses grouped-query attention, which providers such as DeepSeek and Qwen have phased out over the past few generations. This may also explain the somewhat smaller context window of “only” 256K tokens, which can be extrapolated to around one million tokens during inference.
(Image: Aleph Alpha)
Kolibri-1 also supports reasoning, so it can explain its answers through a kind of “thought process”. Its length can be controlled at three levels, which helps prevent the model from thinking unnecessarily long about simple questions. So far, the model holds few surprises.
Training data
The surprises come with the training data, which is prepared differently from that of competitors. This starts “right at the beginning” of the model, with its tokenizer.
The tokenizer is responsible for splitting text into words or their components, called tokens. Many multilingual models, especially those from Chinese providers, have to make compromises here so they can efficiently represent as many words (or parts of words) from different languages as possible. Because Aleph Alpha focuses on German and English, it doesn’t face the same problem. As a result, Kolibri can represent English words, and especially German words, with fewer tokens. That matters not only when you pay by the token, but also for internal processing. Saving 10% of tokens can make processing faster by nearly the same amount.
Kolibri-1 is also particularly well suited to legal and political content. Just for fun, you can try seeing how many tokens it takes to process the entire German Bundestag glossary. Kolibri-1 needs 1,137 tokens. The highly innovative DeepSeek-V4.1-Flash, included for comparison, needs 1,910 — quite a difference! That means you can save a lot of tokens and work with shorter contexts. Processing can also be faster because many words no longer have to be assembled from their components. This also improves prediction accuracy.
Using this tokenizer, Aleph Alpha trained the model on 20 trillion tokens during pretraining, using content through June 18, 2026. That’s very recent, since base training typically takes many weeks or months. Like many other providers, Aleph Alpha uses Common Crawl data, but it describes a deduplication process in considerable detail. In addition to static tests, the developers use other LLMs, such as Gemma-4-26B-A4B. However, the data described in the technical report isn’t enough to reach the stated 21.3%. Instead, the developers used another model (Mistral-NeMo-12B) to have English content rephrased in German. They also used translated data, some of which they curated manually to keep the translations from being too literal.
Aleph Alpha devotes considerable effort to quality assurance, using other models as “judges”. This, along with correlations calculated from the results, helps identify particularly “good” documents, which are then included in a “data mix”. Different sources are assigned different weights based on the quality of their documents.
This in-house data pipeline makes it easier for the provider to integrate newer data into future model versions through the training process.
(Image: Aleph Alpha)
In the subsequent so-called midtraining stage, Aleph Alpha uses 3.4 trillion tokens of “higher-quality” content, including logical reasoning, mathematical documents and code. During midtraining, the context window grows from 16K to 64K, eventually reaching 256K in the final pretraining phase. This saves significant computing capacity in the resource-intensive early training stages. Kolibri-1 also uses data from the pipeline during midtraining, mixing it according to quality. The developers use a variant of DeepSeek’s Auxiliary Free Load Balancing, which they call Exact Quantile Balancing, to train all experts “evenly”. Aleph Alpha also uses so-called Load-Error Injection to ensure that all experts are included in microbatches. Different advances in these “balancing” methods have now caught on at nearly all providers, especially those that use many experts.
During post-training, the model has to learn to answer questions correctly. For this, too, the developers use German and English data from different areas, such as tool use, instruction following and code. Some of this data was also generated by other models: GLM-5.2, GLM-5.3 and Qwen3.8-27B. The latter is surprising, since the model has only been available since late August. Aleph Alpha teaches the model to control its reasoning depth and express its reasoning in German. In the latter respect, it differs from nearly all other available models, which always think in English. Another distinctive feature is the Merlin-Arthur protocol, which is intended to train the model out of hallucinating. It works like a game in which relevant and irrelevant parts of an answer are hidden. The model then has to decide which answer fits better without that information, and receives a reward for choosing correctly. The method appears to work: benchmarks show that the model hallucinates noticeably less often.
For all training data, Aleph Alpha takes care to comply with the EU AI Act and the GDPR. The provider also aligns the model with the “democratic consensus” in Europe. This wording could be understood to mean that there are also answers that Chinese models are not allowed to give for political reasons.
The training process is, of course, extremely resource-intensive. It uses the Muon optimizer to adjust the model weights, while the ubiquitous AdamW optimizer continues to handle the scaling factors and embedding weights.
The infrastructure required is enormous: pretraining took place on 768 Nvidia B200 GPUs across 96 nodes, consuming around 392,000 GPU-hours according to the model card (about 45 GPU-years). For reinforcement learning during post-training, Aleph Alpha used a cluster of 256 Nvidia B300 GPUs. Particularly interesting is that 144 GPUs were dedicated solely to inference. That’s presumably because the GRPO method requires different answers, which it then evaluates against one another. Completing all of this from the mid-June knowledge cutoff through the end of September is a remarkable achievement. There was certainly some groundwork: Aleph Alpha had previously trained the Kolibri Origin model.
To make training reproducible, Aleph Alpha uses what it calls a ModelFactory. You can think of it as a kind of recipe, in code, that describes the model’s training precisely. These recipes are version-controlled, making it possible to reproduce the entire training process. This requires the necessary infrastructure and a willingness to invest a corresponding amount of computing time. Energy costs also shouldn’t be overlooked: Aleph Alpha estimates 950 MWh, which at €0.30 per kWh — a price that is certainly lower for customers of this size — comes to €285,000.
Trying it out
Since the model is available with open weights, it’s easy to try out. vLLM supports the model with a plugin, and llama.cpp already supports it with a small patch.
As expected, the model is very fast. That’s because it has just under 4 billion active parameters. It’s therefore almost in the same speed class as Qwen3.5-35B-A3B. The model’s answers are all correct. Its (trained) German reasoning process is particularly striking. Its answer about the car wash is a little roundabout, but with many other models, there’s reason to suspect that the question was already included in the training data if the answer is very short and even phrased ironically with emojis.
Videos by heise
All in all, the model works very well. Whether it can really avoid hallucinations better than other models remains to be seen in real-world use. The model also handled questions like “Did the Hanover-based Heise publishing house, which writes a lot about computers, also organize the CeBIT computer expo?” without getting confused, correctly answering that Deutsche Messe AG organized CeBIT through 2018. Asking the reverse — “Which trade shows does the Heise publishing house in Hanover organize?” — gets the response that the model doesn’t know and that you should check the website. Not bad!
The bottom line
Kolibri-1 is a truly solid model that speaks German very well and also reasons in German. That makes it a natural fit for its intended use cases in companies and government agencies. It’s technically well equipped for the job, since it can run independently and under the user’s control on their own infrastructure.
The technical report makes for fascinating reading, and it doesn’t skimp on details that others often leave out. It explains not only the tokenizer’s design, but also how the training set was assembled, which other models played a role and how everything came together. You can clearly see how much engineering went into it. That somewhat makes up for the “dated” architecture.
The ModelFactory is also intriguing: It builds models from simple, readable code components. It’s a model of transparency, a hallmark of a model that complies with the GDPR and the EU AI Act.
(dahe)
