Why Are LLMs Weaker in Hungarian
And What It Means for Your AI Project
- AI In Business
On a Hungarian factual question set, GPT-4o answers roughly half the questions correctly. The same class of model is far ahead of that in English.
The gap is structural, it has four separate causes, and it is worth understanding before you decide what an AI system in your company is allowed to do in Hungarian.
The same model, two levels of competence
In 2025 a joint team from the HUN-REN Hungarian Research Centre for Linguistics and Shanghai AI Laboratory published OpenHuEval, the first benchmark built specifically around the Hungarian language and Hungarian context. It contains 3,953 questions across eight dimensions, from proverbs and history to business and finance. The results are worth reading closely.
On HuSimpleQA, a set of short factual questions about Hungary, GPT-4o answered 50.3% correctly. DeepSeek-V3 reached 32.7%. Qwen2.5-72B managed 14.9%. On a Hungarian fill-in-the-blank task, the strongest model completed an entire question correctly 17.2% of the time. These are the same models that dominate English leaderboards.
There is a second finding that matters even more for anyone selecting a model. When the researchers compared model rankings on the English benchmarks with the rankings on their Hungarian counterparts, 70% of the models changed position. The model that wins in English is frequently a different model from the one that wins in Hungarian. A vendor comparison run in English tells you very little about what will happen in Hungarian.
Four things drive the gap, and each has a different consequence for your project.
1. Hungarian is a rounding error in the training data
Start with the raw material. In the most recent Common Crawl archive, the web corpus that sits underneath most large models, English accounts for 40.8% of pages. Hungarian accounts for 0.53%. That is roughly one Hungarian page for every 77 English ones.
Volume alone would be manageable. The harder problem is what kind of Hungarian gets crawled. A large share of Hungarian web text is machine translated from English, scraped, duplicated, or simply thin. The Hungarian a model learns is therefore partly translated Hungarian, which is exactly the flat, faintly foreign register that a Hungarian executive notices in the first paragraph and stops trusting.
This is why generic AI output in Hungarian tends to feel almost right. The grammar holds. The idiom gives it away.
2. The tokenizer was built for a different kind of language
Models read text in tokens, and tokenizers are trained on the same English-dominated distribution. English averages roughly 1.2 to 1.4 tokens per word. Agglutinative languages, where a single word carries what English spreads across a preposition, an article and an auxiliary, run to twice that and beyond.
Hungarian is a textbook case. A form such as „elszámoltathatatlanságukról” is one word carrying a stem, a causative, a potential, a negation, a possessive and a case ending. A tokenizer that never saw enough Hungarian shreds it into fragments that carry no meaning on their own. The model then has to reassemble the sense from pieces that were never designed to hold it.
Three costs follow directly:
- Cost. The same document consumes markedly more tokens in Hungarian. On a document-heavy workload, that lands straight on the invoice.
- Context. Hungarian text fills a fixed context window faster. In a retrieval system, that leaves you with thinner evidence per query.
- Accuracy. Fragmentation and accuracy move together. In one large multilingual study, each additional token per word was associated with an 8 to 18 percentage point drop in task accuracy, depending on subject and model.
Understanding language is only one part of the challenge. We explored how natural language processing works in more detail in What is NLP? →
3. Under the surface, the model reasons in English
The OpenHuEval team did something unusual. They opened up the visible reasoning traces of two reasoning models and classified what happened inside them on Hungarian tasks.
They found a pattern they named explicit translation insertion. Faced with a Hungarian question, the model first translates the key phrase into English, reasons in English, then answers in Hungarian. It appeared in 5.5% of DeepSeek-R1’s reasoning segments and 16.8% of QwQ’s. The weaker the model’s Hungarian, the more it leans on the English detour.
The second finding is sharper. On hard English maths problems, when these models produce a wrong final answer, 72% to 78% of those failed attempts still contain at least one correct intermediate thought. The model knew, then lost it. On the Hungarian factual task, that figure falls to 46% and 42%. In most Hungarian failures the correct thought is missing from the trace entirely. This is a knowledge gap, and it stays there however hard you push the model to reason.
For a business system, the distinction is decisive. Reasoning failures degrade gracefully and can often be caught by asking the model to check itself. Missing knowledge produces a confident, fluent, wrong answer about a Hungarian regulation, a Hungarian institution or a Hungarian customer. Self-checking leaves it standing, because the model has nothing to check against.
4. Almost nobody measures Hungarian properly
The Hungarian evaluation sets that do exist are mostly machine translations of English ones. HuLU, the established Hungarian understanding benchmark, covers seven tasks, four of them translated from English, and it is restricted to multiple-choice and true or false formats. That measures comprehension. It stays silent on whether a model can write a usable Hungarian customer letter, follow a Hungarian instruction, or recognise when it does not know something.
The practical consequence: when a vendor tells you their model supports Hungarian, ask what that claim was measured on. In most cases the honest answer is that Hungarian appeared in a translated benchmark and was never tested on anything a Hungarian company would recognise as work.
What this changes in an AI project
AI in Hungarian works, provided you design around a known constraint. In practice that means five things.
- Evaluate in Hungarian, on your own tasks. Fifty real cases from your own operation, scored by someone in your company who knows what a good answer looks like. This is a two-day exercise and it beats every public leaderboard, because 70% of models change rank between English and Hungarian anyway.
- Give the model the knowledge it lacks. Hungarian-specific facts, your regulations, your product names, your terminology, all belong in a retrieval layer that feeds the model at query time. The gap sits in what the model knows about Hungary, so supply it and the gap closes.
- Keep a terminology glossary under version control. Hungarian professional vocabulary drifts fast in AI output. A short, controlled list of your own terms, injected into every prompt, holds the register steady.
- Budget the token tax. Model Hungarian volume well above the English token count when you build the cost case for a document-heavy workload.
- Put a human in the loop wherever the answer is Hungary-specific. Legal, regulatory, tax, public administration. This is precisely where the confident wrong answer appears.
The gap is a design constraint you can plan for
The performance gap in Hungarian comes from data volume, tokenization, missing knowledge and thin measurement. All four are engineering problems with engineering answers. The companies that get burned are the ones that select a model on an English benchmark, deploy it into a Hungarian process, and discover the difference in production.
The ones that succeed measure first. It costs a fortnight, and it changes the whole shape of the project.
Technology alone rarely determines the success of an AI project. We explored the organizational side of successful AI adoption in AI and Humans Together: How to Transform the Organization? →
Understanding the technology matters
The decisive question is a practical one: where and how can it create business value in your company?
That answer depends on how your company operates in practice, on where your data sits, and on which of your processes run in Hungarian. The AI Compass Audit is a four-week, fixed-price engagement that ends with a clear answer: which AI pilot is worth starting, with what success criteria, and at what risk. If you would first like to clarify your questions, you can begin with a free 30-minute consultation.
Sources
- Common Crawl. (2026). CC-MAIN-2026-25 language statistics. Common Crawl. English accounts for 40.8% of crawled pages, while Hungarian represents 0.53% of the corpus. Read article →
- Ligeti-Nagy, N., et al. (2024). HuLU: Hungarian Language Understanding Benchmark Kit. Proceedings of LREC-COLING 2024. Describes the benchmark composition and task formats for evaluating Hungarian language understanding. Read article →
- Lundin, M. (2025). The Token Tax: Systematic Bias in Multilingual Tokenization. Examines the relationship between tokenizer fertility and downstream task accuracy across multiple languages. Read article →
- Yang, X., et al. (2025). OpenHuEval: Evaluating Large Language Models on Hungarian Specifics. Evaluates large language models on Hungarian-specific benchmarks, including ranking shifts, explicit translation insertion and correct-thought ratios. Read article →
Csaba Fekszi
Csaba Fekszi is an IT expert with more than two decades of experience in data engineering, system architecture, and AI-driven process optimization. His work focuses on designing scalable solutions that deliver measurable business value.
Related posts
A practical guide for CFOs and senior decision-makers at mid-sized companies
The real barriers to enterprise AI, and what the companies that succeed do differently
Are you sure AI is the right next step?
We help uncover the real opportunities, limitations, and realistic next steps.