AI Infrastructure

Best Local LLM for Translation (2026): Open Translation Models You Can Run Offline

The open-weight translation models and general LLMs worth running on your own hardware in November 2026, checked against their model cards and the WMT26 findings: Hy-MT2, Index-Translate, TranslateGemma, PLaMo 3 Translate, Lumen, Qingqiu-MT, Gemma 4, NLLB-200 and MADLAD-400, with language coverage, licences, memory, a document workflow and when confidentiality rules bring translation in-house.

The best local LLM for translation in November 2026 depends on languages and licence. For commercial work, Tencent's Hy-MT2 (33 languages) and Index-Translate (150 languages) are Apache 2.0 and start at about 1 GB. Gemma 4 31B is the strongest desktop-sized general model for translation, and in WMT26's human evaluation the open 14.8B Lumen ranked far ahead of Google Translate. NLLB-200 remains non-commercial.

Quick picks: the best local LLM for translation, by job

We read every model card and licence below on 14 November 2026, took file sizes from the repositories listed in the sources, and used the WMT26 General Machine Translation findings, published for the conference on 28 and 29 October 2026, as the main independent evidence. Vendor benchmark figures are labelled as such.

JobPickWhy
Commercial use, laptop or CPUHy-MT2-1.8BApache 2.0, 33 languages, 1.13 GB at Q4_K_M
Commercial use, 8 GB GPUIndex-Translate-9B or Hy-MT2-7BApache 2.0; 5.78 GB and 4.62 GB at Q4_K_M; glossary and format instructions
Many languages, permissive licenceIndex-Translate (150) or MADLAD-400 (400+)Both Apache 2.0
One command in OllamaTranslateGemma 4B, 12B or 27B55 languages; 3.3, 8.1 and 17 GB; Gemma terms of use
Japanese in either directionPLaMo 3 TranslateGlossary, context and politeness controls; licence needs registration for business use
One model for chat and translation, 24 GBGemma 4 31BApache 2.0; best desktop-sized general model in WMT26’s comparison
Best open model across WMT26’s pairsLumen (14.8B)Apache 2.0; average human rank 6.8 in WMT26; 29.5 GB unquantised
CPU-only, one language pairOPUS-MTSmall per-pair models from Helsinki-NLP

Dedicated translation models or a general LLM?

Two kinds of model translate well on local hardware. Dedicated translation models are trained or fine-tuned on parallel text and do little else. General LLMs, such as Gemma 4 or Qwen3.5, translate as one skill among many.

The evidence we could verify points in three directions.

At the same size, tuning for translation wins. The Index-Translate technical report compares models on common test sets. The general Qwen3.5-9B scores 0.8316 on FLORES COMET-22. Index-Translate-9B, built on that same model, scores 0.8789, Hy-MT2-7B scores 0.8747 and TranslateGemma-12B scores 0.8732. This is the vendor’s own evaluation, but the base-versus-tuned gap is a like-for-like comparison.

Instruction-following varies more than raw quality. On the same report’s test of terminology, format and style instructions, Index-Translate-9B scores 0.8209, Hy-MT2-7B 0.6079 and TranslateGemma-12B 0.3068. If you need a glossary enforced or JSON keys preserved, that column matters more than COMET.

A newer, larger general model can beat an older specialist. Before producing reference translations, WMT26’s organisers compared open-weight candidates on English-source data in 12 target languages, scored by GPT-5.4 without references. Gemma 4 31B averaged 80.3 and TranslateGemma-27B, which is built on Gemma 3, averaged 67.1. Gemma 4 was weak in Estonian (52.1) and Icelandic (43.1), which is why per-language testing matters.

WMT26’s test suites add a caution: the findings paper reports a marked tendency towards hallucination across current systems, and one suite, on Korean-to-English, found general-purpose LLMs far more prone to hallucination and to drifting from the source than dedicated systems, especially with sensitive content, truncated input and text that contains instructions.

The best open source translation models compared, November 2026

Sizes are the makers’ parameter counts. File sizes are GGUF Q4_K_M builds unless stated, in decimal gigabytes.

Model (maker)SizesLanguagesWeights licence4-bit fileReleased
Hy-MT2 (Tencent)1.8B, 7B, 30B-A3B33Apache 2.01.13, 4.62, 18.24 GBMay 2026
Index-Translate (IndexTeam)2B, 9B, 35B-A3B preview150Apache 2.01.31, 5.78 GBSeptember 2026
TranslateGemma (Google)4B, 12B, 27B55Gemma terms of use, gated download3.3, 8.1, 17 GB in OllamaJanuary 2026
PLaMo 3 Translate (Preferred Networks)0.8B, 2B, 8B, 31B53PLaMo Community License0.57, 1.81, 5.28, 19.53 GBOctober 2026
Qingqiu-MT-9B (Kingsoft Office)9B20Apache 2.05.63 GB (community build)August 2026
Lumen (WMT26 submission)14.8B23 WMT26 directionsApache 2.029.5 GB BF16 onlyAugust 2026
EuroLLM-22B-Instruct22B35Apache 2.0Community buildsDecember 2025
Tower+ (Unbabel)2B, 9B, 72B22CC BY-NCCommunity buildsJune 2025
MADLAD-400 (Google)3B, 7B, 10B400+Apache 2.0Not an LLM; T5November 2023
NLLB-200 (Meta)600M, 1.3B, 3.3B200CC BY-NC 4.0Not an LLM; sentence-levelJuly 2022
SeamlessM4T v2 (Meta)2.3B96 text, 101 speech inCC BY-NC 4.0Speech and textNovember 2023

Notes on the newer entries:

  • Hy-MT2 succeeds Hunyuan-MT and HY-MT1.5. Tencent’s card gives prompt templates for terminology, style, delimiters and structured data, and an extreme 1.25-bit build of the 1.8B model that needs 440 MB. Tencent sponsors a WMT26 task, so treat its own comparisons as vendor claims.
  • Index-Translate is built on Qwen3.5 and also translates text in images through a vision projector file, without separate OCR.
  • PLaMo 3 Translate was trained with a combined input and output limit of 32,768 tokens and accepts a glossary, surrounding context and earlier translations as structured inputs. Its licence is free for non-commercial use; businesses with revenue up to 1 billion yen register for free commercial use, and larger ones need an agreement. Its makers publish a quantisation table: the 8B model loses 0.0019 on their Japanese–English test at Q4_K_M and 0.0130 at Q2_K.
  • TranslateGemma is the easiest to start (ollama run translategemma:12b) but is built on Gemma 3 and expects one fixed prompt format.
  • Cohere’s Tiny Aya Global (3.35B) and Command A Translate are open weights under CC BY-NC 4.0, so they are for evaluation rather than commercial deployment.

General LLMs that translate well

If you already run a general model, it may be good enough, and it saves a second deployment.

  • Gemma 4 (Apache 2.0) lists out-of-the-box support for more than 35 languages and pre-training on more than 140. The 31B model is the desktop-sized general model with the best WMT26 result, and the E2B, E4B and 12B models also accept audio for speech-to-translated-text.
  • Qwen models are the most common base for fine-tuned translation systems: six of the trained WMT26 submissions were built on them, including Lumen and Qingqiu-MT-9B. Untuned, the 9B and 27B checkpoints ranked in the lower half.
  • gpt-oss-120b ranked below Gemma 4 31B despite being about four times the size, so size alone is a poor guide.

Give a general model a strict instruction: name both languages, ask for the translation only, and state the register. Without that, chat models tend to add notes and alternatives that break a pipeline.

Licences: which translation models allow commercial use

  • Permissive (Apache 2.0): Hy-MT2, Index-Translate, Qingqiu-MT-9B, Lumen, EuroLLM, MADLAD-400 and the general Gemma 4 models.
  • Conditional: TranslateGemma is distributed under Google’s Gemma terms and its Hugging Face download is gated. PLaMo 3 Translate needs registration for business use and a separate agreement above 1 billion yen in revenue.
  • Non-commercial: NLLB-200, SeamlessM4T v2, Tower+, Tiny Aya Global and Command A Translate are CC BY-NC. Meta’s card also says NLLB-200 is a research model not released for production deployment.

Publishing has its own rule. Amazon KDP’s content guidelines require authors to declare AI-generated translations when they publish a book, in the same way as AI-generated text and images.

What WMT26 found: human evaluation of open and closed systems

WMT is the annual shared task where translation systems are judged by professional annotators, not only by automatic metrics. The 2026 edition covered 23 language pairs and 46 systems, with whole documents as input. Its constrained track required open weights and fewer than 20 billion parameters.

The findings paper reports each system’s average rank across the language pairs it entered. Lower is better.

SystemTypeAverage human rank
Gemini 3.1 ProClosed API4.8
LumenOpen, 14.8B, Apache 2.06.8
Qingqiu-MT-9BOpen, 9B, Apache 2.016.5
DeepSeek V4 ProOpen weights, data-centre size17.5
Gemma 4 31BOpen general LLM, Apache 2.022.0
Google TranslateClosed service26.9
gpt-oss-120bOpen general LLM32.1
Qwen 3.6 27BOpen general LLM35.1
Tiny Aya Global 3BOpen, CC BY-NC35.5
Gemma 4 4BOpen general LLM35.6
Tower+ 9BOpen, CC BY-NC41.2
Qwen 3.5 9BOpen general LLM41.5

Three conclusions for local use. A fine-tuned open model below 20 billion parameters can sit next to the best closed systems: Lumen reached the top cluster in 13 of 23 language pairs overall. Gemma 4 31B, with no translation tuning, ranked ahead of Google Translate on average. And small general models are a poor choice for translation: Qwen 3.5 9B ranked last of the systems in this table, while Qingqiu-MT-9B, a fine-tune of that same model, is third in it.

The paper also notes that low-resource languages remain hard, with most systems scoring below 60 on them, and that the organisers no longer publish automatic scores because participants tune to the metrics.

Language coverage: check your pairs, not the headline number

The number on the model card is the count of languages a model was trained to handle, not a promise of equal quality.

  • 33 languages (Hy-MT2): major European and Asian languages plus Tibetan, Kazakh, Mongolian, Uyghur and Cantonese.
  • 35 (EuroLLM-22B): every official EU language plus 11 others, with a 32K window.
  • 53 to 55 (PLaMo 3 Translate, TranslateGemma): broad coverage of widely used languages.
  • 150 (Index-Translate): the widest of the current LLM-based models.
  • 200 and 400+ (NLLB-200, MADLAD-400): older encoder-decoder models that still reach languages nothing else does.

Outside a model’s list, output degrades sharply. In the Index-Translate report’s low-resource test, 1,040 directions among 62 languages, Hy-MT2-7B answered in the wrong language 35.7% of the time, against 1.7% for TranslateGemma-12B and 4.0% for Index-Translate-9B. Add a language-identification check to any pipeline that handles rarer languages.

Hardware tiers for an offline translation model

MemoryWhat fitsNotes
CPU or 4 GBHy-MT2-1.8B (1.13 GB), Index-Translate-2B (1.31 GB), PLaMo 3 Translate 0.8B (0.57 GB), OPUS-MTFast enough for interactive use on a laptop
8 GB GPUHy-MT2-7B (4.62 GB), Index-Translate-9B (5.78 GB), Qingqiu-MT-9B (5.63 GB), PLaMo 3 Translate 8B (5.28 GB), TranslateGemma 4B (3.3 GB)The practical tier for a translation server
12–16 GBTranslateGemma 12B (8.1 GB), Gemma 4 12B (7.12 GB)Room for long documents in context
24 GBHy-MT2-30B-A3B (18.24 GB), TranslateGemma 27B (17 GB), Gemma 4 31B (18.32 GB), PLaMo 3 Translate 31B (19.53 GB)Highest quality on one consumer card
32 GB and upLumen at BF16 (29.5 GB)Published unquantised; 4,096-token context

Translation output is about as long as its input, so plan context for both. For a general model shared with other work, our guide to local models by VRAM and the VRAM calculator cover the sizing, and the Gemma 4 deployment guide lists that family’s builds.

How to run a local LLM translator

Ollama, for a quick start. TranslateGemma is in the Ollama library and expects a single user message in a fixed form, with two blank lines before the text:

ollama run translategemma:12b
You are a professional German (de) to English (en) translator. Your goal is to accurately convey the meaning and nuances of the original German text while adhering to English grammar, vocabulary, and cultural sensitivities. Produce only the English translation, without any additional explanations or commentary. Please translate the following German text into English:


Guten Morgen, wie geht es Ihnen?

llama.cpp, for Hy-MT2 and Index-Translate. Tencent recommends temperature 0.7, top-p 0.6, top-k 20 and a repetition penalty of 1.05 for the 1.8B and 7B models, with no system prompt. Its GGUF card says the files depend on a kernel from a named llama.cpp pull request, so build llama.cpp as the card describes.

llama-server -hf tencent/Hy-MT2-7B-GGUF:Q4_K_M \
  --temp 0.7 --top-p 0.6 --top-k 20 --repeat-penalty 1.05 --port 8080

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Translate the following text into English. Note that you should only output the translated result without any additional explanation:\n\nDer Vertrag tritt am 1. Januar in Kraft."}]}'

vLLM, for volume. Hy-MT2 and PLaMo 3 Translate publish FP8 and other GPU-ready builds with vLLM instructions, and Unbabel recommends vLLM for Tower+. Batch throughput is where a dedicated server pays off; our inference server comparison explains the trade-offs.

Transformers, for the older models. NLLB-200 and MADLAD-400 are encoder-decoder models that load through Hugging Face Transformers, not through chat front ends.

Document translation workflow: chunking, glossaries and formatting

Translating a contract or a manual is a pipeline, not a prompt.

  1. Extract text with structure. Pull text from DOCX, HTML or PDF with its headings, tables and lists marked. Scanned pages need OCR first; our local OCR model guide covers that step.
  2. Chunk by section, inside the model’s limit. Limits differ widely: NLLB-200 was trained on inputs of at most 512 tokens and is sentence-level, Lumen’s context is 4,096 tokens, and PLaMo 3 Translate handles 32,768 tokens of input and output combined. Split at paragraph or section boundaries, never mid-sentence.
  3. Carry context forward. Pass the previous segment and its translation, or a short summary, so pronouns, formality and terms stay consistent. PLaMo 3 Translate accepts earlier translation pairs as history, and Hy-MT2 has a template for background information.
  4. Enforce terminology. Give the model a glossary of approved term pairs for each chunk, containing only the terms that occur in it. Hy-MT2, PLaMo 3 Translate and Index-Translate all document a glossary input. WMT26’s terminology task summed up its findings in its title: the hard part is finding the terms, not using them. Then check in code that each required target term appears.
  5. Protect formatting. Translate text nodes, not markup. Hy-MT2 documents prompts that keep delimiters, JSON keys and placeholders such as {{var}} or %s unchanged, and WMT26 added a software domain of JSON strings for exactly this skill. Validate that the output still parses.
  6. Check automatically, then by a person. Compare numbers, dates and names between source and target, detect untranslated or wrong-language segments, and flag length outliers. Legal, medical and published text still needs a qualified reviewer.

Keep the source, the glossary version, the model version and the reviewer’s changes together, so any sentence can be traced.

When confidentiality rules push translation in-house

The documents that most need translating are often the ones that should not leave the building: contracts under a confidentiality clause, due-diligence files, patient records, unfiled patent drafts, internal investigations and privileged legal advice.

A free web translator is a third-party processor with its own terms. DeepL’s privacy policy states that content entered into its free service is processed for a limited period to train and improve its neural networks; its paid tiers are governed by different terms. Enterprise translation APIs can be contracted properly, but each one adds a processor to assess, an agreement to sign and, for EU personal data, a transfer question. Our GDPR checklist for AI tools lists those checks.

A model on your own server removes the question. Source text, translations and logs stay in the network, and the model runs without an internet route, as in the air-gapped deployments guide. That does not remove the need for review: an in-house model can mistranslate a clause as confidently as a cloud one.

Translation is also only half of working across languages. Searching a document base in several languages at once is a retrieval problem, covered in our article on multilingual retrieval for European enterprises.

How VDF AI fits

VDF AI does not run, host, resell or integrate with the translation models in this guide, and this article does not describe which model a VDF deployment uses. What it offers organisations is the layer around a model: who may translate what, with which approved model, and what record is kept.

The platform is deployed on your own servers, inside an air-gapped network or in VDF’s cloud. Agents can be given a translation tool that takes a source and target language and an optional glossary of term mappings, and translation can run on a model inside your perimeter. VDF AI Router registers on-premises model endpoints beside any cloud models your policy allows, and its air-gapped mode keeps every request local.

Sources

All of the following were opened on 14 November 2026; benchmark figures quoted from model cards are the makers’ own.


Translating documents that cannot leave your network? See the translation tool or book a demo.

Frequently asked questions

What is the best local LLM for translation?

It depends on your languages and licence needs. For commercial use on modest hardware, Tencent's Hy-MT2 (33 languages) and Index-Translate (150 languages) are Apache 2.0 translation models with builds from about 1 GB to 18 GB. Among general models that fit one graphics card, Gemma 4 31B is the strongest we could verify: WMT26's organisers scored it above every dedicated open translation model they compared before picking it for reference drafts. The strongest open system across WMT26's human evaluation was Lumen, a 14.8B Apache 2.0 model that led the constrained track with an average rank of 6.8, well ahead of Google Translate.

Is a dedicated translation model better than a general LLM?

At the same size, usually yes. In the Index-Translate technical report, the general Qwen3.5-9B scores 0.8316 on FLORES COMET-22, while three translation-tuned models of 7 to 12 billion parameters score between 0.8732 and 0.8789. Dedicated models also follow glossary and formatting instructions more reliably, and one WMT26 test suite found general LLMs far more prone to hallucination than dedicated systems. A large general model can still win: Gemma 4 31B outscored TranslateGemma-27B in WMT26's preliminary comparison.

Can I use NLLB-200 commercially?

No. Meta's NLLB-200 checkpoints are licensed CC BY-NC 4.0, and the model card describes NLLB-200 as a research model that is not released for production deployment. The card adds that it translates single sentences, was trained on inputs of at most 512 tokens, and is not intended for document translation or for medical and legal text. If you need wide language coverage with commercial rights, look at MADLAD-400 (Apache 2.0, more than 400 languages) or Index-Translate (Apache 2.0, 150 languages) instead.

Which offline translation model covers the most languages?

MADLAD-400 from Google covers more than 400 languages under Apache 2.0, in 3B, 7B and 10B sizes on a T5 architecture from 2023. NLLB-200 covers 200 languages but is non-commercial. Among current LLM-based models, Index-Translate covers 150 languages, TranslateGemma 55, PLaMo 3 Translate 53, EuroLLM 35 and Hy-MT2 33. Coverage and quality are different things: a model asked for a language it was not trained on often answers in the wrong language, so test your own pairs before committing.

How much VRAM does a local translation model need?

Much less than a chat model of similar usefulness. Hy-MT2-1.8B is a 1.13 GB file at Q4_K_M and Index-Translate-2B is 1.31 GB, so both run on a laptop CPU. The 7 to 9 billion parameter models, Hy-MT2-7B, Index-Translate-9B and PLaMo 3 Translate 8B, are 4.6 to 5.8 GB and fit an 8 GB graphics card. TranslateGemma 12B is 8.1 GB in Ollama. The 27 to 31 billion parameter models are 17 to 20 GB at 4-bit and need a 24 GB card.

Is it safe to put confidential documents into an online translator?

Read the service's terms first. DeepL's privacy policy, for example, says that content entered into its free translator is processed for a limited period to train and improve its neural networks, and asks users of the free version to enter only content they are willing to transfer; its paid plans have different terms. Contracts, patient records, unfiled patents and privileged legal documents usually carry duties that such terms cannot satisfy. A model running on your own hardware keeps the source text, the translation and the logs inside your network.

Filed under
machine translationlocal LLMopen-weight modelsmultilingual AImodel licencesdata confidentiality
On-Prem AI

Plan your on-prem AI deployment

Book an architecture call and we will scope a private, on-prem AI deployment for your environment — integrations, hardware, and governance included.

Keep reading