Qwen Code runs on a local LLM through its OpenAI-compatible provider: add the model to modelProviders in settings.json with the /v1 address of Ollama, LM Studio, llama.cpp or vLLM, select it with /model, and set contextWindowSize to what the server loaded. Gemini CLI has no local provider; it reaches local models only through a proxy that speaks the Gemini API.
Qwen Code is Alibaba’s open-source terminal coding agent, published under Apache 2.0 in the QwenLM/qwen-code repository. Its README says it was originally based on Google’s Gemini CLI 0.8.2 and stopped syncing with upstream from version 0.1. That history explains the search pattern: people who liked Gemini CLI’s interface but wanted a local model ended up here, because Qwen Code speaks the OpenAI, Anthropic and Gemini protocols and Gemini CLI speaks only Google’s.
The configuration keys and defaults below come from the Qwen Code and Gemini CLI documentation in their repositories, read on 14 November 2026 against Qwen Code 0.27.3, released the day before, and Gemini CLI 0.68.0 of 10 November. A Qwen Code 1.0 preview exists on a separate branch with its own docs; this guide follows the main version. Which model to run is covered in our review of open-weight coding models, and the same job for OpenAI’s agent is in our Codex guide for local models.
Qwen Code and Gemini CLI: what works with local models
| Qwen Code | Gemini CLI | |
|---|---|---|
| Native local model support | Yes, through the openai provider and a local baseUrl | No |
| Protocols | OpenAI Chat Completions and Responses, Anthropic, Gemini | Gemini API and Vertex AI |
| Local servers in the docs | Ollama, vLLM, LM Studio, plus SGLang for Qwen3.x | None; LiteRT-LM for a routing classifier only |
| Custom base URL | baseUrl on each model entry, or OPENAI_BASE_URL | GOOGLE_GEMINI_BASE_URL, for Gemini-format endpoints |
| Account needed for local use | No | A Gemini API key value, which a proxy can supply |
| Who it is for today | Anyone | Gemini Code Assist Standard and Enterprise, per its README |
The last row is new. Gemini CLI’s README now opens with a notice that free individual access and Google AI Pro and Ultra plans moved to Antigravity CLI on 18 June 2026, and that Gemini CLI is maintained for enterprise Code Assist workflows on Google Cloud and for paid API and Vertex keys.
How Qwen Code connects to a local model
Install it with the project’s script, with npm (Node.js 22 or newer) or with Homebrew:
npm install -g @qwen-code/qwen-code@latest
Qwen Code reads ~/.qwen/settings.json. Four parts of that file matter for a local model:
modelProvidersmaps a provider id to an array of model entries. For a local server the id isopenai. Each entry needs anid, and normally abaseUrl, anameand anenvKey.envKeyis the name of an environment variable, not a key. Local servers need no real key, so the docs use placeholders such asOLLAMA_API_KEY="ollama". The value can live in theenvblock of the same file, in~/.qwen/.envor in your shell.security.auth.selectedTypeset toopenaitells Qwen Code which protocol to start with, so it does not open the/authmenu.model.namesets the default;/modelswitches during a session.
Edits to modelProviders are picked up by a running session without a restart. The interactive route is /auth, then Custom Provider, which the docs describe as the option for a local server or proxy.
For a first test you can skip the file. The docs list three variables for the OpenAI-compatible protocol:
export OPENAI_API_KEY="ollama"
export OPENAI_BASE_URL="http://localhost:11434/v1"
export OPENAI_MODEL="qwen3-coder:30b"
qwen
The file is still worth writing, because it is the only place to pin the context window and timeouts per model.
Qwen Code with Ollama
-
Pull a model with tool support:
ollama pull qwen3-coder:30b -
Raise the context on the server. Ollama’s OpenAI-compatible API cannot set a context size per request, and its default is 4K tokens on machines with less than 24 GiB of VRAM. Ollama recommends at least 64,000 tokens for agents and coding tools:
OLLAMA_CONTEXT_LENGTH=65536 ollama serve -
Add the model to
~/.qwen/settings.json. This follows the local example in Qwen Code’s docs, with a current model:{ "env": { "OLLAMA_API_KEY": "ollama" }, "modelProviders": { "openai": [ { "id": "qwen3-coder:30b", "name": "Qwen3-Coder 30B (Ollama)", "envKey": "OLLAMA_API_KEY", "baseUrl": "http://localhost:11434/v1", "generationConfig": { "timeout": 300000, "streamIdleTimeoutMs": 600000, "contextWindowSize": 65536 } } ] }, "security": { "auth": { "selectedType": "openai" } }, "model": { "name": "qwen3-coder:30b" } } -
Start
qwenin a project folder and check the server while it works:ollama psThe
CONTEXTcolumn should show 65536, the same number ascontextWindowSize.
Ollama’s launch command covers several coding agents, but Qwen Code is not on its integrations list, so there is no one-line setup.
Qwen Code with LM Studio
-
Load the model with the context you want, then start the server:
lms load qwen/qwen3-coder-30b --context-length 65536 lms server start -
Add an entry to the
openaiarray, using the identifier LM Studio serves as theid:{ "id": "qwen/qwen3-coder-30b", "name": "Qwen3-Coder 30B (LM Studio)", "envKey": "LMSTUDIO_API_KEY", "baseUrl": "http://localhost:1234/v1", "generationConfig": { "timeout": 300000, "contextWindowSize": 65536 } } -
Add
"LMSTUDIO_API_KEY": "lm-studio"to theenvblock, as the docs’ example does, and select the model with/model.
If LM Studio loads a model on demand, it uses the context length saved for that model in LM Studio, which may be smaller than the value in your entry. Load the model explicitly, or change its saved default in LM Studio.
Qwen Code with llama.cpp and vLLM
llama.cpp
Current builds of llama-server listen on port 9931 unless told otherwise and enable the Jinja chat template, which OpenAI-style tool calling depends on. Pass the port, an alias and a context size; the rest of the flags are in our guide to running the llama.cpp server:
llama-server -hf ggml-org/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M \
--alias qwen3-coder-30b --host 127.0.0.1 --port 8080 -c 65536
The entry is the same shape as before, with "id": "qwen3-coder-30b", "baseUrl": "http://127.0.0.1:8080/v1" and "contextWindowSize": 65536. Set a placeholder variable for envKey, such as the docs’ VLLM_API_KEY="not-needed".
vLLM
For a GPU server shared by a team, the Qwen3-Coder-Next model card gives this vLLM command, to which we add a served name:
vllm serve Qwen/Qwen3-Coder-Next --port 8000 --tensor-parallel-size 2 \
--enable-auto-tool-choice --tool-call-parser qwen3_coder \
--served-model-name qwen3-coder-next
{
"id": "qwen3-coder-next",
"name": "Qwen3-Coder-Next (vLLM)",
"envKey": "VLLM_API_KEY",
"baseUrl": "http://localhost:8000/v1",
"generationConfig": {
"contextWindowSize": 262144
}
}
Qwen Code’s docs say to set contextWindowSize to the server’s maximum model length, which vLLM reports as max_model_len on its /v1/models route. The card notes that the default is 256K and suggests a smaller value such as 32,768 if the server fails to start. If the server uses a self-signed certificate behind https://, Qwen Code’s troubleshooting page covers the Node.js error that results. For the differences between these runtimes, see our vLLM, Ollama and llama.cpp comparison.
Context settings and tool-calling caveats
Why the context window must be pinned
Qwen Code infers the context window from the model id. A name it does not recognise gets a generic 200K window, and a recognised name gets whatever the built-in lookup holds for that model. Neither is likely to match a local server that loaded 32K or 64K. Auto-compaction triggers at a fraction of the window, 0.85 by default under context.autoCompactThreshold. If Qwen Code believes it has 200K tokens and the server holds 65,536, compaction never starts and the server runs out first. Setting contextWindowSize on the entry fixes this.
Timeouts
Local models are slow to start and sometimes slow between chunks. The docs give three controls:
timeoutingenerationConfig, the request timeout in milliseconds.streamIdleTimeoutMs, how long the model may stay silent between streamed chunks. It overrides the globalQWEN_STREAM_IDLE_TIMEOUT_MS, and0disables the guard.- A separate 15-minute cap on any one stream, raised or disabled with
QWEN_STREAM_MAX_LIFETIME_MS.
Tool calling
Qwen Code sends tool definitions on every request, so the server must parse the model’s tool calls into the OpenAI format.
- Ollama and LM Studio handle this for models that carry the tools capability.
qwen3-coderis tagged for tools in Ollama’s library. - llama.cpp needs the Jinja template, on by default. The server README adds that some models need a
--chat-template-fileoverride to get a template compatible with tool use. - vLLM needs
--enable-auto-tool-choiceand a--tool-call-parserthat matches the model. The Qwen3-Coder-Next card usesqwen3_coder; Qwen Code’s docs giveqwen3_xmlwith--reasoning-parser qwen3as an example for Qwen3.x models and warn that parser names change between server releases. - Renamed Qwen3.x models lose thinking control. If a vLLM or SGLang server exposes a Qwen3.x thinking model under a name that does not begin with
qwen, the docs say to declare"capabilities": { "reasoning": { "profile": "qwen-chat-template" } }on the entry. Otherwise side requests, such as the permission classifier used by Auto mode, keep thinking switched on.
The Qwen3-Coder models avoid the last problem: their cards say they support only non-thinking mode. General guidance on models that call tools reliably is in our tool-calling model guide.
Testing the setup and common errors
-
List the model ids the server exposes. Ollama, LM Studio,
llama-serverand vLLM all answer the OpenAI-style models route:curl http://localhost:11434/v1/models -
Run one prompt in headless mode with the model named explicitly:
qwen -m qwen3-coder:30b -p "List the top-level folders in this repository" -
Open an interactive session, run
/modeland confirm the entry is listed under the name you gave it.
| Symptom | Likely cause | Fix |
|---|---|---|
The model is missing from /model | Provider id is not openai or another supported id, so the whole entry was skipped with a warning | Use openai, or map a custom id with providerProtocol and restart |
| Two entries, one shown | Same protocol, id and baseUrl; the first wins | Change the id or the URL |
| Connection refused | Wrong port, or baseUrl without /v1 | Check the server; llama-server defaults to 9931 |
| The agent loses the task after a few steps | Server context smaller than contextWindowSize | Raise the server context or lower the setting |
| Tool calls appear as text | Server is not parsing tool calls | Enable the tool-call parser or the Jinja template |
| Streams stop after a silence | Idle guard on a slow model | Raise streamIdleTimeoutMs |
The id must be exactly what the server reports. Ollama uses tags such as qwen3-coder:30b, LM Studio uses paths such as qwen/qwen3-coder-30b, and llama-server reports the file path unless you pass --alias.
Which local Qwen coder models fit which hardware
| Model | Ollama tag and download | Parameters | Memory with context | Fits |
|---|---|---|---|---|
| Qwen3-Coder-30B-A3B-Instruct | qwen3-coder:30b, 19 GB | 30.5B, 3.3B active | ≈ 23 GB at 4-bit with one 64K session (our arithmetic) | 24 GB GPU with a shorter window; 32 GB GPU or Mac |
| Qwen3.8-27B | qwen3.8:27b, 18 GB | 27B, dense | ≈ 23 GB with 64K (Cline’s estimate) | 24 GB GPU at the limit; 32 GB comfortable |
| Qwen3.6 35B A3B | qwen3.6:35b-a3b, 23 GB | 35B mixture-of-experts build, per its name | ≈ 26 GB with 64K (Cline’s estimate) | 32 GB GPU or Mac |
| Qwen3-Coder-Next | qwen3-coder-next, 52 GB | 80B, 3B active | ≈ 56 GB with 64K (Cline’s estimate) | 64 GB unified memory or more; one 80 to 96 GB GPU |
Download sizes are from Ollama’s library. The estimates marked Cline come from the recommended-model table in Cline Desktop’s documentation, which our Cline set-up guide also tabulates, and the first row from the memory arithmetic in our coding-model review. Both Qwen3-Coder cards list a native context of 262,144 tokens under Apache 2.0, and both say to reduce the context to a value such as 32,768 if you run out of memory.
The two mixture-of-experts coder models activate about 3B parameters per token, which keeps them quick for their size. Qwen3.8-27B is a general model with thinking on by default; its builds and memory tiers are in our Qwen3.8 hardware guide. To test a combination before downloading, use the VRAM calculator.
One sampling note: the Qwen3-Coder-30B card suggests a temperature of 0.7 with top_p 0.8, and the Qwen3-Coder-Next card suggests 1.0 with top_p 0.95. Qwen Code passes such values through samplingParams in generationConfig if you want to set them.
Gemini CLI local LLM: what is and is not possible
Gemini CLI’s authentication page offers three methods: sign in with Google, a Gemini API key, or Vertex AI. There is no Ollama provider, no OpenAI-compatible provider and no setting that names a local model as the main model.
What the documentation does contain:
- A local routing classifier. The experimental
gemini gemma setupcommand downloads a Gemma 3 1B model of about 1 GB and serves it with LiteRT-LM. It classifies each request as simple or complex and sends it to Gemini Flash or Gemini Pro. The answer still comes from a hosted model. - A base URL override.
GOOGLE_GEMINI_BASE_URLoverrides the base URL for Gemini API requests when you use API-key authentication. It must use HTTPS unless it points atlocalhost,127.0.0.1or[::1].GOOGLE_VERTEX_BASE_URLdoes the same for Vertex AI.
The override is a redirect, not a protocol switch. Whatever answers at that address has to speak the Gemini API. Ollama, LM Studio, llama.cpp and vLLM do not, so pointing the variable straight at them fails.
Gemini CLI with Ollama through LiteLLM
LiteLLM’s proxy accepts Gemini-format requests and publishes a Gemini CLI tutorial. It sets two variables:
export GOOGLE_GEMINI_BASE_URL="http://localhost:4000"
export GEMINI_API_KEY=sk-<your-litellm-api-key>
To serve a non-Gemini model, the tutorial uses model_group_alias, which maps the Gemini model name the CLI requests to another entry. Its examples route to hosted providers. Combining it with LiteLLM’s documented Ollama provider gives a local route:
model_list:
- model_name: local-qwen
litellm_params:
model: ollama_chat/qwen3-coder:30b
api_base: http://localhost:11434
router_settings:
model_group_alias: {"gemini-3.1-pro-preview": "local-qwen"}
The alias key has to match the model name your Gemini CLI version sends; setting GEMINI_MODEL makes that predictable. Three cautions apply. Google does not document or support this arrangement, and we assembled the local variant from two LiteLLM pages without a statement from either project that the pair is tested. Gemini CLI’s README says its harness is not tuned even for newer Gemini models, so prompts written for Gemini will not suit a 30B local model well. And the proxy becomes one more component to secure; our LiteLLM alternatives review compares the gateways that can fill that role.
What about Antigravity CLI?
Antigravity CLI is where Google sent individual Gemini CLI users. Its model page lists Google-hosted and partner-hosted models and mentions no local option. Antigravity’s documentation does describe local models for its Python SDK, through LiteRT or an OpenAI-compatible server such as Ollama, but that page is about the SDK, not the agy command.
If the goal is a Gemini-CLI-style terminal agent on a local model, Qwen Code is the direct route.
Approvals, sandboxing and keeping data on the machine
A local model makes more mistakes with tools than a hosted frontier model, so the approval mode matters. Qwen Code has five: Plan, which is read-only; Ask Permissions, where every edit and shell command needs approval; Auto-Edit; Auto, the default, where a classifier decides; and YOLO. Switch to Ask Permissions, set tools.approvalMode to default, until you trust the model on your codebase.
For isolation, Qwen Code can run inside macOS Seatbelt, whose default profile restricts writes outside the project folder, or in a Docker or Podman container. Our guide to sandboxing agents that execute code explains when each level is enough.
Three settings govern what leaves the machine besides model requests:
{
"privacy": {
"usageStatisticsEnabled": false
},
"general": {
"enableAutoUpdate": false
},
"model": {
"catalogRefresh": false
}
}
- Usage statistics are enabled by default for all authentication methods and go to an Alibaba Cloud collection endpoint.
QWEN_USAGE_STATISTICS_ENABLED=falsedoes the same from the environment and takes precedence. - The model catalogue is refreshed from models.dev at most once every 24 hours; disabling usage statistics also stops it.
- OpenTelemetry export under
telemetry.*goes to a destination you configure. Leavetelemetry.logPromptsoff unless you intend to store prompts.
Gemini CLI has the same privacy.usageStatisticsEnabled setting, also true by default.
How VDF AI fits
VDF AI does not run, host, resell or integrate with Qwen Code, Gemini CLI, Antigravity CLI or LiteLLM. A terminal agent on a local model gives one developer a private workflow. It leaves the model choice, the approval mode and the telemetry switches in a file under each person’s home folder, with no shared record of what the agent read or changed.
VDF Code is VDF AI’s governed coding assistant for organisations that need those decisions made once. Its product page describes deployment in a VPC, in your data centre or fully air-gapped, on Kubernetes, OpenShift or bare metal, with editor extensions for VS Code, JetBrains IDEs, Neovim and Visual Studio. VDF AI Router routes requests across local open-weight models and approved external endpoints under allow and deny lists, and its air-gap mode keeps routing on local models only.
Sources
Repository documentation, model cards and vendor pages consulted on 14 November 2026.
- Qwen Code repository and README and releases
- Qwen Code: model providers, authentication and settings
- Qwen Code docs in the repository: approval mode, sandbox, terms and privacy and troubleshooting
- Gemini CLI repository and README and releases
- Gemini CLI: configuration reference, authentication and local model routing
- Google Developers Blog: transitioning Gemini CLI to Antigravity CLI
- Antigravity: models and SDK local models
- LiteLLM: Gemini CLI tutorial and Ollama provider
- Ollama: context length and OpenAI compatibility
- LM Studio CLI: lms load
- llama.cpp server README
- Model cards: Qwen3-Coder-30B-A3B-Instruct, Qwen3-Coder-Next and Qwen3.8-27B; ggml-org GGUF build
- Ollama library: qwen3-coder, qwen3-coder-next, qwen3.8 and qwen3.6
- Cline: local models in Cline Desktop, for the memory estimates
Running coding agents on local models across a team? See how VDF Code handles it on infrastructure you control, or get in touch.