Claude Code Local LLM Setup: Ollama, LM Studio and llama.cpp (2026)

A Claude Code local LLM setup points the ANTHROPIC_BASE_URL variable at a server on your machine that speaks Anthropic's Messages API: Ollama on port 11434, LM Studio 0.4.1 or later on port 1234, or llama.cpp's llama-server. Use a model with reliable tool calling and at least 64K tokens of context. Anthropic does not support non-Claude models, so troubleshooting is yours.

Claude Code is Anthropic’s coding agent. The client runs on your machine but is designed to send every request to a Claude model. A local setup works because Ollama, LM Studio, llama.cpp and vLLM now answer in Anthropic’s own API format.

Every variable, port and version below was checked on 14 October 2026 against Anthropic’s Claude Code documentation and the Ollama, LM Studio, llama.cpp, vLLM, Claude Code Router and LiteLLM docs. This guide covers the plumbing. For which open-weight model to run, see our coding model shortlist; for how Claude Code differs from open-source agents that support local models by design, see our harness comparison.

How Claude Code connects to a local model

Claude Code reaches its model over the Anthropic Messages API, and Anthropic documents three controls for sending that traffic somewhere other than its own API:

• ANTHROPIC_BASE_URL sets the server address. Claude Code posts to /v1/messages under it, so give the server root rather than a /v1 path.
• ANTHROPIC_AUTH_TOKEN sends a credential as Authorization: Bearer, while ANTHROPIC_API_KEY sends it as x-api-key. A local server usually ignores the value, but a credential variable is also what stops Claude Code asking you to sign in.
• --model names the model on the server. Claude Code treats a name it does not recognise as a current model and assumes a 200K-token window for it.

For permanent settings, use the env block of ~/.claude/settings.json, which overrides shell exports; /status shows what a session is using.

Anthropic’s position is explicit: its gateway documentation says it “doesn’t support routing Claude Code to non-Claude models through any gateway”. The setup works in practice, but you are outside the support boundary. Nor is the client open source: Claude Code’s public GitHub repository carries a licence that reserves all rights and refers to Anthropic’s Commercial Terms.

Servers with a native Anthropic endpoint

When a server only speaks the OpenAI format

Older guides put a translation proxy in the middle, because local servers used to offer only OpenAI-style endpoints. Two projects still fill that gap:

• Claude Code Router (MIT, v3.1.3 released 9 October 2026) is now a local gateway and control plane with a desktop app and an npm CLI. Install it with npm install -g @musistudio/claude-code-router (Node.js 22 or newer) and run ccr ui. The gateway listens on http://127.0.0.1:3456 and puts providers that speak OpenAI Chat or Responses, Anthropic Messages or Gemini behind one local endpoint, with routing rules, fallbacks and request logs.
• LiteLLM runs a proxy on port 4000 that exposes /v1/messages and, in its documentation’s words, translates between provider formats. For Ollama models, LiteLLM recommends the ollama_chat/ prefix.

Both see every prompt, file excerpt and tool result that passes through them. Keep them bound to localhost and switch off features you do not use; Claude Code Router also ships web search, browser automation and chat-app relays.

Claude Code with Ollama

Ollama’s own Claude Code page is the reference for this route. It asks for a context of 64K tokens or more for larger repositories, and Ollama’s default depends on GPU memory: 4K below 24 GiB of VRAM, 32K from 24 to 48 GiB and 256K from 48 GiB.

1. Pull a model that supports tool calling:

   ollama pull qwen3-coder:30b

2. Start the server with a larger context, or use the context slider in the Ollama app’s settings:

   OLLAMA_CONTEXT_LENGTH=65536 ollama serve

3. Launch Claude Code through Ollama’s launcher, available since Ollama 0.15:

   ollama launch claude --model qwen3-coder:30b

   Or set the variables yourself, using the values Ollama’s guide gives:

   export ANTHROPIC_BASE_URL=http://localhost:11434
   export ANTHROPIC_AUTH_TOKEN=ollama
   export ANTHROPIC_API_KEY=""
   claude --model qwen3-coder:30b

4. Run /status in Claude Code to confirm the base URL, and ollama ps in another terminal to see the context the model actually loaded with.

Ollama’s compatibility page lists what its Messages endpoint leaves out: tool_choice, prompt-caching cache_control blocks, the token-counting endpoint, PDF documents and image URLs. Without token counting, Claude Code falls back to a character-based estimate of how full the context is.

A note on Claude Code with Ollama cloud: tags ending in :cloud run in Ollama’s cloud rather than on your machine and need an Ollama sign-in. Ollama says it does not train on those prompts, but it is still a hosted service.

Claude Code with LM Studio

LM Studio 0.4.1, announced on 30 January 2026, added POST /v1/messages, and its announcement says the endpoint works with both GGUF and MLX models. The steps use LM Studio’s lms command line; the desktop app can run the same server.

1. Download a model and load it with enough context. LM Studio’s guide asks for more than about 25K tokens; 64K leaves room for larger repositories:

   lms get qwen/qwen3-coder-30b
   lms load qwen/qwen3-coder-30b --context-length 65536

2. Start the server on its default port:

   lms server start --port 1234

3. Point Claude Code at it with the variables from LM Studio’s guide:

   export ANTHROPIC_BASE_URL=http://localhost:1234
   export ANTHROPIC_AUTH_TOKEN=lmstudio
   export CLAUDE_CODE_ATTRIBUTION_HEADER=0
   claude --model qwen/qwen3-coder-30b

If you switch on Require Authentication in LM Studio, put an LM Studio API token in ANTHROPIC_AUTH_TOKEN instead of the placeholder.

Why the attribution variable: Claude Code prepends a short attribution block to its system prompt, and before version 2.1.181 that block changed on every request sent to a custom base URL. A changing prefix defeats a local server’s prompt cache, as vLLM’s documentation also warns. Setting CLAUDE_CODE_ATTRIBUTION_HEADER=0 removes the block.

Before 0.4.1, LM Studio offered no Anthropic endpoint, so reaching it from Claude Code meant a translating router such as Claude Code Router. A direct connection now needs no router; keep one in between only if you want its routing rules or logs.

Claude Code with llama.cpp or vLLM

llama-server

llama.cpp’s server has an Anthropic-compatible /v1/messages route and, unlike Ollama, a /v1/messages/count_tokens route, so Claude Code gets exact token counts. Its README is careful to promise no strict compatibility with Anthropic’s specification, only that the route works with many applications.

One change will break older tutorials. From build b11521, published on 9 October 2026, llama-server listens on port 9931 by default instead of 8080; the Docker images keep 8080 inside the container. Pass --port explicitly and the difference stops mattering:

llama-server -hf unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M \
  --alias qwen3-coder --host 127.0.0.1 --port 8080 -c 65536 --jinja

export ANTHROPIC_BASE_URL=http://127.0.0.1:8080
export ANTHROPIC_AUTH_TOKEN=local
claude --model qwen3-coder

The -c value matters: the default of 0 takes the context length from the model file, which for a 256K model can need far more memory than you have. The -hf download above is a community quantization, so test the exact file you deploy. If you protect the server with --api-key, the README’s Anthropic example passes the key in an x-api-key header, which is the header ANTHROPIC_API_KEY fills; Claude Code asks once to approve a key set that way.

vLLM

vLLM’s documentation has its own Claude Code page and serves the Anthropic Messages API directly. Most models need --enable-auto-tool-choice plus the right --tool-call-parser, and the page says Hugging Face names containing a slash do not work in this setup, so set --served-model-name. Its example maps every Claude alias to the local model:

ANTHROPIC_BASE_URL=http://localhost:8000 \
ANTHROPIC_API_KEY=dummy \
ANTHROPIC_AUTH_TOKEN=dummy \
ANTHROPIC_DEFAULT_OPUS_MODEL=my-model \
ANTHROPIC_DEFAULT_SONNET_MODEL=my-model \
ANTHROPIC_DEFAULT_HAIKU_MODEL=my-model \
claude

vLLM is the server-class option when several developers share one GPU box. Our inference server comparison covers the trade-offs between vLLM, Ollama and llama.cpp.

Tool calling and context requirements

Claude Code is an agent loop: nearly every turn reads a file, searches, edits or runs a command through a tool call. A model that cannot produce well-formed tool calls stalls on the first step, however well it chats. Ollama’s guide notes that tool calling depends on compatible models, and vLLM’s says Claude Code needs strong tool-calling capabilities.

Three settings keep Claude Code’s assumptions in line with what the server can do:

• CLAUDE_CODE_MAX_CONTEXT_TOKENS. Claude Code assumes 200K tokens for a model name it does not recognise. Set this to the context your server really loaded, such as 65536, and Claude Code compacts the conversation at that size instead of overrunning the server.
• CLAUDE_CODE_MAX_OUTPUT_TOKENS. For unrecognised models the default output cap is 32,000 tokens, reserved out of the same window. Lower it on small contexts.
• The alias variables. Set ANTHROPIC_DEFAULT_OPUS_MODEL, ANTHROPIC_DEFAULT_SONNET_MODEL and ANTHROPIC_DEFAULT_HAIKU_MODEL, plus CLAUDE_CODE_SUBAGENT_MODEL if you use subagents, to the local model name, as vLLM’s guide does, so that background tasks and aliases resolve to a model the server actually has.

Choosing the model itself is covered in our shortlist of open-weight coding models, and the VRAM calculator sizes weights plus context for your hardware. Three tool-calling examples, as their model cards and Ollama’s library describe them:

What changes without a Claude model

Then there is the model itself. Our local-versus-Claude comparison collects the published benchmark runs that put open-weight models and Claude through the same harness. If you want an open-source agent that treats local models as a first-class option, see our guides to OpenCode with a local model and the Pi coding agent.

Keeping code and telemetry on the machine

A local base URL moves inference onto your hardware. It does not, on its own, stop the client talking to Anthropic. The data-usage and gateway pages list what remains and how to switch it off:

• CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 disables auto-updates, telemetry, error reporting, the /feedback command, release notes and availability checks. Setting it to 0 still disables them; unset it to turn them back on.
• Anthropic states that its usage metrics never include code, prompts or file paths. A /feedback report can carry the conversation, including code; the variable above disables the command.
• The WebFetch tool sends each requested hostname, and only the hostname, to api.anthropic.com for a blocklist check, whatever the model provider. skipWebFetchPreflight: true stops that, and a bare WebFetch deny rule removes the tool entirely.
• Official plugin marketplace registration has its own switch, CLAUDE_CODE_DISABLE_OFFICIAL_MARKETPLACE_AUTOINSTALL=1.
• Session transcripts stay on disk in plain text under ~/.claude/projects/ for 30 days by default; cleanupPeriodDays changes that.

Put together, a user-level ~/.claude/settings.json for an Ollama setup looks like this:

{
  "env": {
    "ANTHROPIC_BASE_URL": "http://localhost:11434",
    "ANTHROPIC_AUTH_TOKEN": "ollama",
    "ANTHROPIC_API_KEY": "",
    "CLAUDE_CODE_MAX_CONTEXT_TOKENS": "65536",
    "CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
    "CLAUDE_CODE_DISABLE_OFFICIAL_MARKETPLACE_AUTOINSTALL": "1"
  },
  "skipWebFetchPreflight": true,
  "permissions": {
    "defaultMode": "default",
    "deny": ["WebFetch", "WebSearch"]
  }
}

With auto-updates off, plan how you will update instead. Anthropic’s system requirements still list an internet connection, so test the setup with outbound traffic blocked before relying on it. Claude Code’s built-in sandbox, off by default, limits the hosts and paths shell commands can reach on macOS, Linux and WSL 2; for stronger boundaries see our guide to sandboxing agents that execute code.

Windows setup notes

• Install. Run irm https://claude.ai/install.ps1 | iex in PowerShell, which does not need administrator rights, or winget install Anthropic.ClaudeCode, which does not update itself.

• Shell. Git for Windows is optional. With it, Claude Code uses Git Bash for its Bash tool; without it, commands go through its PowerShell tool.

• Variables. Set them for the session in PowerShell, or put the same env block in %USERPROFILE%\.claude\settings.json:

  $env:ANTHROPIC_BASE_URL = "http://localhost:11434"
  $env:ANTHROPIC_AUTH_TOKEN = "ollama"
  claude --model qwen3-coder:30b

• VS Code. For Claude Code with Ollama in VS Code, set the variables in claudeCode.environmentVariables in VS Code’s user settings. The extension checks credentials there before it launches.

• Sandbox. Claude Code’s sandbox does not run on native Windows. Use WSL 2 if you need it.

• WSL 2 talking to Windows. In WSL’s default NAT mode, reach a Windows-side server through the host IP (ip route show | grep -i default | awk '{ print $3}'), with Ollama listening beyond 127.0.0.1, for example OLLAMA_HOST=0.0.0.0:11434, which also exposes it to your network unless a firewall blocks it. On Windows 11 22H2 or later, networkingMode=mirrored under [wsl2] in .wslconfig lets both sides use localhost.

How VDF AI fits

A local model behind Claude Code suits one developer who accepts that Anthropic will not support the arrangement. It gives a team no central policy or audit trail, and any Claude Code update can change what reaches the local server. VDF AI does not run, host or integrate with Claude Code.

For organisations that want those controls on their own infrastructure, VDF Code is VDF AI’s governed alternative. Its product page describes extensions for VS Code, JetBrains IDEs, Neovim and Visual Studio, deployment in a VPC or on Kubernetes, OpenShift or bare metal, including air-gapped sites, and policies that allow or block models, restrict tools per repository and gate privileged operations behind approval. Prompts, model calls, tool invocations and edits go to an audit log.