AI Infrastructure

MCP with Ollama, LM Studio, Open WebUI and llama.cpp: Local Model Set-up Guide (2026)

How to use Model Context Protocol servers with a model on your own machine: LM Studio's mcp.json, Open WebUI's native MCP connection and the mcpo proxy, llama.cpp's MCP flags and web UI, and the clients that bridge Ollama, which has no MCP client of its own. Includes a worked filesystem and web example, model picks, context cost and security. Checked against LM Studio 0.4.28, Ollama 0.40.16, Open WebUI 0.13.3 and llama.cpp b12387 on 14 November 2026.

To use MCP with a local model you need an MCP host in front of it. LM Studio is one: add servers to its mcp.json file. Open WebUI connects to Streamable HTTP servers natively and to stdio servers through its mcpo proxy. llama.cpp's server and web UI both speak MCP. Ollama does not, so pair it with a client such as ollmcp, Open WebUI or Goose.

The Model Context Protocol is the common way to hand tools to a model: a server offers tools such as “read this file” or “fetch this page”, and a host application passes them to the model and runs the calls it asks for. Most guides assume a hosted model. This one covers the local case, where the questions are which desktop apps can act as hosts, how each is configured, and whether a model that fits on your hardware can drive the tools.

Every setting below was read in the projects’ own documentation, READMEs and release pages on 14 November 2026, against LM Studio 0.4.28, Ollama 0.40.16, Open WebUI 0.13.3, mcpo 0.0.20 and llama.cpp build b12387. For which servers suit a company, see our review of MCP servers for enterprise use; for the protocol itself, the comparison of MCP and plain APIs.

Which local apps are MCP hosts today

AppMCP roleTransportsWhere servers are configured
LM Studio 0.4.28Host, since 0.3.17Local (stdio) and remote (URL)mcp.json, edited in the app
Ollama 0.40.16None; tool calling API onlyNot applicableUse a separate client
Open WebUI 0.13.3Client, since v0.6.31Streamable HTTP only; stdio and SSE through mcpoAdmin settings, Integrations
llama.cpp web UIHost in the browser, since March 2026Streamable HTTP, SSE and WebSocketMCP servers page in the UI
llama.cpp llama-serverStarts stdio servers, since July 2026stdio only--mcp-servers-config JSON file
ollmcp 0.35.1Terminal client for Ollama modelsstdio, SSE, Streamable HTTPollmcp mcp add

Two transport terms decide what works where. A stdio server is a program the host starts as a child process and talks to over standard input and output; it opens no port. A Streamable HTTP server runs as its own web service and is reached by URL. The reference servers repository notes that all of its servers use stdio and only the test server also offers Streamable HTTP, which is why Open WebUI needs a proxy for most of them. The current specification revision is 2026-07-28; the older HTTP+SSE transport is deprecated.

LM Studio MCP: the mcp.json file

LM Studio has acted as an MCP host since version 0.3.17, announced on 25 June 2025, and supports local and remote servers.

  1. In a chat, switch to the Program tab in the right-hand sidebar.

  2. Choose Install, then Edit mcp.json. The file opens in the in-app editor. On disk it is ~/.lmstudio/mcp.json on macOS and Linux and %USERPROFILE%/.lmstudio/mcp.json on Windows.

  3. Add servers under mcpServers. LM Studio follows Cursor’s notation: command, args and env for a local server, url and headers for a remote one. This is the remote example from LM Studio’s documentation:

    {
      "mcpServers": {
        "hf-mcp-server": {
          "url": "https://huggingface.co/mcp",
          "headers": {
            "Authorization": "Bearer <YOUR_HF_TOKEN>"
          }
        }
      }
    }
  4. Save. LM Studio loads the servers and starts a separate process for each one. Local servers that rely on npx or uvx need those programs on your PATH.

When the model asks for a tool, LM Studio shows a confirmation dialog in which you can read and edit the arguments, then allow the call once or always. Tools you allowed permanently are managed under App Settings > Tools & Integrations. Some sites offer an “Add to LM Studio” button, a deep link that writes the entry for you.

MCP through LM Studio’s API

From LM Studio 0.4.0 the same servers work over the REST API. A request to /api/v1/chat lists them under integrations, either by ID for a server in mcp.json (for example "mcp/playwright") or as an ephemeral_mcp entry with a server_url. Each route has its own switch in Server Settings: “Allow calling servers from mcp.json” and “Allow per-request MCPs”. An allowed_tools list limits which tools the model sees.

Ollama MCP: no built-in client, so add one

Ollama runs models and exposes tool calling through its API: you send tool definitions, the model returns a call, your code runs it. It does not connect to MCP servers. The documentation index lists no MCP client feature, the request for one (issue 7865, opened in November 2024) is still open, and so is a later request for MCP tools in the Ollama app. Something else has to be the host.

OptionWhat it isGood for
ollmcpMIT-licensed terminal client built for Ollama; v0.35.1 of 30 September 2026Quick tests, human approval of each call
Open WebUIWeb chat interface in front of OllamaA browser UI, several users
GooseOpen-source agent with desktop and CLI apps; its extensions connect through MCPLonger agent tasks
A coding agent via ollama launchOpenCode and similar agents, which are MCP clients themselvesWork inside a code repository

The quickest test is ollmcp, which needs Python 3.11 or later, supports stdio, SSE and Streamable HTTP servers, and confirms each tool call by default:

uv tool install --upgrade ollmcp
ollmcp mcp add filesystem -- npx -y @modelcontextprotocol/server-filesystem ~/Documents/mcp-test
ollmcp

Goose is covered on Ollama’s own integrations pages: in Goose, open Settings, choose Configure Provider, pick Ollama and confirm the host http://localhost:11434. For coding agents, our OpenCode set-up guide shows where MCP servers go in that agent’s config. Older tutorials recommend mcphost; its repository is now archived, so prefer a maintained client.

Two Ollama details matter for any of these. First, context: Ollama’s docs say the default is 4K tokens on machines with under 24 GiB of VRAM, 32K from 24 to 48 GiB and 256K above that, and that agents and web search should have at least 64,000 tokens. Set it with the slider in the Ollama app or with OLLAMA_CONTEXT_LENGTH=64000 ollama serve. Second, Ollama does ship one MCP server: a Python example, web-search-mcp.py, that offers its hosted web search and fetch API to any MCP client. It needs an API key from a free Ollama account, and queries go to Ollama’s service.

Open WebUI MCP: native connection or the mcpo proxy

Open WebUI has two routes. Its docs are frank that OpenAPI tool servers remain the preferred path for most deployments, and that MCP makes sense when you already rely on MCP servers.

Native MCP (Streamable HTTP)

  1. As an administrator, open Settings > Integrations.
  2. Under External Tool Servers, select + Add Connection.
  3. Set Type to MCP (Streamable HTTP), enter the server URL and choose the authentication: None, Bearer, Session, OAuth or OAuth 2.1.
  4. Save, then enable the tool in a chat from the + menu.

Native connections are admin-only. If Open WebUI runs in Docker and the server runs on the host, use host.docker.internal rather than localhost. Set a fixed WEBUI_SECRET_KEY, or OAuth-connected tools fail with a token decryption error after the container is recreated.

mcpo for stdio servers

mcpo (MIT, version 0.0.20) wraps an MCP server command and publishes it as an OpenAPI server:

uvx mcpo --port 8000 --api-key "top-secret" -- uvx mcp-server-fetch

With --config, mcpo reads a Claude Desktop-style mcpServers file and mounts every server on its own path, such as http://localhost:8000/fetch, each with generated docs at /fetch/docs. In Open WebUI, add each path as a separate tool server under Settings > Integrations; the root URL will not work. A connection added in your personal settings is called from your browser, so localhost is your computer. One added in the admin settings is called from the Open WebUI back end.

Tool use depends on the Function Calling setting, which defaults to Native; Legacy mode injects tools into the prompt and allows a single round. Our comparison of Open WebUI, LibreChat and AnythingLLM covers how the three differ on MCP.

llama.cpp MCP: llama-server flags and the web UI

llama.cpp has two separate MCP features. Installing and securing the server itself is covered in our llama-server walkthrough.

Server-side stdio servers. Since a change merged on 25 July 2026, llama-server can start MCP servers itself. Declare them in a Cursor-compatible JSON file and pass it at start-up:

llama-server -m model.gguf --mcp-servers-config mcp.json

Each entry accepts command, args, env, cwd and timeout_ms, which defaults to 30,000 per tool call. The server starts every entry once to list its tools, stops it, and starts it again when a tool is called. Tools appear as <server>_<tool> in the web UI and under GET /tools. Only stdio is supported here. --mcp-servers-json takes the same JSON inline.

The web UI as MCP host. The chat interface served by llama-server gained an MCP client on 6 March 2026: a server management page, an agentic tool loop, prompts and resources. It runs in your browser, so it connects to servers by URL over Streamable HTTP, SSE or WebSocket. The --ui-mcp-proxy flag exists only to reach public servers that send no CORS headers, and it refuses local and private addresses.

Three points from the README to keep in mind. All of this is marked experimental. Enabling MCP or the built-in --tools restricts --cors-origins to localhost by default, and exposing them on a non-loopback address without an API key requires --allow-unauthenticated-tools. And llama-server now listens on port 9931 by default, so older instructions that open port 8080 need --port 8080. If you are choosing a runtime, our comparison of vLLM, Ollama and llama.cpp covers the trade-offs.

Worked example: a filesystem server and web fetch

Two reference servers make a safe first test: one reads a folder you choose, the other fetches a web page as Markdown.

  1. Create a test folder and install the launchers: Node.js for npx and uv for uvx.

  2. Put this in the host’s config. The same mcpServers block works in LM Studio’s mcp.json, in a file passed to llama-server --mcp-servers-config and in an mcpo config file:

    {
      "mcpServers": {
        "filesystem": {
          "command": "npx",
          "args": [
            "-y",
            "@modelcontextprotocol/server-filesystem",
            "--read-only",
            "/Users/you/Documents/mcp-test"
          ]
        },
        "fetch": {
          "command": "uvx",
          "args": ["mcp-server-fetch"]
        }
      }
    }
  3. On Windows, change the filesystem entry’s command to cmd and put "/c", "npx" at the front of args, as the servers README instructs.

  4. Ask something that needs both tools: “List the files in my test folder, then fetch https://example.com and summarise it.” Approve each call and check the arguments.

The filesystem server may touch only the directories you pass on the command line. --read-only registers only the tools that read and list, so the model never sees write_file or edit_file; --read-only-dir protects single folders while others stay writable. The fetch server returns 5,000 characters per call by default and the model pages through the rest with start_index. It obeys robots.txt for requests the model makes and refuses private and loopback addresses unless you pass --allow-private-ips.

Fetch reads a page you already know. Search needs another server:

ServerBack endWhat it needs
mcp-searxng (MIT, v2.10.1)Your own SearXNG instanceSEARXNG_URL, JSON output enabled in SearXNG, Node.js 22 or later
Brave Search MCP server (official, v2.1.4)Brave Search APIBRAVE_API_KEY
Ollama web-search-mcp.pyOllama’s hosted search and fetchOLLAMA_API_KEY from a free account
{
  "mcpServers": {
    "searxng": {
      "command": "npx",
      "args": ["-y", "mcp-searxng"],
      "env": { "SEARXNG_URL": "http://localhost:8080" }
    }
  }
}

Only the SearXNG route keeps the query log on infrastructure you control, and even then SearXNG forwards queries to public search engines.

Which local models call MCP tools reliably

To the model, an MCP tool is an ordinary function definition, so the requirement is plain tool calling. LM Studio’s tool-use documentation warns that small models, and models not trained for tools, produce malformed calls that the app cannot parse. Open WebUI’s docs make the same point: when a model ignores a tool, try a stronger model before debugging the connection.

Three models that Ollama’s library tags for tools, as starting points:

ModelOllama tagDownloadNotes
Qwen3.5-9Bqwen3.5:9b6.6 GBApache 2.0; Qwen reports 66.1 on BFCL-V4
gpt-oss-20bgpt-oss:20b14 GBApache 2.0; OpenAI’s card lists native function calling
Qwen3.8-27Bqwen3.8:27b18 GBApache 2.0; 256K native window

The benchmark figure is the vendor’s own. Model selection is a subject of its own, covered in our guide to local models for agents and tool calling, and the VRAM calculator shows whether weights plus the context you need will fit.

How much context MCP tool definitions use

Every connected server adds its tool names, descriptions and parameter schemas to each request. LM Studio’s MCP page warns that servers designed for large hosted models can use so many tokens that a local model slows down and overflows its context. LM Studio’s token counter has included tool definitions since 0.3.17, so you can watch the cost.

LM Studio’s API documentation gives a feel for the scale. Its example request with one permitted tool from a remote server reports 419 input tokens. Its example with a browser-automation server and no tool filter reports 2,614 input tokens for a two-word prompt. They are different servers, so this is an illustration rather than a measurement, but the direction holds. Results cost context too: one default fetch call returns up to 5,000 characters.

Practical limits for a local set-up:

  • Connect few servers. Two or three per task, not a permanent collection.
  • Filter tools. allowed_tools in LM Studio’s API and disabledTools in an mcpo config remove definitions the model does not need. LM Studio notes that fewer definitions also speed up prompt processing.
  • Raise the context window. Ollama’s 4K default on smaller GPUs is too little for tool loops; its docs suggest 64,000 tokens for agents.
  • Prefer narrow tools. A read-only filesystem server defines fewer tools than a read-write one.

Local MCP servers run code with your user’s rights

The convenience of stdio is also its risk. The host starts the server as a child process under your account. The llama.cpp README states it plainly: the process runs with the same privileges as the server, so only declare commands you trust. LM Studio’s page says some servers can run arbitrary code, access local files and use your network connection.

  • Know what you launch. npx -y package downloads and runs whatever version is current. Pin a version, or install the package once and point command at it.
  • Limit reach. Give the filesystem server one narrow folder and --read-only. Do not hand a model a shell tool on a machine that holds credentials. llama.cpp’s built-in tools include exec_shell_command, and its --tools-runtime option can move them into a Docker or Podman container.
  • Keep approvals on. LM Studio confirms each call unless you chose “always allow”. ollmcp has a human-in-the-loop mode. The llama.cpp web UI stores the tools you have permanently allowed.
  • Treat remote servers as third parties. Prompts, tool arguments and results travel to that service. Tokens in mcp.json headers sit in a plain file. In Open WebUI only administrators can add MCP servers.
  • Expect prompt injection. A fetched page or a file can contain text written to steer the model. Read-only tools and approvals limit the damage; our prompt injection prevention guide covers the controls.
  • Stay on loopback. Do not expose mcpo or llama-server with tools to a network without an API key.

Once several people share servers and credentials, a central control point becomes worth its cost; our MCP gateway comparison covers that layer, and the MCP gateway page maps the specification’s security practices.

How VDF AI fits

Everything above suits one person at one machine: you choose the servers, you approve the calls and the model runs on your hardware. A team needs different answers: who approves a tool, which role may call it, and where the record of each call is kept. VDF AI does not run, host, resell or integrate with LM Studio, Ollama’s app, Open WebUI or llama.cpp’s web UI.

VDF AI Agents is the governed alternative for organisations. Per its product page, agents take tools from an MCP-powered registry, access to tools is role-based, high-impact actions wait at human approval gates, and every prompt, retrieval and tool call is written to an audit log, on your own infrastructure if required. For the models behind those agents, VDF AI Router registers Ollama and other on-premises deployments next to approved cloud models, and its air-gap mode keeps routing on local models only.

Sources

Documentation, READMEs, release pages and issue trackers read on 14 November 2026.


Moving from one person’s MCP set-up to tools a whole team can use safely? See VDF AI Agents for a governed tool registry on your own infrastructure, or book a demo.

Frequently asked questions

Does Ollama support MCP?

Not as a client. Ollama's documentation describes tool calling through its API, and the long-running GitHub request for Model Context Protocol support, issue 7865, was still open on 14 November 2026. To use MCP servers with an Ollama model you put an MCP client in front of it: the ollmcp terminal client, Open WebUI, Goose, or a coding agent started with ollama launch. Ollama does publish one MCP server of its own, a Python example that exposes its hosted web search and fetch API to any MCP client.

How do I add an MCP server to LM Studio?

Open the Program tab in the right-hand sidebar, choose Install, then Edit mcp.json, and add an entry under mcpServers in Cursor's notation: command and args for a local server, or url and headers for a remote one. LM Studio loads the servers when you save the file and starts a separate process for each. The file lives at .lmstudio/mcp.json in your home folder. LM Studio has been an MCP host since version 0.3.17, and it asks for confirmation before each tool call unless you allow that tool permanently.

Does Open WebUI support MCP?

Yes, in two ways. Since v0.6.31 an administrator can add a server under Settings, Integrations, External Tool Servers with the type MCP (Streamable HTTP); stdio and SSE servers are not supported natively. For those, Open WebUI maintains mcpo, a small proxy that wraps an MCP server command and publishes it as an OpenAPI tool server, which you then add as an ordinary tool connection. Open WebUI's own MCP page still calls OpenAPI the preferred integration path for most deployments.

Does llama.cpp support MCP?

Yes. The web UI that ships with llama-server gained an MCP client in March 2026, with a server management page, tool calls in an agentic loop, prompts and resources; it reaches servers by URL over Streamable HTTP, SSE or WebSocket. Since July 2026 llama-server can also start stdio MCP servers itself from a Cursor-style JSON file passed with --mcp-servers-config, and lists their tools next to its built-in ones. Both features are marked experimental in the server README, which warns against enabling them in untrusted environments.

How do I give LM Studio or another local app web search through MCP?

Add a search server next to a fetch server. The reference mcp-server-fetch server retrieves a single URL as Markdown and needs no key. For search, mcp-searxng queries a SearXNG instance you run with JSON output enabled, Brave publishes an official server that needs a Brave Search API key, and Ollama's example server uses its hosted search with a key from a free Ollama account. Search results are long, so raise the model's context before you blame the model for stalled answers.

Are local MCP servers safe to run?

A stdio MCP server is a program started under your user account, so it can do whatever you can: read files, run commands and use the network. LM Studio's documentation and the llama.cpp README both say to add only servers you trust. Reduce the risk by pinning package versions instead of running whatever npx fetches, starting the filesystem server with --read-only on a narrow folder, keeping per-call approval switched on, and remembering that any page a fetch tool reads can contain instructions aimed at the model.

Filed under
Model Context ProtocolOllamaLM StudioOpen WebUIllama.cpplocal LLM
On-Prem AI

Plan your on-prem AI deployment

Book an architecture call and we will scope a private, on-prem AI deployment for your environment — integrations, hardware, and governance included.

Keep reading