Ollama Multiple Users: Concurrency, Security and When Ollama Is Production Ready

Ollama serves multiple users once you change three defaults: it processes one request per model at a time, it listens only on localhost, and its API has no authentication. Raise OLLAMA_NUM_PARALLEL within your GPU memory, put an authenticating TLS proxy in front, and it suits a small team. For dozens of simultaneous users, servers built around continuous batching, such as vLLM or SGLang, fit better.

Most Ollama installations begin on one laptop or workstation. The questions in this guide arrive when colleagues want the same model: how many requests it can take at once, what that costs in memory, who can reach the API, and whether to keep Ollama or change servers. Facts come from Ollama’s documentation, FAQ and release notes, the US National Vulnerability Database, and the Open WebUI, vLLM and SGLang documentation, all checked on 14 October 2026, when Ollama’s current release was v0.40.4 from 12 October.

For where Ollama sits in a larger architecture, read where Ollama fits in an enterprise stack; for the engines side by side, see our inference server comparison. This guide is the practical step between the two.

How Ollama handles concurrent requests

Ollama’s FAQ describes two levels of concurrency: several models can be loaded at once if memory allows, and each loaded model can work on several requests in parallel. These settings control it:

With the defaults, a second person waits until the first answer is complete. Two more behaviours matter for a team. When a request needs a model that does not fit beside those already loaded, Ollama queues every new request until idle models unload and the new one loads, so juggling several large models on one GPU causes stalls. And on a multi-GPU server, Ollama places a model on a single GPU whenever it fits there, spreading it across all GPUs only when it does not.

On Linux, set the variables in a systemd override, as Ollama’s FAQ describes:

sudo systemctl edit ollama.service

[Service]
Environment="OLLAMA_NUM_PARALLEL=4"
Environment="OLLAMA_CONTEXT_LENGTH=32768"
Environment="OLLAMA_KEEP_ALIVE=-1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"

sudo systemctl daemon-reload
sudo systemctl restart ollama

Keeping the model resident with OLLAMA_KEEP_ALIVE=-1 spares the first user of the day a load delay, at the price of holding the memory permanently. Set the context on the server, too: Ollama’s compatibility notes say OpenAI-style clients have no way to set the context size, so they get whatever the server or a Modelfile defines.

How much memory multiple users need

Parallel slots are not free. Ollama’s FAQ says each parallel request adds its own context, so a 2K window with four parallel requests becomes 8K, and memory grows with OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH. Size it with the method from our GPU sizing guide: keep 10% of GPU memory free, subtract the model, and divide what remains by the cache per token.

gpt-oss makes a clear example because OpenAI publishes its attention layout. With a 16-bit cache, gpt-oss-20b stores 24,576 bytes per token in its full-attention layers and gpt-oss-120b 36,864, while their sliding-window layers add only a few megabytes per slot. Ollama’s library lists the models at 14 GB and 65 GB.

Ollama’s FAQ says the 8-bit cache uses about half the memory of 16-bit with a very small loss of precision. It requires Flash Attention, which Ollama enables automatically where the hardware supports it, and it applies to every model on the server. Slots also share the GPU’s speed: more parallel requests raise total throughput but slow each individual answer.

If 10 to 20% of named users are active at the busiest moment, as our sizing guide assumes, four slots cover a team of roughly 20 to 40 for chat. Agents and coding tools are different: Ollama’s own guidance asks for at least 64,000 tokens of context for them, so give them a separate model or server. Our VRAM calculator runs the same arithmetic for other models.

Ollama has no authentication: what exposed servers face

Ollama’s API documentation is explicit that the local API on port 11434 requires no authentication; its API keys exist for Ollama’s cloud. That is safe enough on one machine, because the server binds to 127.0.0.1. The risk starts when someone sets OLLAMA_HOST=0.0.0.0 so colleagues can connect. From then on, every client that can reach the port can call every endpoint, including the ones that pull, create, push and delete models.

The US National Vulnerability Database shows why that matters:

• CVE-2026-103663, published 8 October 2026: a path traversal in /api/pull lets an unauthenticated attacker write a file outside the model store. Where the server can write to /usr/lib/ollama, which the entry says is the default in most Ollama Docker images, the file runs as root after the next restart. Fixed in 0.35.0.
• CVE-2026-7482, May 2026: a crafted model sent to /api/create makes the server read beyond a buffer during quantization. The leaked memory can include environment variables, API keys, system prompts and other users’ conversations, and can be sent out through /api/push. The entry also notes that the 0.0.0.0 setting is widely used and that large numbers of servers are exposed on the public internet. Fixed in 0.17.1.
• CVE-2026-108836, October 2026: server-side request forgery through the pull API, fixed in 0.34.2.
• CVE-2025-63389: NVD records the unauthenticated model-management endpoints themselves as a critical issue in versions up to 0.12.3.

Two rules follow. Run a current release, 0.35.0 or later, and never let users reach the model-management endpoints.

Putting Ollama behind an authenticating reverse proxy

1. Keep Ollama on localhost. Leave OLLAMA_HOST at its default when the proxy runs on the same machine. If it must listen elsewhere, bind it to a private address and firewall the port so only the proxy reaches it.

2. Terminate TLS and require a key at the proxy. Ollama’s FAQ shows nginx forwarding to the server with the Host header set to localhost:11434. This version adds TLS, per-team keys and an allowlist of inference paths:

   # /etc/nginx/conf.d/ollama.conf (http context)
   map $http_authorization $ollama_client {
       default                            "";
       "Bearer CHANGE-ME-ANALYTICS-KEY"   "analytics";
       "Bearer CHANGE-ME-LEGAL-KEY"       "legal";
   }
   
   log_format ollama '$time_iso8601 client=$ollama_client addr=$remote_addr '
                     '"$request" status=$status seconds=$request_time';
   
   server {
       listen 443 ssl;
       server_name llm.internal.example;
       ssl_certificate     /etc/nginx/tls/llm.crt;
       ssl_certificate_key /etc/nginx/tls/llm.key;
       access_log /var/log/nginx/ollama.log ollama;
   
       # Inference only: pull, create, push, copy and delete stay unreachable.
       location ~ ^/(api/(chat|generate|embed|tags|show)|v1/(chat/completions|completions|embeddings|responses|messages|models.*))$ {
           if ($ollama_client = "") { return 401; }
           proxy_set_header Authorization "";
           proxy_set_header Host localhost:11434;
           proxy_pass http://127.0.0.1:11434;
           proxy_buffering off;
           proxy_read_timeout 600s;
       }
   
       location / { return 403; }
   }

3. Let answers stream. proxy_buffering off passes tokens to the client as they are generated, and the longer proxy_read_timeout stops slow answers being cut off at nginx’s 60-second default.

4. Add single sign-on for people. Keys suit scripts and IDE tools. For browsers, nginx’s auth_request module can hand sign-in to an OpenID Connect proxy such as oauth2-proxy, which returns the user’s name and email as headers you can log.

5. Manage models on the host. With the management paths blocked, administrators run ollama pull on the server itself.

Clients then treat the proxy as an OpenAI-compatible endpoint: https://llm.internal.example/v1 as the base URL and the team key as the API key. Per-team keys also turn the access log into a usage record by team.

Giving the team accounts: Open WebUI in front of Ollama

A proxy with keys identifies teams, not people. For a shared chat interface with personal accounts and histories, a front end such as Open WebUI sits between users and Ollama, and Ollama stays on the private network.

• Roles. The first account on a new installation becomes the administrator. Regular users get only the permissions granted by defaults and their groups, and pending accounts can do nothing until an administrator approves them. The documentation recommends pending as the default role for shared instances, set with DEFAULT_USER_ROLE.
• Groups and models. Permissions add up across groups, and each model can be private, shared with chosen users or groups, or public.
• Several Ollama servers. Open WebUI can connect to more than one Ollama instance and picks between them at random for each request; the model names must match exactly on every server. Random selection takes no account of which server already has the model loaded.
• Hardening. Open WebUI does not terminate TLS itself, so it belongs behind a reverse proxy too, and its hardening guide recommends single sign-on through your identity provider with local login disabled.
• Patching. NVD lists three 2026 advisories in Open WebUI’s Ollama routes, fixed in versions 0.9.0, 0.9.6 and 0.11.4, through which signed-in users could reach models or Ollama backends they had not been granted.

Check the licence terms before a large rollout; our Open WebUI licence guide covers them, and our round-up of self-hosted chat front ends covers the alternatives.

Monitoring an Ollama server

Ollama has no Prometheus metrics endpoint; a request for one on GitHub has been open since March 2024. Three sources cover most needs:

• Response fields. Every completed response reports prompt_eval_count, prompt_eval_cached_count, eval_count, eval_duration, load_duration and total_duration, with timings in nanoseconds. Generation speed is eval_count ÷ eval_duration × 10⁹, and a large load_duration shows that the request waited for a model to load.
• Loaded models. GET /api/ps lists each resident model with its size, size_vram, context_length and expires_at. A size_vram below size means part of the model has spilled out of GPU memory.
• The proxy log. Requests per team, latency and status codes. A run of 503 responses means the queue is full and the server needs more slots or more hardware.

curl -s http://127.0.0.1:11434/api/ps

Add the GPU’s own utilisation and memory readings, and alert on 503s, reloads and spills. For tracing prompts and answers across applications, see our LLM observability round-up.

Is Ollama production ready?

For a small internal service, the answer is yes, with conditions: a current release, a proxy that authenticates and filters paths, slots sized to memory, and monitoring assembled from logs and response fields. It is not a multi-tenant serving platform, and most of its 2026 vulnerabilities sit in the model-management endpoints that an exposed server makes reachable.

When to move to vLLM or SGLang

Move when the proxy log shows regular 503s or long queues at peak, when time to first token climbs as people arrive, when a model needs several GPUs for speed rather than capacity, or when operations want Prometheus metrics.

• vLLM. Its README lists continuous batching of incoming requests, PagedAttention for cache memory, chunked prefill and prefix caching, and an OpenAI-compatible server. It exposes Prometheus metrics at /metrics with a vllm: prefix. It runs on Linux only and claims 92% of each GPU’s memory unless told otherwise.
• SGLang. Its documentation describes it as built for low-latency, high-throughput serving with RadixAttention, prefix caching and multi-GPU parallelism, meant for production use.
• Keep the proxy. vLLM’s --api-key authenticates only the /v1, /v2 and /inference paths, and its documentation warns that other endpoints stay open and that the key alone is not enough.
• Plan the model files. Both engines load Hugging Face checkpoints rather than Ollama’s library, so the migration includes the weights, not only the server.

On a cluster, our Kubernetes hosting guide places the same identity layer at the gateway. When demand outgrows capacity, GPU admission control decides which requests get GPU time first.

How VDF AI fits

VDF AI does not run or manage Ollama. VDF AI Router can register an Ollama endpoint as a model deployment, next to vLLM or other on-premises deployments and the cloud models your policy approves, and its air-gap mode routes to local models only. Applications call the router, so moving a model from Ollama to vLLM as traffic grows changes the deployment behind it, not the applications.

The VDF AI platform supplies what a shared model server lacks: user accounts with role-based access control, audit logs, governed agents and private RAG. On premises, native Microsoft Entra ID single sign-on maps Entra security groups to roles.