RTX 5090 for Local LLMs: What 32 GB Runs, Tokens per Second, Dual 5090 and 5090 vs 4090

An RTX 5090 runs 27B to 35B open-weight models at 4-bit entirely in its 32 GB, with 120,000 tokens of context or more. Published tests put Qwen3.8-27B at about 59 tokens per second in LM Studio, and two cards under vLLM at 100 to 110 with multi-token prediction. The catch in November 2026 is price: Tom's Hardware's tracker lists the cheapest US card at $6,919.

The GeForce RTX 5090 is the only 32 GB card in NVIDIA’s consumer range, and it has the highest memory bandwidth of any GeForce card. For language models that combination decides both what loads and how fast it answers.

We read NVIDIA’s specification and comparison pages, Tom’s Hardware’s price index, model files on Hugging Face and each engine’s documentation on 14 November 2026; every benchmark below carries its own source and date. This guide is about the 5090 only. For other cards see our guide to choosing a GPU for local models, and for a ranking of models at each memory size see the VRAM tier guide.

RTX 5090 specs that matter for LLMs

Capacity sets what loads. Following our VRAM calculator, we leave a tenth of the card for the runtime, so 28.8 GB is the planning figure. Bandwidth sets generation speed, because each new token reads the model’s weights once. Reading the prompt is a compute job, which is where the Tensor Cores earn their keep.

Unlike a unified-memory laptop, the card’s 32 GB is all it has. A model that overflows it either fails to load or runs partly from system RAM at a fraction of the speed, so the fit arithmetic below matters more here than on a Mac.

RTX 5090 price and availability in November 2026

NVIDIA’s product page prints no price for the card today; the price field is a placeholder. The best public tracker we found is Tom’s Hardware’s GPU price index, last updated on 3 November 2026:

At that price the 5090 costs about $216 per gigabyte of VRAM, against $62 at launch. When Tom’s Hardware benchmarked two cards in September, it put the price of its complete dual-5090 test platform above $13,000.

We found no tracker that publishes a transparent November sold-price average for used RTX 4090 or 3090 cards, so we quote none. Check completed listings rather than asking prices.

Best LLM for the RTX 5090: what fits with context

A model that loads is not yet a model you can use. The table shows what is left for the KV cache after the weights, and how much context that buys with a 16-bit cache.

File sizes are today’s Hugging Face listings from ggml-org, Unsloth and NVIDIA. Per-token cache follows each model’s attention layout: 64 KiB for Qwen3.8-27B, about 20 KB for Qwen3.6-35B-A3B and about 25 KB for gpt-oss-20b.

How to choose among them:

• Agents and long documents: Qwen3.6-35B-A3B. It is the only model here that keeps its whole window, and as a mixture-of-experts model it reads few bytes per token.
• Strongest dense model: Qwen3.8-27B at 4-bit. Our Qwen 3.8 sizing guide covers its builds.
• Several users at once: gpt-oss-20b, which leaves room for five long sessions; see the gpt-oss setup guide.
• Image and audio input: the Gemma 4 family, covered in our Gemma 4 deployment guide.

What does not fit one card: Qwen3.8-27B in FP8 (30.9 GB), Llama 3.3 70B at Q4_K_M (42.5 GB) and gpt-oss-120b (63.4 GB).

RTX 5090 tokens per second: published measurements

Our estimate, from the method behind the tokens-per-second calculator, divides 60 percent of peak bandwidth by the bytes each token reads. It gives about 55 tokens per second for Qwen3.8-27B at Q4_K_M and about 297 for gpt-oss-20b, for one user. MacStories’ 59 tokens per second on a 19.4 GB file works out at 64 percent of the card’s 1,792 GB/s, so the estimate is close.

Long contexts change the picture. MacStories stopped its 256K-token run on the 5090 with the default 16-bit cache after ten minutes, with 3 percent of the prompt read; with an 8-bit cache the card finished at 30 tokens per second. Tom’s Hardware found that llama.cpp would allocate the full 262K window for Qwen 3.8 27B on a 5090, but that time to first token stretched to roughly 30 minutes, and it suspected a bug. That was September’s build; rerun the test before you rely on either result.

llama.cpp, vLLM or SGLang on one RTX 5090

The same card gave 59 tokens per second in one test and 20 in another because the engines differ.

• llama.cpp, LM Studio and Ollama load GGUF files, use nearly all of the card and gave the best single-user numbers above at short and medium context. llama.cpp tagged version 1.0.0 on 13 November 2026 and still publishes nightly builds; b12387 is from 14 November.
• vLLM is built to serve many requests. Tom’s Hardware used the NVFP4 build and the maintainers’ recipe: 32K of context on one card, about 20 tokens per second without multi-token prediction, which a single card had no memory left to enable. Loading needed 64 GB of RAM plus 64 GB of swap.
• SGLang was almost three times faster than vLLM’s single-card recipe in the same test, with 37,740 tokens of context.

Measure before you standardise. The first command is the one llama.cpp’s benchmark thread uses; the second starts vLLM with the NVFP4 build:

llama-bench -m Qwen3.8-27B-Q4_K_M.gguf -ngl 99 -fa 0,1
vllm serve nvidia/Qwen3.8-27B-NVFP4 --max-model-len 32768

Our comparison of vLLM, Ollama and llama.cpp covers the trade-offs, and the guide to serving Ollama to several users the access controls a shared endpoint needs.

Setting up an RTX 5090 with Ollama

Ollama is the shortest path from a new card to a working model, with one setting to change.

1. Pull a build that fits. Ollama’s qwen3.8:27b tag is an 18 GB download with a 256K window listed.
2. Raise the context length. Ollama picks its default from the card’s memory: 32K tokens for 24 to 48 GiB of VRAM, which is where a 5090 falls. Its documentation says agents, web search and coding tools should have at least 64,000.
3. Check where the model landed. ollama ps shows the context in use and, in the PROCESSOR column, whether the model sits entirely on the GPU.

ollama pull qwen3.8:27b
OLLAMA_CONTEXT_LENGTH=64000 ollama serve
ollama ps

If PROCESSOR shows anything other than 100% GPU, the context you asked for no longer fits beside the weights. Lower it, or switch to a smaller build, before judging the card’s speed.

Dual RTX 5090 for LLMs: what scales, power and case

Two cards give 64 GB, or 57.6 GB by our rule. What that adds:

gpt-oss-120b, at 63.4 GB, still does not fit.

Whether the second card adds speed depends on how the model is divided:

• llama.cpp layer split, the default, places different layers on each card. The documentation calls it pipelined: a token passes through one card, then the other, so one conversation runs at about one card’s pace. Its row mode divides weights across cards, and its tensor mode is marked experimental.
• Ollama loads a model on a single GPU when it fits, because that avoids moving data across the PCI bus, and spreads it across all GPUs only when it does not.
• vLLM and SGLang tensor parallelism split every layer across both cards with one launch flag, --tensor-parallel-size 2 in vLLM. This is the configuration that took Tom’s Hardware from about 20 to 70–80 tokens per second on Qwen 3.8 27B, with the full 262K window, at some cost to prompt speed. vLLM’s documentation adds that on GPUs without NVLink, pipeline parallelism can give higher throughput, so test both.

Power and space, from NVIDIA’s figures and Tom’s test bench:

• Power. Two cards draw up to 1,150 W on their own. NVIDIA requires a 1,000 W supply for one. Tom’s Hardware ran two on a 1,600 W unit, close to the 1,800 W that a 15 A, 120 V circuit can deliver in total.
• Cables. Each card needs its own 600 W cable or four 8-pin leads.
• Space. The Founders Edition is a two-slot, 304 mm card, and NVIDIA’s clearance note asks for three slots (61 mm) plus room for the power cable. Two cards therefore need slots spaced well apart.
• System memory. vLLM’s loading needs, above, apply twice over.

RTX 5090 vs 4090 vs used 3090 for LLMs

• 5090 vs 4090. Generation is about 1.6 times faster on the 5090, close to the 1.8 times its bandwidth suggests, while short prompts read at the same speed. The extra 8 GB matters more. Tom’s Hardware fitted Qwen 3.8 27B at Q4_K_M on 24 GB cards only with an 8-bit cache and about 112,000 tokens of context, and noted that those cards need every byte, so a desktop session should run on a separate graphics adapter.
• 5090 vs used 3090. The 3090 writes about half as fast and reads prompts at just over a third of the speed, with the same 24 GB limits. It remains the cheap way into 24 GB, and two of them hold what one 5090 cannot, at 700 W.
• Which to buy. At November’s prices the 5090 is hard to justify on speed alone. Buy it when you need 32 GB on one card: 4-bit 27B to 35B models with long context, or image and video models that state a 32 GB minimum.

Image and video generation on an RTX 5090

Language models are not the card’s only local workload. Black Forest Labs says its distilled FLUX 3 Klein model, in the INT8 build, generates a 1-megapixel image in about 4 seconds on an RTX 5090. For video, Lightricks sets the minimum for LTX-2.5 at an NVIDIA GPU with 32 GB of VRAM, and recommends an 80 GB A100 or an H100, so the 5090 is the entry card for that model.

Our guides to local image generation models and local video generation models size the rest of the field.

When to step up to a workstation card

Step up when one of these is true:

• You need the full 262K window on a dense 27B model, fast. Tom’s Hardware’s conclusion was that this takes more than 32 GB on one card, or two cards.
• You want FP8 or 8-bit weights. Qwen3.8-27B in FP8 is 30.9 GB before any context.
• You need 70B to 120B models, or several long sessions at once.
• You want error-correcting memory or a card rated for a workstation chassis.

NVIDIA’s RTX PRO 5000 Blackwell comes with 48 or 72 GB of GDDR7 with ECC at 300 W in a dual-slot card, and the 72 GB version lists 1,344 GB/s of bandwidth. The RTX PRO 6000 Blackwell has 96 GB. In llama.cpp’s thread it writes 281 tokens per second and reads prompts at 16,619 on the 7B test, the 5090’s speed class with three times the memory. Our serving comparison of the RTX PRO 6000, H100 and H200 covers it for teams.

If capacity matters more than speed, unified-memory machines are the other route: RTX Spark laptops reach 128 GB, and AMD’s Strix Halo machines reach 128 to 192 GB at a fraction of the bandwidth.

How VDF AI fits

VDF AI does not sell, resell or host graphics cards, and nothing in its platform is specific to the RTX 5090. The card serves a model; what a team builds around that model is a separate question.

One developer with a 5090 decides everything alone. An organisation has to settle who hosts the model, who approves models and tools, who may call them, what gets recorded and where data stays. VDF AI Router can register an Ollama endpoint or another custom on-premises deployment beside the cloud models policy permits, and its air-gap mode restricts routing to local models; the platform adds governed agents, role-based access control and audit logs. Teams that write software can pair it with VDF Code, a governed coding assistant deployed in your own VPC or data centre.