AMD Radeon AI PRO R9700 for Local LLMs: Specs, ROCm vs Vulkan, Benchmarks and Dual GPU

The Radeon AI PRO R9700 is a 32 GB, 640 GB/s, 300 W workstation card that launched at $1,299. For local LLMs, one card holds 27B to 32B models at 4-bit with long context, at roughly 20 to 30 tokens per second for one user; two cards hold 70B models at 4-bit and four hold gpt-oss-120b. ROCm 10.1 supports it on Linux and Windows, and llama.cpp runs it through both ROCm and Vulkan.

AMD announced the R9700 at Computex on 20 May 2025, sold it in prebuilt workstations from 23 July 2025 and put it on retail shelves on 27 October 2025. It uses the same RDNA 4 graphics processor as the Radeon RX 9070 XT, which ROCm identifies as gfx1201, with twice the memory. A year later, its value for language models depends on the software around it as much as on the silicon.

Specifications come from AMD’s product pages, software status from AMD’s ROCm 10.1 documentation and each project’s repository, and benchmarks from Puget Systems, Phoronix and community results on llama.cpp’s GitHub, all checked on 4 November 2026. Estimated speeds apply the arithmetic of our tokens-per-second calculator and say so. For the wider card market, see our guide to choosing a GPU for local models.

Radeon AI PRO R9700 specs and price

For language models, the 32 GB and 640 GB/s matter most: capacity decides whether a model loads, bandwidth how fast it generates. AMD lists FP8 matrix rates but no FP4, so the card’s fast paths are 16-bit, FP8 and integer formats. Error correction applies only under Linux.

R9700 vs R9700S

AMD also lists the R9700S, with identical specifications apart from passive cooling: it relies on the system’s fans rather than its own. Wccftech reported its arrival on 11 December 2025, without a price. AMD’s footnote on both pages describes its R9000 cards as workstation products not designed or validated for data-centre use, but excepts the R9600 and R9000S cards. For an ordinary workstation, the actively cooled R9700 is the one to buy.

What fits in 32 GB, and in 64 or 128 GB with two or four cards

For each card count we apply the VRAM calculator’s method: weights take parameters × bits ÷ 8, or simply the file size of a published build; each token of context adds a fixed amount of KV cache; and a tenth of each card stays free for the runtime. That leaves 28.8 GB usable on one R9700, 57.6 GB on two and 115.2 GB on four, before the small per-card overhead of splitting a model.

GGUF sizes are ggml-org and Unsloth files on Hugging Face, the FP8 size is Qwen’s own checkpoint, and the Llama figure applies llama.cpp’s 4.89 bits per weight for Q4_K_M. Per-token cache follows each model’s configuration: Qwen3.8-27B stores 64 KiB per token in 16 of its 64 layers, Llama 3.3 70B 320 KiB in all 80.

Three things follow:

• One card is a 30B-class machine. Qwen3.8-27B in 8-bit, 28.6 GB as Q8_0 or 30.9 GB as Qwen’s FP8 release, leaves no room for context, while 4-bit builds leave plenty. Ollama’s qwen3.8:27b tag is an 18 GB download, and our Qwen 3.8 hardware guide sizes the rest of the family.
• Two cards open FP8 serving and 70B at 4-bit. Puget Systems served Qwen3.6-27B in FP8 across two R9700s with vLLM, and Llama 3.3 70B at 4-bit fits with modest context.
• gpt-oss-120b needs three or four cards, because 63.4 GB overflows two cards’ 57.6 GB. Our VRAM tier guide and GPU sizing guide apply the same arithmetic to other models.

R9700 software support in November 2026: ROCm, vLLM, llama.cpp, Ollama and LM Studio

Two findings matter for owners. ROCm 10.1 fixed GPU resets and crashes during PyTorch training or fine-tuning on RX 9070-series and R9700 cards, so update before fine-tuning. And Puget Systems found that multi-GPU vLLM hung under KVM/VFIO passthrough and needed bare metal, and that containers need the /dev/kfd and /dev/dri devices plus the video and render groups. To pick a serving engine, our side-by-side of three local inference engines applies to AMD as much as to NVIDIA.

R9700 ROCm vs Vulkan in llama.cpp

llama.cpp drives the R9700 through two backends: ROCm, through HIP, and Vulkan, which on Linux usually means Mesa’s RADV driver. Phoronix compared them on an R9700 in a System76 workstation on 5 October 2026, using the Lemonade local AI server 2026.39.1, which bundles llama.cpp b10825. Vulkan generated tokens slightly faster, while ROCm tended to deliver the first token noticeably sooner, though not in every case. A November 2025 Phoronix test with ROCm 7.1 had also found Vulkan ahead on several models.

Community results on llama.cpp’s GitHub add detail. One user’s March 2026 tests on Ubuntu 24.04 with RADV measured 29 tokens per second for Qwen3.5-27B at Q4_K_M and about 148 for Qwen3.5-35B-A3B. Setting PCIe ASPM to performance mode added about 11 percent on the dense model, and with a one-line kernel change as well, that model reached about 80 percent of the card’s 640 GB/s.

Install both builds, then compare them on your model and context length:

llama-bench -m Qwen3.8-27B-Q4_K_M.gguf -p 4096 -n 256

Choose Vulkan for throughput or ROCm for responsiveness if your numbers split the way Phoronix’s did.

R9700 LLM performance: estimates and published benchmarks

Our calculator’s method divides usable bandwidth, at 60 percent of peak, by the bytes each token reads, for one user with a 2,048-token prompt and a 512-token reply.

Estimated tokens per second; “tight fit” means the weights fit 24 GB with little room for context.

Two patterns stand out. Measured dense-model speeds reach 73 to 81 percent of the card’s bandwidth, above our 60 percent assumption, so treat our dense estimates as conservative. And prompt processing is where NVIDIA pulls away. On paper the gap is small, with AMD listing 191 dense FP16 matrix TFLOPS and NVIDIA’s whitepaper giving the RTX 5090 209.5 with FP32 accumulation, yet the community comparison measured the 5090 reading prompts 2.6 to 3.4 times faster as context grew. Measure prompt speed rather than reading it off a spec sheet. Puget also timed image generation on one card at 3.6 seconds per 1024 × 1024 image with Z-Image Turbo; our guide to local video generation models covers heavier media workloads.

Dual R9700 and four-card setups

Extra cards add memory first and speed second, depending on how the model is split:

• llama.cpp layer split, the default, puts different layers on different cards. Each token passes through the cards in turn, so one conversation runs at about one card’s pace: by our method, about 9 tokens per second for Llama 3.3 70B at Q4_K_M on two cards and 77 for gpt-oss-120b on four. Extra cards mainly help when several requests run at once.
• llama.cpp row and tensor split divide each layer across cards. Tensor mode is marked experimental; a September 2026 change enabled llama.cpp’s own AllReduce on ROCm, which its author measured at about 16 percent faster prompt processing on a two-card AMD system. Puget hit an assertion with row split on Qwen3.6-27B and used layer split instead.
• Tensor parallelism, which vLLM uses, splits every layer and raises speed per user as well as throughput under load. With it, our estimate for Llama 3.3 70B at 4-bit on two cards rises to about 16 tokens per second; Puget’s FP8 results show how much tuning matters, from 15.9 to 62.8 tokens per second for one user on the same two cards.

Four cards draw up to 1,200 W at rated board power before the CPU, and each wants a PCIe 5.0 x16 slot; Puget’s test machine used a Threadripper PRO workstation board. AMD’s specifications list no card-to-card link, so the cards communicate over PCIe.

R9700 vs RTX 5090, RTX 4090, RTX 3090, Arc Pro B70 and RTX PRO 6000

• R9700 vs RTX 5090. Same 32 GB, but 2.8 times the bandwidth on the NVIDIA card, and in the community test 1.5 times the generation speed and 2.6 to 3.4 times the prompt speed. The R9700 draws 275 W less and launched $700 cheaper; two of them give 64 GB for $2,598.
• R9700 vs RTX 4090 and RTX 3090. The used NVIDIA cards stream faster but hold 24 GB, which leaves 27B to 32B models at 4-bit with little context. The R9700’s extra 8 GB is what keeps long contexts possible.
• R9700 vs Arc Pro B70. The B70 offers similar capacity and bandwidth for $350 less, at 230 W. Puget’s single-card results put it slightly ahead on 8B models, 35.4 against 29.1 tokens per second for Llama 3.1 8B, though Puget notes the B70 numbers came from an earlier method. Intel’s specification page lists no FP8 rate for the B70.
• R9700 vs RTX PRO 6000. Three times the memory and 2.8 times the bandwidth on one card, with mature CUDA serving, at more than twelve times the R9700’s launch price.

Who the R9700 suits

• A developer or small team running 27B to 32B models. One card holds Qwen3.8-27B or Gemma 4 31B at 4-bit with long context at roughly 20 to 30 tokens per second, on Linux or Windows.
• Teams that want 64 GB or more without data-centre cards. Two cards hold FP8 27B models or 70B models at 4-bit; four hold gpt-oss-120b with room for ten long sessions.
• Organisations standardising on AMD, which gain experience with the ROCm stack that AMD Instinct servers also run.

It suits less well anyone who needs the fastest single-user speed under 32 GB, where an RTX 5090 is 1.5 to 2.8 times faster depending on the test; prompt-heavy coding agents and retrieval, where NVIDIA’s lead is largest; and shared production serving, where data-centre GPUs and CUDA tooling still dominate. For more capacity in one box, a Mac Studio with M5 Ultra offers up to 512 GB of unified memory, and RTX Spark laptops reach 128 GB.

How VDF AI fits

The cards serve models; they do not run VDF AI’s platform, whose services our self-hosting requirements size by vCPU, memory and storage. The two connect through VDF AI Router, which can register a vLLM or Ollama endpoint on R9700 cards as a custom on-premises deployment beside the cloud models your policy permits; in air-gap mode, it routes to local models only.

Around that endpoint, the platform adds governed agents, multi-agent orchestration with Networks, private RAG over company documents, role-based access control and audit logs. Swapping the hardware behind an endpoint, from Radeon to GeForce or to rack servers, changes the router’s deployment entry rather than the applications. For software teams, VDF Code is the governed coding assistant that runs on infrastructure you control.