GLM-5.3 Local Deployment: Requirements, Model Size and GLM-5.3-Flash Hardware

GLM-5.3 local deployment needs data-centre GPUs: Z.ai's FP8 checkpoint is 755.6 GB, and vLLM's recipe fits it on one node of eight H200s, with eight B200s for the full 1M-token window. GLM-5.3-Flash, the MIT-licensed 320B sibling, is 328.3 GB in FP8 and runs on four H200-class GPUs, or as a community 4-bit build on a 256 GB Mac.

Z.ai published two open-weight models at the end of August 2026: GLM-5.3, its flagship for coding and long-running agent work, and GLM-5.3-Flash, a smaller multimodal model with a hybrid attention design. Both get searched for as local models, yet only one of them runs on hardware that fits under a desk.

This guide covers what was published, how large each download is, what the two licences require, the memory arithmetic for servers and unified-memory machines, and the serving commands. Model cards, configuration files, licences, file listings and serving recipes were checked on 14 October 2026. It follows the format of our Kimi K3 deployment guide, and every memory figure uses the method behind our LLM VRAM calculator.

GLM-5.3 and GLM-5.3-Flash at a glance

Parameter counts are from Z.ai’s model table. The Hub’s file metadata counts 753.3 billion parameters for GLM-5.3 and 321.3 billion for Flash; the checkpoints also hold a multi-token-prediction layer, and Flash carries a 24-layer vision encoder.

Z.ai describes GLM-5.3 as a post-training upgrade on the same base model as GLM-5.2, and Flash as the first natively multimodal model in the GLM-5 series. Reasoning is always on in both: a request picks low, high or max effort, with max as the default. Z.ai’s API also lists GLM-5.3-FlashX, a faster hosted variant that is not among the downloadable weights.

GLM-5.3 size: every build you can download

Sizes are the total of each repository’s model files on Hugging Face on 14 October 2026. Only the first two rows come from Z.ai; the rest are third-party builds.

Three details matter when you compare rows. The MLX Flash build is not uniform 4-bit: its card describes mixed 4-, 5- and 6-bit weights, which is why it is larger than a plain 4-bit estimate, and it loads through mlx-vlm because mlx-lm does not support the architecture. Quality falls with file size, too. Unsloth’s own table gives its Flash builds a top-1 accuracy of 92.2% at UD-Q4_K_XL and 70.9% at UD-IQ1_S. NVIDIA’s NVFP4 Flash card reports results within two points of BF16 on all six benchmarks it lists, but that build needs Blackwell GPUs.

Licences: MIT for Flash, a security-review clause for GLM-5.3

GLM-5.3-Flash ships under the standard MIT License, with Z.AI Co., Ltd as copyright holder and nothing added. Keep the notice with your copies and you may use, modify, fine-tune and redistribute it.

GLM-5.3 uses Z.ai’s own GLM-5.3 License. Its grant reads like MIT and names the weights, running, deploying and fine-tuning explicitly. Clause 2 adds the condition that matters:

• Who it applies to. A licensee whose group, meaning the licensee plus its affiliates, operates a Model-as-a-Service business and has aggregate revenue above 10 billion US dollars over any consecutive 12 months.
• What it requires. Such a licensee “must pass Z.AI’s security review” before using the weights or derivatives for any commercial purpose, with the review’s scope and method reasonably determined by Z.ai.
• What counts as Model as a Service. Giving a third party inference or fine-tuning access, for example through an API, with meaningful control over inputs, parameters or training data. End-user products that embed the model only inside specific features or harnesses are excluded, as is simply relaying requests to models hosted by others.

Most enterprises will never meet the trigger. The exposure sits with large groups: once any affiliate sells API access to language models, the review covers every commercial use of GLM-5.3, internal deployments included, because unlike some other open-weight licences this one has no internal-use carve-out. Our guide to open-weight licence clauses covers the wider review a legal team should run.

GLM-5.3 hardware requirements

Start with the weights. The FP8 release takes 755.6 GB, so eight 80 GB H100s, 640 GB in total by NVIDIA’s specification, cannot hold it, while eight 141 GB H200s, 1,128 GB, can. vLLM’s GLM-5.3 recipe states the same boundary: the FP8 checkpoint fits one 8×H200 node, an 8×B200 node with 180 GB per GPU is its choice for the full 1M-token context, and the BF16 checkpoint needs several nodes.

The cache explains why those two nodes differ. GLM-5.3 uses latent attention: per token, each of its 78 layers stores a compressed 512-value latent plus a 64-value position key, 576 values in all. That is about 90 KB per token at 16-bit, or 45 KB with the FP8 cache the recipe uses. The sparse-attention selector keeps a small index cache of its own on top, which our formula leaves out.

Tensor parallelism changes the arithmetic. With a single latent per token there are no attention heads to divide, so every GPU keeps its own copy of the cache; SGLang’s documentation names this duplication as the cost of running DeepSeek-style latent-attention models with plain tensor parallelism across eight GPUs. The free memory of one GPU, not of the whole node, therefore decides how much context a node can hold:

By this arithmetic an H200 node holds about five 128K-token sessions but not one at the full window, while a B200 node holds a complete 1M-token session, which is consistent with the recipe’s choice of B200 for 1M tokens. The recipe’s AMD measurements point the same way, with caches of about 1.48 million tokens on eight MI325X GPUs and 2.01 million on eight MI350X at 80% memory utilisation.

On unified-memory machines, only community builds fit. Our method counts 80% of the pool as usable, so a Mac Studio with M5 Ultra and 512 GB offers about 410 GB. Unsloth’s UD-IQ4_XS GGUF at 365.3 GB fits with roughly 44 GB left, room for about 490,000 tokens of 16-bit cache if the runtime keeps the compressed latent; check the cache size llama.cpp reports at load. The 4-bit MLX build at 418.3 GB does not fit. Four DGX Sparks linked over their 200 Gb/s ports reach the same 512 GB, with the network in every token’s path.

A 256 GB machine leaves about 205 GB, less than the smallest build at 216.7 GB. Unsloth’s guide says its 238.6 GB 2-bit build works on 256 GB devices such as two linked Sparks or a Mac Studio. That leaves under 20 GB for the operating system and the cache, well short of the fifth of memory our rule reserves, so treat it as an experiment rather than a deployment.

Speed here is bound by memory bandwidth. Each token reads about 40 billion active parameters, roughly 19 GB at UD-IQ4_XS, so the 1.2 TB/s M5 Ultra tops out near 37 tokens per second for one user by the method of our tokens-per-second calculator, an upper-bound estimate rather than a benchmark.

GLM-5.3-Flash local hardware tiers

Flash is the model that fits outside a data centre, thanks to its cache as much as its size. Only 11 of its 45 layers keep a per-token cache, each storing a 512-value latent with no separate position key: about 11 KB per token at 16-bit, an eighth of GLM-5.3’s figure. The other 34 layers use Kimi Delta Attention, a linear-attention design with a fixed state per session. By our arithmetic from config.json (34 layers × 64 heads × a 128 × 128 matrix), that state is roughly 0.07 to 0.15 GB, depending on the precision the runtime keeps it in. A full 1M-token session therefore needs only about 11 GB of cache.

For Flash on DGX Spark, the practical answer is two units. One 128 GB Spark holds only the 1-bit build, which Unsloth’s own table scores well below the 4-bit one; Unsloth demos its 120.4 GB 3-bit build on 128 GB devices, which leaves less than our rule reserves for the system. Two linked Sparks hold the 4-bit-class builds, but decoding still runs at one unit’s 273 GB/s: about 15 tokens per second at best for one user, against roughly 64 on a 1.2 TB/s M5 Ultra. Both are estimates from our calculator’s method, not measurements.

On GPU servers, vLLM’s Flash recipe serves the FP8 weights with tensor parallelism across four GPUs, with examples on four H200s and on one GB200 tray. It lists Hopper and newer NVIDIA GPUs, AMD MI350X and MI355X, and Huawei Ascend 950PR, and reports a cache pool of 14.92 million tokens on four MI355X GPUs, enough for about 114 concurrent sessions at 128K. Our GPU sizing guide shows how to turn a pool like that into a user count.

KTransformers offers a workstation route: its tutorial runs the official FP8 experts on a CPU with an AVX-512 FP8 kernel beside an RTX 40 or 50 series GPU, asks for at least 350 GB of free system memory, and publishes no throughput figures.

GLM-5.3 on premise: serving with vLLM, SGLang and llama.cpp

1. Pin the download. Fetch the weights at a fixed commit and keep a record of it. Both repositories were still being updated on 12 October 2026, and a 4 September commit changed the chat template’s handling of tool results.

   hf download zai-org/GLM-5.3-Flash --revision <commit-sha> --local-dir /models/glm-5.3-flash

2. Install a recent engine. vLLM’s GLM-5.3 recipe lists vLLM 0.28.0 or newer and Transformers 5.15.0 or newer in its prerequisites, but both recipe pages carry a 0.29.0+ badge, so start from 0.29; vLLM 0.31.0 was current on 14 October. SGLang’s cookbook covers GLM-5.3 on H200, B200, B300, GB300 and AMD MI300X to MI355X, and Flash on H100 through GB300 and MI355X.

3. Start the server. This is the recipe’s single-node FP8 command for Flash on four GPUs, with a shorter served model name:

   vllm serve zai-org/GLM-5.3-Flash \
     --tensor-parallel-size 4 \
     --kv-cache-dtype fp8 \
     --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
     --tool-call-parser glm47 \
     --reasoning-parser glm47 \
     --enable-auto-tool-choice \
     --served-model-name glm-5.3-flash

   For GLM-5.3, point at zai-org/GLM-5.3 and use --tensor-parallel-size 8 with the same parsers and multi-token-prediction settings. The Flash recipe notes that in its split prefill and decode setup, Hopper GPUs need a 16-bit cache.

4. Cap context and sessions. Set --max-model-len to what your workloads need rather than the 1M maximum, then tune --max-num-seqs, which the GLM-5.3 recipe calls the main knob for fitting long context.

5. Decide the thinking settings. Z.ai’s API rejects requests that try to disable thinking. Choose reasoning_effort per request, and for chat clients pass clear_thinking: true through the chat template arguments, as the model cards recommend; it defaults to false.

6. Use llama.cpp on smaller machines. Upstream llama.cpp added GLM-5.3-Flash, vision encoder included, on 30 September 2026 in pull request #27773, and multi-token prediction followed on 7 October. Unsloth’s guides use llama.cpp for both models:

   llama-server -hf unsloth/GLM-5.3-Flash-GGUF:UD-IQ4_XS --jinja -c 131072 --temp 1.0 --top-p 0.95

7. Check what Ollama would actually run. Its library lists only glm-5.3:cloud and glm-5.3-flash:cloud, which execute on Ollama’s servers, so prompts leave your network. On a Mac, the mlx-community Flash build runs through mlx-vlm instead.

GLM-5.3-Flash vs DeepSeek-V4-Flash and the full GLM-5.3

DeepSeek-V4-Flash ships at about half the size of GLM-5.3-Flash and activates fewer parameters per token, so each request costs less memory and bandwidth. GLM-5.3-Flash brings image input and a cache small enough for long sessions on modest hardware. Against its own flagship, Flash gives up parameters for a far smaller footprint and a plain MIT licence.

Vendor benchmarks for all three come from the vendors’ own harnesses; our coding model round-up collects the independent results we could find, and your own evaluation should decide. Our deployment notes for the DeepSeek, Qwen and Gemma 4 families cover the alternatives.

How VDF AI fits

A GLM-5.3 deployment gives you a model endpoint. Employees still need access rules, a record of what was asked and answered, and a way to send each task to a model that can handle it. VDF AI supplies that layer on premises: governed agents and workflows, private RAG over company documents, role-based access control and audit logs.

VDF AI Router registers custom on-premises deployments, such as a vLLM endpoint serving GLM-5.3-Flash, next to the cloud models your policy approves, and its air-gap mode keeps routing on local models only. A team can start with Flash on four GPUs and add a GLM-5.3 node later while applications keep calling the same router.