Best Local LLM for Writing (2026): Fiction, Long-Form Prose and Interactive Stories

Best local LLM for writing: quick picks by memory

We built this list on 14 November 2026 from Hugging Face model cards, the GGUF files that llama.cpp-based apps load, and the EQ-Bench Creative Writing v3 leaderboard, which is the one public writing evaluation we found that already includes this autumn’s models. File sizes are decimal gigabytes from the repositories named in the sources.

For general model choice by graphics card, without the writing lens, use our guide to local models by VRAM.

What the creative writing benchmark says

EQ-Bench Creative Writing v3 gives each model 32 writing prompts, three times each, at temperature 0.7 and min-p 0.1. Since 25 October 2026 a rubric pass is graded by GLM 5.3, and the Elo ranking comes from pairwise matchups judged by Claude Opus 5. The leaderboard also reports a slop score, the rate of words and phrases that language models overuse, and a repetition score. Lower is better for both.

Of the 146 models listed when we checked, these are the ones a single desktop can run:

Three things stand out. Size is not the whole story: gpt-oss-120b, a strong reasoning model, scores below the 4B-class Gemma 4 E4B on prose. The gap between the 12B and 27B tiers is large, about 380 Elo points between Gemma 4 12B and Qwen3.8-27B. And the open models that beat everything above, such as Kimi K3.1 Pro at 2,173 and GLM-5.3 at 2,076, are data-centre checkpoints, as our Kimi K3.1 Pro hardware guide explains.

Treat these numbers as a shortlist. The judges are language models, the pieces are short, and the test does not measure whether chapter 30 still agrees with chapter 3. EQ-Bench’s separate long-form leaderboard does test eight consecutive chapters, scored out of 100: there Hemmingway-1 reaches 74.2, Gemma 4 31B 56.5 and Qwen3.8-27B 53.3, while Muse Glimmer and Gemma 4 E4B are not yet listed.

8 GB: drafting help, not a co-author

At this size, expect a capable assistant for scenes, dialogue passes and line edits, with more stock phrasing than larger models.

• Gemma 4 E4B (8B parameters with embeddings, Apache 2.0) is 4.98 GB at Q4_K_M, has a 128K-token window and is one of the smallest models with a current creative score.
• Qwen3.5-9B (9.7B, Apache 2.0) is 5.68 GB at Q4_K_M with a native 262,144-token window. Qwen advises keeping at least 128K of context when its thinking mode is on, which an 8 GB card cannot hold, so switch thinking off for drafting.

Smaller still, and you are choosing among the models in our small language model guide, which are better at rewriting a paragraph than inventing one.

12–16 GB: Gemma 4 12B and its style tune

Gemma 4 12B is the natural fit: 7.12 GB at Q4_K_M, 12.67 GB at Q8_0, a 256K window and Apache 2.0. On 12 GB, run the 4-bit file and spend the rest on context; on 16 GB, the 8-bit file leaves room for a long chapter.

Gemma-4-12B-StyleTune (Apache 2.0, 7.95 GB at Q4_K_M) is the same model with one tensor retrained on narrative text. Its author, Gryphe, trained only the output projection layer, the last step before a token is chosen, and left every other weight untouched. On the 31B version the author measured 60% fewer clichés per 100 words across 200 prompts. That is a self-reported figure on the author’s own test, but the method is cheap to try: swap one file and compare.

A 16 GB card can also load Mistral Small 3.2 24B at Q3_K_M (11.47 GB) or Gemma 4 26B A4B at UD-Q3_K_M (12.73 GB). Both score close to Gemma 4 12B on the leaderboard, so the heavier quantisation buys little for prose.

24 GB: the best local LLMs for writing fiction

Four models are worth a writer’s time here, and all are about 17 to 19 GB at 4-bit.

• Muse Glimmer 30B (Meta, August 2026, Apache 2.0 with a separate usage policy) was built for agent work, yet it is the highest-ranked permissively licensed desktop model on the creative leaderboard. Meta’s 17 GB build targets 24 GB cards, the window is 131,072 tokens or more, and Meta reports 74.9 tokens a second on an RTX 5090 before speculative decoding. It is in Ollama as muse-glimmer:30b. Our explainer on the missing Llama 5 covers how it relates to Meta’s Llama line.
• Qwen3.8-27B (Alibaba, August 2026, Apache 2.0) is 16.56 GB as Unsloth’s UD-Q4_K_M build, with a native 262,144-token window. It has the lowest slop score of the Apache-licensed models in the table. Our Qwen 3.8 hardware guide lists every build.
• Gemma 4 31B (18.32 GB at Q4_K_M) scores lower, but Gemma-4-31B-StyleTune (19.48 GB) exists for exactly that reason, and both keep the 256K window.
• Hemmingway-1 and Hemmingway-1.5 are fine-tunes of Qwen3.8-27B aimed at natural-sounding prose. Hemmingway-1 is the best-ranked model of this size, 17th overall. Both are CC BY-NC 4.0: fine for a private draft, not for a book you sell.

One caution on fine-tunes. Qwen3.8-27B-antislop was trained with reinforcement learning to write less like a language model, and its card reports large gains on the maker’s own rubric. On the independent leaderboard it ranks below its base model, with a higher slop score. A fine-tune’s card tells you what it was optimised for, not how it will handle your genre.

48 GB and up, and Macs with unified memory

More memory does not currently buy a better open prose model. The next checkpoints with published creative scores are far larger: Inkling-Small has 266 billion parameters and scores below Qwen3.8-27B, and GLM-5.3 has 753 billion.

What 48 GB, two 24 GB cards or a 64 to 128 GB Mac does buy is precision and context for the same 27 to 31B models:

• Qwen3.8-27B at Q8_0 is 29.05 GB, and Gemma 4 31B at Q8_0 is 32.64 GB.
• Meta lists 64 GB of VRAM for Muse Glimmer at full precision and 32 GB for its dynamic quantised build.
• The rest goes to the context window, so a whole manuscript can sit in memory.

On Apple Silicon, Meta measured 23.7 tokens a second on an M4 Max and 26.6 on an M5 Max for Muse Glimmer’s 17 GB build, without speculative decoding. Ollama publishes MLX builds of Gemma 4, Qwen3.8 and Muse Glimmer. Our Mac model guide maps models to each memory size.

Context length for long manuscripts

A novel does not fit in a default window. Three settings matter.

1. The model’s limit. Qwen3.8-27B and Qwen3.5-9B: 262,144 tokens. Gemma 4 12B, 26B and 31B: 256K. Gemma 4 E4B: 128K. Muse Glimmer: 131,072 or more.
2. The runtime’s default. Ollama picks a 4K window below 24 GiB of VRAM, 32K from 24 to 48 GiB and 256K above that. Raise it in the app’s settings or start the server with OLLAMA_CONTEXT_LENGTH=64000 ollama serve. KoboldCpp takes --contextsize.
3. Your memory. Every token of context costs memory on top of the weights. The VRAM calculator shows how much window a given card leaves.

When the manuscript is longer than the window you can afford, keep a running synopsis, a character sheet and the last few chapters in context, and store everything else as reference notes. KoboldCpp’s memory, author’s note and world info fields do this by hand, and its ContextShift feature drops the oldest tokens without reprocessing the whole prompt. For reference material at scale, the trade-off between a huge window and retrieval is covered in our long context versus RAG comparison.

Sampler settings: temperature, min-p, DRY and XTC

Samplers decide which of the model’s candidate tokens is chosen. For prose, they matter as much as the model. These are the documented defaults in llama.cpp’s server, which Ollama, KoboldCpp and LM Studio build on in different ways:

KoboldCpp’s documentation describes DRY as an anti-repetition sampler to use instead of, or with, repetition penalty, and suggests trying XTC at a threshold of 0.15 and a probability of 0.5.

A sensible starting point for fiction, drawn from the sources above: temperature 0.7 to 1.0, min-p 0.05 to 0.1, DRY on with the default base, and repeat penalty left off. Use the model maker’s values as the baseline: temperature 1.0, top-p 0.95 and top-k 64 for Gemma 4, and temperature 0.7 with top-p 0.8 for Qwen3.8 with thinking off. Change one setting at a time and regenerate the same passage to hear the difference.

Thinking mode is a separate decision. Qwen3.8 thinks by default and keeps earlier reasoning in the conversation, which is useful for plotting and continuity checks but spends context. For plain drafting, switch it off.

Instruction models or creative fine-tunes?

An instruction-tuned base model, such as Qwen3.8-27B or Gemma 4, follows directions well: “rewrite this scene in close third person, past tense, under 600 words” gets what it asks for. Its weakness is voice. Left alone, it reaches for the same phrases, which is what the slop score counts.

Creative fine-tunes attack that weakness in three ways, each with a current example:

• Retrain the output layer only (the Gemma 4 StyleTunes): style changes, knowledge and instruction-following stay.
• Reinforcement learning against writing rubrics (Qwen3.8-27B-antislop): optimises for the judge it was trained with.
• Full fine-tune on human-sounding text (Hemmingway-1): the largest measured gain, under a restrictive licence.

For a manuscript you intend to sell, start with an Apache 2.0 base model and a careful system prompt that names your tense, point of view and banned phrases. Add a fine-tune only after checking its licence and comparing it on your own pages.

Writing front ends: KoboldCpp, SillyTavern, LM Studio and Open WebUI

KoboldCpp is the shortest path for fiction, because the story tools and the full sampler set are in one download. SillyTavern describes itself as a locally installed interface, but it can connect to cloud APIs, so check which connection is active. Open WebUI’s branding terms are covered in our Open WebUI licence explainer.

Best local LLM for roleplay and interactive fiction

Roleplay, in the sense writers and game designers use it, is collaborative fiction: the model voices one or more characters and the author steers the scene turn by turn. It is used to test dialogue, rehearse a character’s voice, run a tabletop session or prototype a branching story.

The technical needs differ from drafting. Sessions run for hundreds of turns, so context management matters more than raw quality, and a model must hold a persona without drifting. Three tools do most of the work: a character card that fixes voice and background, a lorebook or world info entry that is inserted only when its keywords appear, and a context-shifting back end. Qwen3.8-27B and Muse Glimmer 30B handle long sessions within a 24 GB budget; Gemma 4 12B is the practical choice below that. The Gemma 4 StyleTunes are the fine-tunes here that were measured on roleplay prompts, 200 of them, by their author.

Keep the same licence discipline as for prose. Many community fine-tunes for this use are merges with unclear terms, and anything you plan to publish or ship in a game should rest on a model whose licence you can cite.

Privacy for unpublished manuscripts and licences for publishing

An unpublished manuscript is a trade secret until it is sold, and often covered by a contract. A local model keeps it on your disk: nothing is uploaded, logged by a provider or retained for training. That holds only while every part is local, so check the front end’s connection and keep saves in an encrypted, backed-up folder.

On the publishing side:

• Model licence. Apache 2.0 (Qwen3.8, Qwen3.5, Gemma 4, Mistral Small 3.2, the StyleTunes) allows commercial use. Muse Glimmer adds Meta’s usage policy. CC BY-NC 4.0 (Hemmingway-1 and 1.5) does not allow commercial use.
• Retailer rules. Amazon KDP requires authors to declare AI-generated text, images or translations when publishing, and does not require disclosure of AI-assisted content, such as editing or brainstorming.
• Copyright. The US Copyright Office’s January 2025 report concluded that generative AI output can be protected only where a human author has determined sufficient expressive elements, and that prompts alone do not meet that bar. Other countries differ.

None of this is legal advice. If a contract or a registration depends on it, ask a publishing lawyer.

How VDF AI fits

VDF AI is not a fiction-writing tool. It does not run, host, resell or integrate with the fine-tunes or writing front ends in this guide, and it does not ship or resell any of the models. Everything above describes one author on one machine.

Organisations that handle other people’s unpublished text, such as publishers, agencies, legal teams and research groups, face a different question: who hosts the model, which models are approved, who can see which documents and what record is kept. VDF AI runs on your own servers, in an air-gapped network or in its cloud. VDF AI Chat gives staff a governed chat workspace with role-based agent access and a per-turn audit record, and VDF AI Router registers on-premises model endpoints, with an air-gapped mode that keeps every request on local models. To discuss a deployment, contact us.