Best Local LLM for Home Assistant (2026): Model Picks by Hardware, Ollama Set-up and Local Voice

Quick picks by hardware

Integration behaviour comes from Home Assistant’s documentation as published for release 2026.11.1, and model sizes from Ollama’s library and the vendors’ model cards, all read on 14 November 2026. Sizes are download sizes; memory in use is higher once context is added.

How Home Assistant uses a local LLM

Home Assistant’s voice feature is called Assist. A request passes through a pipeline with up to four stages:

1. Wake word. A satellite device or phone notices the trigger phrase.
2. Speech-to-text. Audio becomes a sentence.
3. Conversation agent. Something decides what the sentence means and what to do.
4. Text-to-speech. The answer is spoken back.

The conversation agent is where a language model fits. The default agent, called Home Assistant, matches sentences against built-in patterns: fast, predictable and limited to phrasings it knows. An LLM agent accepts free wording (“it’s a bit dark in here”) and can answer questions about your devices.

The core integrations that provide a local agent, as of release 2026.11:

Outside core, the Home LLM custom integration by acon96 (v0.5.1, 10 November 2026, for Home Assistant 2026.5.0 or newer) can run a model inside Home Assistant with a built-in llama.cpp back end, and publishes small models fine-tuned for device control, including Home-Llama-3.2-3B and Home-FunctionGemma-270m.

Prefer handling commands locally

Each assistant that uses an AI agent has an option, in its three-dot menu, to prefer handling commands locally. Home Assistant’s documentation explains the trade-off: with the option selected, any request the built-in agent can answer is answered by it, quickly and without the model; with it deselected, everything goes through the model. With a local model, keep it on. “Turn off the kitchen light” never reaches the GPU, and the model is saved for requests the patterns cannot handle.

Home Assistant Ollama set-up

1. Run Ollama where the GPU is. Ollama binds to 127.0.0.1 on port 11434 by default. To let Home Assistant reach it from another machine, set the OLLAMA_HOST environment variable to an address on your network, as Ollama’s FAQ describes.

2. Pull a model that supports tools.

   ollama pull qwen3.5:9b

3. Add the integration. In Home Assistant, open Settings > Devices & services, select Add Integration, choose Ollama and enter the server URL, for example http://192.168.1.50:11434. Then pick the model; Home Assistant can download it during set-up if it is not present.

4. Allow device control. Open the integration’s options (the cogwheel) and under AI tools select Device control. The documentation calls controlling Home Assistant with Ollama an experimental feature.

5. Create the assistant. Go to Settings > Voice assistants, select Add assistant, and choose the Ollama agent under Conversation agent. Speech-to-text and text-to-speech are set separately on the same screen.

6. Test in text first. Use Start a conversation from the assistant’s menu before you involve a microphone.

The options that matter for a local model:

The API key field is optional and only needed if you put Ollama behind an authenticating proxy.

Using LM Studio, llama.cpp or vLLM

The llama.cpp integration takes a base URL ending in /v1 and an optional API key, then asks which model to use. The documentation gives typical addresses: http://localhost:8080/v1 for llama.cpp, port 8000 for llama-cpp-python and vLLM, and port 1234 for LM Studio, with Serve on Local Network on if Home Assistant runs on a different device. Current llama-server builds listen on port 9931 unless you pass --port, so match whatever you started. It offers the same Device control tools as the Ollama integration. Our comparison of vLLM, Ollama and llama.cpp explains when each server makes sense.

What a model needs to control devices

Tool calling. Home Assistant offers the model a set of tools for device control, and the Ollama documentation is explicit that only models that support tools may control Home Assistant. In Ollama’s library, look for the “tools” label on the model page. A model without it can chat but cannot switch anything.

A short entity list. Assist can see and control only the entities you expose. For local models the Ollama page recommends exposing fewer than 25 and warns that smaller models are more likely to make mistakes. The best-practices page gives the reason: with an LLM agent, more exposed entities mean a larger context for every request.

Know what is exposed for you. With Expose new entities on, which is the default, Home Assistant automatically exposes new lights, switches, fans, covers, climate devices, media players, scenes and some sensors. Locks, alarm panels and scripts are never exposed automatically. Garage doors and gates count as covers, so they are. Review the Expose tab under Settings > Voice assistants before you hand control to a model, and remove anything you would not want opened by a misheard sentence.

Good names. The model works from entity names, areas and aliases. “Living room lamp” in the area “Living room” is easy; a device called by its factory name is not.

Two agents for small models. The documentation notes that smaller models may not hold a conversation reliably once tools are selected. Its workaround is to add the Ollama integration twice with the same model: one entry with no AI tools for conversation, one with Device control for commands.

Home Assistant and MCP

Home Assistant has two Model Context Protocol integrations, both in core since 2025.2, and they work in opposite directions:

• Model Context Protocol Client gives your conversation agent extra tools from an MCP server that Home Assistant can reach by URL. A server that only runs over stdio needs a proxy in front of it, and the agent must be one that can use tools, which the documentation says includes Ollama.
• Model Context Protocol (the server) lets another AI app control Home Assistant. Release 2026.10 added a one-click card for it under Settings > System > AI; per the release notes, a new set-up exposes everything in Home Assistant and allows only administrator accounts to connect, and both can be changed afterwards.

With a small local model, add client tools sparingly. Every tool is more context and one more thing to choose wrongly, and a model that controls your home should not be able to reach much else.

Model picks by hardware tier

The Ollama models below carry the tools label in Ollama’s library and an Apache 2.0 licence on their model cards. Benchmarks are the vendors’ own. For a wider survey of compact models, see our guide to small language models, and for agent benchmarks in depth, the local tool-calling model guide.

Mini PC or CPU only

Without a GPU, every token is slow, so keep the model small and the job smaller.

• Qwen3.5-4B (qwen3.5:4b, 3.3 to 4.0 GB). Qwen’s card reports 50.3 on BFCL-V4, against 66.1 for the 9B model, so expect more wrong or malformed calls.
• Qwen3.5-2B (qwen3.5:2b, 2.7 to 3.1 GB) when memory is tight.
• Home LLM’s fine-tuned models, such as Home-Llama-3.2-3B, through that custom integration rather than Ollama’s library. Its README says they run on a Raspberry Pi and other modest hardware; check each model’s own licence.

On this tier, expose a handful of entities, keep local handling on, and use Speech-to-Phrase for speech. Treat the model as a fallback for unusual phrasing. Home Assistant publishes no response times for language models on CPUs, and we found no vendor figure to quote.

8 to 12 GB GPU

This is the practical entry point for an LLM agent that feels responsive.

• Qwen3.5-9B (qwen3.5:9b, 6.6 to 7.6 GB) is the first choice at 8 GB. The card reports 66.1 on BFCL-V4 and 79.1 on TAU2-Bench, and the model has a 262,144-token native window, far more than Home Assistant’s 8K default uses.
• Ministral 3 8B (ministral-3:8b, 6.0 GB) is a slightly smaller alternative with tool support.
• Gemma 4 12B (gemma4:12b, 7.7 to 8.0 GB) suits a 12 GB card. Google’s card lists native function calling and a 256K window.

At 8 GB a 6.6 GB model leaves little memory for context. Check the fit in the VRAM calculator, and run ollama ps to confirm the model reports 100% GPU; a model split between GPU and CPU answers much more slowly.

16 to 24 GB GPU

• gpt-oss-20b (gpt-oss:20b, 14 GB) is the pick for a 16 GB card. OpenAI’s card says it runs within 16 GB of memory and describes 21B parameters with 3.6B active per token and native function calling.
• Qwen3.8-27B (qwen3.8:27b, 18 GB) and Gemma 4 26B A4B (gemma4:26b, 16 to 19 GB) fit 24 GB. Gemma’s card gives 25.2B total parameters with 3.8B active.

Bigger models misread fewer requests and cope with more entities, which is the reason to buy the larger card. If you are choosing hardware, our GPU buying guide for local models and model picks by VRAM cover the options.

Local voice assistant: speech-to-text, text-to-speech and wake word

The model is one stage. The other three decide whether the assistant is pleasant to use, and Home Assistant documents local options for each. They run as apps on Home Assistant OS and connect through the Wyoming integration; a Wyoming service can also run on another machine.

Piper is the voice Home Assistant documents, not the only local one. Gradium’s Phonon server also answers over Wyoming and has a Home Assistant app; our comparison of local text-to-speech models covers it and the alternatives.

The choice of speech-to-text is tied to the choice of agent. Speech-to-Phrase recognises only sentences the built-in agent understands, so it suits a setup without an LLM, or one where the LLM is rarely needed. Home Assistant’s guide recommends Whisper when you have powerful hardware and plan to pair the assistant with an LLM-based agent, because free wording needs open transcription. Our guide to self-hosting Whisper covers model sizes and servers, and the overview of voice AI on your own servers covers the business version of the same pipeline.

Voice hardware

• Home Assistant Voice Preview Edition. Launched in December 2024 at a recommended price of $69 or €59. It has a dual-microphone array with an XMOS audio chip, a physical mute switch that cuts power to the microphones, a 3.5 mm output and a Grove port, and its software, firmware and hardware are open.
• Phones and tablets. The Home Assistant apps for Android and Apple devices can talk to Assist.
• ESP32 boards. Home Assistant has tutorials for the ESP32-S3-BOX and the ATOM Echo.

Satellites stream audio to Home Assistant for wake-word checks. The docs say a Raspberry Pi 4 can serve five streaming satellites at once.

Latency: what the documentation supports

Home Assistant publishes the speech figures in the table above. It publishes nothing for the language model stage, and neither do the model vendors for this workload, so we give no numbers. What the documentation does support is where the time goes:

• Skip the model when you can. With local handling preferred, routine commands never wait for it.
• Shorten the prompt. The exposed-entity list is sent with each request. Fewer entities means less to process.
• Keep the model loaded. The default keep-alive of -1 avoids reloads.
• Leave thinking off. The integration’s own description says thinking makes the AI take longer to respond.
• Stay on the GPU. A model that spills into system memory slows sharply.

Measure on your own hardware with a stopwatch and the same five sentences each time.

Limits and safety

• It is experimental. Home Assistant labels LLM device control through Ollama that way. Models mishear, pick the wrong entity and occasionally claim an action they did not take.
• Decide what a wrong call may cost. A light switched in the wrong room is an annoyance. A garage door or heater is not. Unexpose anything in the second group.
• Local means local only if every stage is. A local model with cloud speech still sends audio out. Home Assistant’s paid cloud service offers hosted speech as an alternative; choosing it changes the privacy picture.
• Do not expose Ollama to the internet. Bind it to your home network, not to a public address.

How VDF AI fits

Home automation is not VDF AI’s market. VDF AI does not run, host, resell or integrate with Home Assistant or any of the voice hardware above, and nothing in this guide depends on it.

The connection is the skill, not the product. Choosing a tool-calling model, limiting what it can touch and deciding which actions need a person are the same questions an organisation faces with AI agents at work, where they also need approved model lists, access by role and an audit record. That is what VDF AI provides for businesses; VDF AI Router is the part that governs which on-premises or cloud model each request may use.