Local LLM
A language model hosted on the organisation's own infrastructure — on-premise servers, edge devices, or private cloud.
What is Local LLM?
Local LLMs keep prompts and outputs within the enterprise perimeter, eliminating API egress risk and giving predictable per-token costs. Frameworks like Ollama, vLLM, and llama.cpp make local hosting practical for 7B–70B parameter models. See Local LLM, Benefits of Running LLMs Locally, and Local AI Enterprise Playbook.
Why it matters for on-premise & regulated AI
Local LLMs are the foundation of every on-premise AI architecture: without a model you can run on your own GPUs, everything else — private RAG, governed agents, air-gapped deployment — is impossible. Open-weight models (Llama, Mistral, Qwen families) now cover the majority of enterprise tasks, and routing lets you reserve external frontier calls for the few workloads that genuinely need them, if policy allows them at all.
Read the full guide: Local LLM — in-depth article →
Related terms
Putting Local LLM to work?
VDF AI runs governed AI agents on your own infrastructure — on-premises, sovereign cloud, or air-gapped. Book a working session to map the architecture.
Talk to VDF AI