AI Infrastructure

Local LLM on Android and iPhone (2026): Apps That Work, Models by Phone RAM, and Reaching a Home Server Privately

How to run a language model fully on a phone in November 2026: the Android and iOS apps that are still maintained and which are open source, the models that fit 4 to 6, 8 and 12 to 16 GB of phone RAM with verified file sizes, measured speeds, what Apple and Google already ship on-device, battery and heat, and how to reach a home or office server from the phone over a private network without exposing it.

To run a local LLM on Android or iPhone, install PocketPal AI or Google AI Edge Gallery, download a model of 1 to 4 billion parameters and chat with the phone offline. Gemma 4 E2B, Qwen3.5-4B and LFM3-1.7B fit most current phones. For larger models, keep them on a computer at home or work and reach it from the phone over a private network.

Phones now have enough memory and a fast enough GPU to run a small language model with no server. That gives you an assistant that works in aeroplane mode and keeps prompts on the device. It also has hard limits: memory, heat and battery cap the model at a few billion parameters.

This guide covers both routes: models that run on the phone itself, and a phone used as a private window onto a bigger model you host elsewhere. App versions come from each project’s GitHub releases and store listings, model sizes from the files on Hugging Face, and platform behaviour from Apple, Google, Ollama and Tailscale documentation, all looked up on 14 and 15 November 2026.

Quick picks: how to run an LLM on a phone

You wantUseModel to start with
The easiest start on AndroidGoogle AI Edge GalleryGemma 4 E2B
Any GGUF model, Android or iPhonePocketPal AIQwen3.5-4B or LFM3-1.7B
Apple’s MLX runtime on iPhoneLocallyGemma 4 or Qwen3.5 in MLX format
A command line on AndroidTermux with llama.cppLFM3-1.7B
A bigger model than the phone can holdA home or office server, reached over Tailscale or LM LinkWhatever the server runs

Local LLM apps for Android and iPhone that are still maintained

Phone LLM apps come and go, and several early ones have stopped receiving updates. These were active when we looked:

AppPlatformsLicenceLatest release we sawRuns models with
PocketPal AIAndroid, iOS 15.1+MIT, open sourcev1.23.0, 10 November 2026llama.cpp (GGUF)
Google AI Edge GalleryAndroid 12+, iOS 17+Apache 2.0, open source1.0.23, 4 November 2026Google’s LiteRT runtime
LocallyiOS and iPadOS 18.1+Free; published by LM Studio’s developer, Element Labs1.67.0, 4 NovemberApple MLX
SmolChatAndroidApache 2.0, open sourcev17, 1 November 2026llama.cpp (GGUF)
ChatterUIAndroidAGPL-3.0, open sourcev0.10.0, 22 September 2026On-device models, or remote APIs
MaidAndroidMIT, open sourcev3.0.0, 10 March 2026llama.cpp locally, or remote servers such as Ollama
EnchantediOS, macOSApache 2.0, open sourcev1.8.2, 15 June 2026Nothing on-device: a client for an Ollama server

Notes on each:

  • PocketPal AI searches and downloads GGUF files from Hugging Face inside the app, including gated ones. Its README lists Metal on iOS, OpenCL on Adreno GPUs and Qualcomm’s Hexagon NPU on Android, a built-in benchmark, on-device speech recognition and custom assistants it calls Pals. The App Store lists it as free with in-app purchases; version 1.23.0 added a paid local API server.
  • Google AI Edge Gallery is Google’s showcase for on-device models: chat, questions about photos, audio transcription, a prompt lab and agent skills. Its README says no internet is required once a model is on the device.
  • Locally is the iPhone companion to LM Studio. Its 4 November release notes add Liquid’s LFM 3 models, and its App Store privacy label reads “Data Not Collected”. We found no public source repository for it.

Two names to leave off your list. MLC Chat is still on the App Store, but its last version there is 1.6 from 27 September 2024; the MLC LLM project itself remains active as a toolkit for developers building their own apps. Ollama has no phone app: its download page offers macOS, Linux and Windows only, so a phone can only be a client to it.

Run an LLM on Android

With an app. The steps are the same in PocketPal AI and AI Edge Gallery:

  1. Install the app from Google Play. AI Edge Gallery also publishes an APK with each GitHub release.
  2. On Wi-Fi, open the model list and download one model. Start small: Gemma 4 E2B in AI Edge Gallery, or a 4-bit GGUF of Qwen3.5-4B in PocketPal.
  3. Load the model and send a test prompt. Then switch on aeroplane mode and send another to confirm nothing depends on the network.
  4. In PocketPal, run the benchmark once. It reports tokens per second and memory use, which tells you whether a larger model is realistic.

With Termux and llama.cpp. Termux’s package repository carries llama.cpp, with optional Vulkan and OpenCL backends as separate packages. Install it and start the server on a small model:

pkg update && pkg install llama-cpp
llama-server -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M -c 4096

The -hf flag downloads the named quantization from Hugging Face. Open http://127.0.0.1:9931 in the phone’s browser to chat; current llama.cpp builds, including Termux’s v1.0.0 package, listen on port 9931 by default rather than the older 8080, as our llama-server guide explains. This route gives you the newest llama.cpp features and an OpenAI-style API on the phone, at the cost of typing on a touch keyboard.

Run an LLM on iPhone

On iOS there are three runtimes to choose from.

  • GGUF through llama.cpp. PocketPal AI runs the same GGUF files people use on PCs, accelerated by Metal. It needs iOS 15.1 or later.
  • MLX. Locally uses Apple’s own framework for Apple silicon and needs iOS 18.1. MLX conversions of multimodal models are often larger than the text-only GGUF: the 4-bit MLX build of Gemma 4 E2B is 3.55 GB.
  • LiteRT. Google AI Edge Gallery brings Gemma 4 to iOS 17 or later with the same task format as Android.

The steps mirror Android: install from the App Store, download one model on Wi-Fi, test in aeroplane mode. iPhone settings do not show how much RAM the device has, so go by file size and by which models the app offers for your phone. If an app closes when you load a model, the file is too large for that device; pick the next size down.

Which models fit your phone’s RAM

A phone shares its memory with the system and every other app, and both Android and iOS close apps that take too much. Our rule of thumb is to keep the model file under about a third of the phone’s RAM. Sizes below are the actual downloads on Hugging Face.

Phone RAMModels that fitFile sizeLicence
4 to 6 GBQwen3.5-0.8B (Q4_K_M)0.53 GBApache 2.0
LFM2.5-1.2B (Q4_K_M)0.73 GBLFM Open License
Llama 3.2 1B (Q4_K_M)0.81 GBLlama 3.2 Community License
LFM3-1.7B (Q4_K_M)1.10 GBLFM Open License v1.0
Qwen3.5-2B (Q4_K_M)1.27 GBApache 2.0
8 GBSmolLM3-3B (Q4_K_M)1.92 GBApache 2.0
Llama 3.2 3B (Q4_K_M)2.02 GBLlama 3.2 Community License
Phi-4-mini (Q4_K_M)2.49 GBMIT
Gemma 4 E2B (LiteRT)2.59 GBApache 2.0
Qwen3.5-4B (Q4_K_M)2.71 GBApache 2.0
12 to 16 GBGemma 4 E4B (LiteRT)3.66 GBApache 2.0
Gemma 4 E4B (GGUF Q4_0)4.59 GBApache 2.0
Qwen3.5-9B (Q4_K_M), 16 GB phones only5.63 GBApache 2.0

How to choose within a tier:

  • Gemma 4 E2B and E4B are the multimodal picks: text, images and audio in one model, built by Google for phones and laptops. Google’s card says the on-device build supports up to a 32K context.
  • LFM3-1.7B is the newest small model here. Liquid AI released it in late October 2026, gives it a 256K context and tool calling, and says a 4-bit copy takes about 1 GB, “so it fits on any phone or laptop”. Liquid also says it is not recommended for agentic coding or complex multi-step reasoning.
  • Qwen3.5-4B is the strongest general model that fits an 8 GB phone and reads images.
  • Phi-4-mini and SmolLM3-3B are text-only and permissively licensed, which matters if you plan to build on them.

Stay at 4-bit. Lower quantization saves little on files this small and costs accuracy, and 8-bit doubles the size for a gain you will not notice on a phone. For how these models compare on tool calling, retrieval and licences, see our task-by-task comparison of small models.

How fast is a local LLM on a phone

Google publishes device measurements on its LiteRT card for Gemma 4 E2B, using a 1,024-token prompt, a 256-token reply and a 2,048-token window:

DeviceBackendPrompt readingGenerationTime to first token
Samsung S26 UltraGPU3,808 tokens/s52.1 tokens/s0.3 s
Samsung S26 UltraCPU557 tokens/s46.9 tokens/s1.8 s
iPhone 17 ProGPU2,878 tokens/s56.5 tokens/s0.3 s
iPhone 17 ProCPU532 tokens/s25.0 tokens/s1.9 s

Three things to take from it. Generation speed on a flagship phone is comfortably faster than reading speed for a 2B-class model. The GPU matters most for reading the prompt, where it is five to seven times faster than the CPU, so use the GPU backend when you paste long text. And these are top-end phones: a mid-range device, or a model twice the size, will be proportionally slower. Run the app’s own benchmark rather than trusting anyone’s table, ours included.

What Apple and Google already ship on-device

Both platforms include a small built-in model. You cannot choose or replace it, but apps can use it, and it explains why some features work offline.

Apple. Apple says Apple Intelligence combines on-device processing with Private Cloud Compute, which lets it draw on larger server-based models without making your data accessible to Apple. For developers, the Foundation Models framework, available from iOS 26, gives any app access to the on-device model; Apple says features built on it work offline and cost nothing per request. It requires a device that supports Apple Intelligence.

Google. On Android, Gemini Nano runs inside a system service called AICore. Google’s developer documentation says it works without a network connection or sending data to the cloud, that AICore has no direct internet access, and that it does not store a record of inputs or outputs after processing. Apps reach it through the ML Kit GenAI APIs for summarising, proofreading, rewriting, image description and speech. Google’s Gemma 4 card adds that Gemma 4 is available on supported Android devices through AICore as Gemini Nano.

The difference from the apps above is control. With a built-in model, the platform decides which model runs and when a request goes to the cloud tier. With PocketPal or AI Edge Gallery, you pick the model file and can verify the phone is offline.

Battery, heat and storage

Running a model keeps the phone’s GPU or CPU at high load for as long as it generates text. Three practical effects follow:

  • Heat and throttling. During long, continuous generation a phone warms up and may slow itself down. Short questions are fine; long sessions are better on a charger and out of a case.
  • Battery. Generation is among the heaviest things a phone does. Treat a long chat like gaming, not like messaging.
  • Storage and memory. Each model is a 1 to 5 GB download, and two or three add up quickly. In Google’s S26 Ultra test the GPU backend held 676 MB in system memory against 1,733 MB for the CPU backend, another reason to prefer the GPU.

Neither Apple nor Google publishes battery figures for third-party models, and we have not seen a controlled measurement worth quoting. Benchmark on your own device.

The alternative: reach a home or office server from the phone

A phone tops out around 4B parameters. A desktop with a graphics card, or one of the machines in our mini PC guide, runs models ten to thirty times larger. The phone then becomes a screen, and the model stays on hardware you control.

LM Link. LM Studio’s built-in option. Its documentation describes links as end-to-end encrypted networks between your own devices, made in partnership with Tailscale, and says that on iPhone you use the Locally app. Android is not mentioned.

Tailscale with your own server. For Ollama or any other server:

  1. Install Tailscale on the server and on the phone and sign both into the same private network, which Tailscale calls a tailnet.
  2. Leave the model server on its default local address and share a chat front end inside the tailnet. Tailscale Serve does this for a web interface listening on a local port:
tailscale serve 3000
  1. Open the address Tailscale prints, in the form device-name.tailnet-name.ts.net, in the phone’s browser. Our comparison of self-hosted chat interfaces covers front ends that add user accounts.
  2. If you prefer a native client such as Enchanted or Maid, the Ollama API itself has to be reachable. Bind it to the server’s Tailscale address rather than to every interface:
OLLAMA_HOST=<tailscale-ip>:11434 ollama serve

Security caveats

  • Never make the model server public. Ollama binds to 127.0.0.1 on port 11434 by default, and its API has no login of its own; our Ollama security review lists what has happened to servers left open. Do not forward the port on your router.
  • Serve is private, Funnel is public. Tailscale’s documentation says Serve routes traffic from devices on your tailnet, while Funnel shares a service over the public internet. Use Serve.
  • Membership is the password. Anyone holding a device on the tailnet can reach what you share. Tailscale notes that access control rules apply to Serve like any other service, so limit the phone to the one port it needs, and remove a lost phone from the tailnet at once.
  • The privacy boundary moves. Prompts now leave the phone. The set-up is as private as the server, its disk and whoever administers it.

Company phones: on-device is private, policy still applies

For a business, on-device models remove one risk: prompts are not sent to a third-party AI service. They leave others in place.

  • The app matters. Several of these apps can also connect to cloud providers or remote servers. On a managed phone, decide which apps are approved and which features are allowed.
  • Models are downloads from the internet. Someone should decide which model files and publishers are acceptable, and check each licence for commercial use.
  • Small models make more mistakes. A 2B model summarising a contract is not equivalent to the assistant your organisation evaluated.
  • Chats are stored on the phone. History, pasted documents and backups fall under the same device rules as email.
  • A home server is unmanaged hardware. Company data sent from a work phone to an employee’s own machine has left the organisation, however private the tunnel.

Write these points into policy rather than leaving them to each employee; our AI acceptable use policy template is a starting point, and the zero-trust guide for on-premises AI covers the network side. Where models belong on phones, sites or servers in an organisation is the subject of our article on CPU and NPU inference at the edge.

How VDF AI fits

VDF AI does not run on phones, and it does not run, host, resell or integrate with any of the phone apps or networking tools in this guide. It is a platform for organisations, deployed on their own servers, private cloud or an air-gapped network. The honest bridge is this: one person’s phone model is a personal tool, while a company needs to know who hosts the model, which models are approved, who can use them and what is recorded.

VDF AI Chat is the organisation-hosted assistant, with role-based access per agent and per knowledge source and an append-only audit record for each turn. VDF AI Router registers on-premises model deployments and has an air-gap mode that restricts routing to local models. To discuss a pilot, contact us.

Sources

Looked up on 14 and 15 November 2026.


Need an assistant your whole organisation can use, on infrastructure you control? See VDF AI Chat, or book a demo.

Frequently asked questions

Can you run an LLM locally on a phone?

Yes. Apps such as PocketPal AI and Google AI Edge Gallery download an open-weight model to the phone and run it on the phone's own processor or GPU, with no server involved after the download. Models of 1 to 4 billion parameters work well: Google measured Gemma 4 E2B at about 52 tokens a second on a Samsung S26 Ultra and 56 on an iPhone 17 Pro. Expect short, practical answers rather than the depth of a large cloud model.

What is the best local LLM app for Android?

PocketPal AI is the most flexible choice: it is open source under the MIT licence, was last released on 10 November 2026, downloads GGUF models from Hugging Face inside the app and can use Adreno GPUs and Qualcomm NPUs. Google AI Edge Gallery, Apache 2.0, is the simplest way to run Gemma 4 with image and audio input on Android 12 or later. SmolChat and ChatterUI are further open-source options, and Termux with llama.cpp suits people who prefer a command line.

What is the best local LLM app for iPhone?

Three are worth installing. PocketPal AI is open source, needs iOS 15.1 or later and runs GGUF models through llama.cpp with Metal acceleration. Locally, from LM Studio's developer, uses Apple's MLX framework, needs iOS 18.1 and also connects to LM Studio on your own computer through LM Link. Google AI Edge Gallery runs Gemma 4 on iOS 17 or later. PocketPal and Locally show Data Not Collected in their App Store privacy labels; Google's label lists some data collection, while its project page says inference needs no internet.

Which model should I run on a phone with 8GB RAM?

Choose a model whose file is under about 3 GB. Gemma 4 E2B is 2.59 GB in Google's LiteRT format and accepts text, images and audio. Qwen3.5-4B is 2.71 GB as a 4-bit GGUF, Phi-4-mini is 2.49 GB, Llama 3.2 3B is 2.02 GB and SmolLM3-3B is 1.92 GB. On phones with 4 to 6 GB, drop to LFM3-1.7B at 1.10 GB or Qwen3.5-2B at 1.27 GB. Larger files risk the system closing the app.

Is there an offline AI chatbot app that works without internet?

Yes. Once a model is downloaded, PocketPal AI, Google AI Edge Gallery and Locally generate answers with the phone in aeroplane mode. Google's project page says no internet is required, and PocketPal's says chats leave the device only through features you switch on, such as web search or a remote server. You still need a connection once to download the model, and optional extras in some apps do use the network, so check which features are enabled.

How do I use an Ollama server at home from my phone privately?

Keep the server off the public internet. Ollama listens only on 127.0.0.1 by default and its API has no login, so do not forward port 11434 on your router. Put the phone and the server on a private network such as Tailscale, share the service only inside that network, and limit which devices may reach it. Remember that your prompts now travel to the server, so the set-up is only as private as that machine.

Filed under
local LLMon-device AIopen-weight modelsmobile AIOllamaprivate AI
On-Prem AI

Plan your on-prem AI deployment

Book an architecture call and we will scope a private, on-prem AI deployment for your environment — integrations, hardware, and governance included.

Keep reading