To run a local LLM on Android or iPhone, install PocketPal AI or Google AI Edge Gallery, download a model of 1 to 4 billion parameters and chat with the phone offline. Gemma 4 E2B, Qwen3.5-4B and LFM3-1.7B fit most current phones. For larger models, keep them on a computer at home or work and reach it from the phone over a private network.
Phones now have enough memory and a fast enough GPU to run a small language model with no server. That gives you an assistant that works in aeroplane mode and keeps prompts on the device. It also has hard limits: memory, heat and battery cap the model at a few billion parameters.
This guide covers both routes: models that run on the phone itself, and a phone used as a private window onto a bigger model you host elsewhere. App versions come from each project’s GitHub releases and store listings, model sizes from the files on Hugging Face, and platform behaviour from Apple, Google, Ollama and Tailscale documentation, all looked up on 14 and 15 November 2026.
Quick picks: how to run an LLM on a phone
| You want | Use | Model to start with |
|---|---|---|
| The easiest start on Android | Google AI Edge Gallery | Gemma 4 E2B |
| Any GGUF model, Android or iPhone | PocketPal AI | Qwen3.5-4B or LFM3-1.7B |
| Apple’s MLX runtime on iPhone | Locally | Gemma 4 or Qwen3.5 in MLX format |
| A command line on Android | Termux with llama.cpp | LFM3-1.7B |
| A bigger model than the phone can hold | A home or office server, reached over Tailscale or LM Link | Whatever the server runs |
Local LLM apps for Android and iPhone that are still maintained
Phone LLM apps come and go, and several early ones have stopped receiving updates. These were active when we looked:
| App | Platforms | Licence | Latest release we saw | Runs models with |
|---|---|---|---|---|
| PocketPal AI | Android, iOS 15.1+ | MIT, open source | v1.23.0, 10 November 2026 | llama.cpp (GGUF) |
| Google AI Edge Gallery | Android 12+, iOS 17+ | Apache 2.0, open source | 1.0.23, 4 November 2026 | Google’s LiteRT runtime |
| Locally | iOS and iPadOS 18.1+ | Free; published by LM Studio’s developer, Element Labs | 1.67.0, 4 November | Apple MLX |
| SmolChat | Android | Apache 2.0, open source | v17, 1 November 2026 | llama.cpp (GGUF) |
| ChatterUI | Android | AGPL-3.0, open source | v0.10.0, 22 September 2026 | On-device models, or remote APIs |
| Maid | Android | MIT, open source | v3.0.0, 10 March 2026 | llama.cpp locally, or remote servers such as Ollama |
| Enchanted | iOS, macOS | Apache 2.0, open source | v1.8.2, 15 June 2026 | Nothing on-device: a client for an Ollama server |
Notes on each:
- PocketPal AI searches and downloads GGUF files from Hugging Face inside the app, including gated ones. Its README lists Metal on iOS, OpenCL on Adreno GPUs and Qualcomm’s Hexagon NPU on Android, a built-in benchmark, on-device speech recognition and custom assistants it calls Pals. The App Store lists it as free with in-app purchases; version 1.23.0 added a paid local API server.
- Google AI Edge Gallery is Google’s showcase for on-device models: chat, questions about photos, audio transcription, a prompt lab and agent skills. Its README says no internet is required once a model is on the device.
- Locally is the iPhone companion to LM Studio. Its 4 November release notes add Liquid’s LFM 3 models, and its App Store privacy label reads “Data Not Collected”. We found no public source repository for it.
Two names to leave off your list. MLC Chat is still on the App Store, but its last version there is 1.6 from 27 September 2024; the MLC LLM project itself remains active as a toolkit for developers building their own apps. Ollama has no phone app: its download page offers macOS, Linux and Windows only, so a phone can only be a client to it.
Run an LLM on Android
With an app. The steps are the same in PocketPal AI and AI Edge Gallery:
- Install the app from Google Play. AI Edge Gallery also publishes an APK with each GitHub release.
- On Wi-Fi, open the model list and download one model. Start small: Gemma 4 E2B in AI Edge Gallery, or a 4-bit GGUF of Qwen3.5-4B in PocketPal.
- Load the model and send a test prompt. Then switch on aeroplane mode and send another to confirm nothing depends on the network.
- In PocketPal, run the benchmark once. It reports tokens per second and memory use, which tells you whether a larger model is realistic.
With Termux and llama.cpp. Termux’s package repository carries llama.cpp, with optional Vulkan and OpenCL backends as separate packages. Install it and start the server on a small model:
pkg update && pkg install llama-cpp
llama-server -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M -c 4096
The -hf flag downloads the named quantization from Hugging Face. Open http://127.0.0.1:9931 in the phone’s browser to chat; current llama.cpp builds, including Termux’s v1.0.0 package, listen on port 9931 by default rather than the older 8080, as our llama-server guide explains. This route gives you the newest llama.cpp features and an OpenAI-style API on the phone, at the cost of typing on a touch keyboard.
Run an LLM on iPhone
On iOS there are three runtimes to choose from.
- GGUF through llama.cpp. PocketPal AI runs the same GGUF files people use on PCs, accelerated by Metal. It needs iOS 15.1 or later.
- MLX. Locally uses Apple’s own framework for Apple silicon and needs iOS 18.1. MLX conversions of multimodal models are often larger than the text-only GGUF: the 4-bit MLX build of Gemma 4 E2B is 3.55 GB.
- LiteRT. Google AI Edge Gallery brings Gemma 4 to iOS 17 or later with the same task format as Android.
The steps mirror Android: install from the App Store, download one model on Wi-Fi, test in aeroplane mode. iPhone settings do not show how much RAM the device has, so go by file size and by which models the app offers for your phone. If an app closes when you load a model, the file is too large for that device; pick the next size down.
Which models fit your phone’s RAM
A phone shares its memory with the system and every other app, and both Android and iOS close apps that take too much. Our rule of thumb is to keep the model file under about a third of the phone’s RAM. Sizes below are the actual downloads on Hugging Face.
| Phone RAM | Models that fit | File size | Licence |
|---|---|---|---|
| 4 to 6 GB | Qwen3.5-0.8B (Q4_K_M) | 0.53 GB | Apache 2.0 |
| LFM2.5-1.2B (Q4_K_M) | 0.73 GB | LFM Open License | |
| Llama 3.2 1B (Q4_K_M) | 0.81 GB | Llama 3.2 Community License | |
| LFM3-1.7B (Q4_K_M) | 1.10 GB | LFM Open License v1.0 | |
| Qwen3.5-2B (Q4_K_M) | 1.27 GB | Apache 2.0 | |
| 8 GB | SmolLM3-3B (Q4_K_M) | 1.92 GB | Apache 2.0 |
| Llama 3.2 3B (Q4_K_M) | 2.02 GB | Llama 3.2 Community License | |
| Phi-4-mini (Q4_K_M) | 2.49 GB | MIT | |
| Gemma 4 E2B (LiteRT) | 2.59 GB | Apache 2.0 | |
| Qwen3.5-4B (Q4_K_M) | 2.71 GB | Apache 2.0 | |
| 12 to 16 GB | Gemma 4 E4B (LiteRT) | 3.66 GB | Apache 2.0 |
| Gemma 4 E4B (GGUF Q4_0) | 4.59 GB | Apache 2.0 | |
| Qwen3.5-9B (Q4_K_M), 16 GB phones only | 5.63 GB | Apache 2.0 |
How to choose within a tier:
- Gemma 4 E2B and E4B are the multimodal picks: text, images and audio in one model, built by Google for phones and laptops. Google’s card says the on-device build supports up to a 32K context.
- LFM3-1.7B is the newest small model here. Liquid AI released it in late October 2026, gives it a 256K context and tool calling, and says a 4-bit copy takes about 1 GB, “so it fits on any phone or laptop”. Liquid also says it is not recommended for agentic coding or complex multi-step reasoning.
- Qwen3.5-4B is the strongest general model that fits an 8 GB phone and reads images.
- Phi-4-mini and SmolLM3-3B are text-only and permissively licensed, which matters if you plan to build on them.
Stay at 4-bit. Lower quantization saves little on files this small and costs accuracy, and 8-bit doubles the size for a gain you will not notice on a phone. For how these models compare on tool calling, retrieval and licences, see our task-by-task comparison of small models.
How fast is a local LLM on a phone
Google publishes device measurements on its LiteRT card for Gemma 4 E2B, using a 1,024-token prompt, a 256-token reply and a 2,048-token window:
| Device | Backend | Prompt reading | Generation | Time to first token |
|---|---|---|---|---|
| Samsung S26 Ultra | GPU | 3,808 tokens/s | 52.1 tokens/s | 0.3 s |
| Samsung S26 Ultra | CPU | 557 tokens/s | 46.9 tokens/s | 1.8 s |
| iPhone 17 Pro | GPU | 2,878 tokens/s | 56.5 tokens/s | 0.3 s |
| iPhone 17 Pro | CPU | 532 tokens/s | 25.0 tokens/s | 1.9 s |
Three things to take from it. Generation speed on a flagship phone is comfortably faster than reading speed for a 2B-class model. The GPU matters most for reading the prompt, where it is five to seven times faster than the CPU, so use the GPU backend when you paste long text. And these are top-end phones: a mid-range device, or a model twice the size, will be proportionally slower. Run the app’s own benchmark rather than trusting anyone’s table, ours included.
What Apple and Google already ship on-device
Both platforms include a small built-in model. You cannot choose or replace it, but apps can use it, and it explains why some features work offline.
Apple. Apple says Apple Intelligence combines on-device processing with Private Cloud Compute, which lets it draw on larger server-based models without making your data accessible to Apple. For developers, the Foundation Models framework, available from iOS 26, gives any app access to the on-device model; Apple says features built on it work offline and cost nothing per request. It requires a device that supports Apple Intelligence.
Google. On Android, Gemini Nano runs inside a system service called AICore. Google’s developer documentation says it works without a network connection or sending data to the cloud, that AICore has no direct internet access, and that it does not store a record of inputs or outputs after processing. Apps reach it through the ML Kit GenAI APIs for summarising, proofreading, rewriting, image description and speech. Google’s Gemma 4 card adds that Gemma 4 is available on supported Android devices through AICore as Gemini Nano.
The difference from the apps above is control. With a built-in model, the platform decides which model runs and when a request goes to the cloud tier. With PocketPal or AI Edge Gallery, you pick the model file and can verify the phone is offline.
Battery, heat and storage
Running a model keeps the phone’s GPU or CPU at high load for as long as it generates text. Three practical effects follow:
- Heat and throttling. During long, continuous generation a phone warms up and may slow itself down. Short questions are fine; long sessions are better on a charger and out of a case.
- Battery. Generation is among the heaviest things a phone does. Treat a long chat like gaming, not like messaging.
- Storage and memory. Each model is a 1 to 5 GB download, and two or three add up quickly. In Google’s S26 Ultra test the GPU backend held 676 MB in system memory against 1,733 MB for the CPU backend, another reason to prefer the GPU.
Neither Apple nor Google publishes battery figures for third-party models, and we have not seen a controlled measurement worth quoting. Benchmark on your own device.
The alternative: reach a home or office server from the phone
A phone tops out around 4B parameters. A desktop with a graphics card, or one of the machines in our mini PC guide, runs models ten to thirty times larger. The phone then becomes a screen, and the model stays on hardware you control.
LM Link. LM Studio’s built-in option. Its documentation describes links as end-to-end encrypted networks between your own devices, made in partnership with Tailscale, and says that on iPhone you use the Locally app. Android is not mentioned.
Tailscale with your own server. For Ollama or any other server:
- Install Tailscale on the server and on the phone and sign both into the same private network, which Tailscale calls a tailnet.
- Leave the model server on its default local address and share a chat front end inside the tailnet. Tailscale Serve does this for a web interface listening on a local port:
tailscale serve 3000
- Open the address Tailscale prints, in the form
device-name.tailnet-name.ts.net, in the phone’s browser. Our comparison of self-hosted chat interfaces covers front ends that add user accounts. - If you prefer a native client such as Enchanted or Maid, the Ollama API itself has to be reachable. Bind it to the server’s Tailscale address rather than to every interface:
OLLAMA_HOST=<tailscale-ip>:11434 ollama serve
Security caveats
- Never make the model server public. Ollama binds to 127.0.0.1 on port 11434 by default, and its API has no login of its own; our Ollama security review lists what has happened to servers left open. Do not forward the port on your router.
- Serve is private, Funnel is public. Tailscale’s documentation says Serve routes traffic from devices on your tailnet, while Funnel shares a service over the public internet. Use Serve.
- Membership is the password. Anyone holding a device on the tailnet can reach what you share. Tailscale notes that access control rules apply to Serve like any other service, so limit the phone to the one port it needs, and remove a lost phone from the tailnet at once.
- The privacy boundary moves. Prompts now leave the phone. The set-up is as private as the server, its disk and whoever administers it.
Company phones: on-device is private, policy still applies
For a business, on-device models remove one risk: prompts are not sent to a third-party AI service. They leave others in place.
- The app matters. Several of these apps can also connect to cloud providers or remote servers. On a managed phone, decide which apps are approved and which features are allowed.
- Models are downloads from the internet. Someone should decide which model files and publishers are acceptable, and check each licence for commercial use.
- Small models make more mistakes. A 2B model summarising a contract is not equivalent to the assistant your organisation evaluated.
- Chats are stored on the phone. History, pasted documents and backups fall under the same device rules as email.
- A home server is unmanaged hardware. Company data sent from a work phone to an employee’s own machine has left the organisation, however private the tunnel.
Write these points into policy rather than leaving them to each employee; our AI acceptable use policy template is a starting point, and the zero-trust guide for on-premises AI covers the network side. Where models belong on phones, sites or servers in an organisation is the subject of our article on CPU and NPU inference at the edge.
How VDF AI fits
VDF AI does not run on phones, and it does not run, host, resell or integrate with any of the phone apps or networking tools in this guide. It is a platform for organisations, deployed on their own servers, private cloud or an air-gapped network. The honest bridge is this: one person’s phone model is a personal tool, while a company needs to know who hosts the model, which models are approved, who can use them and what is recorded.
VDF AI Chat is the organisation-hosted assistant, with role-based access per agent and per knowledge source and an append-only audit record for each turn. VDF AI Router registers on-premises model deployments and has an air-gap mode that restricts routing to local models. To discuss a pilot, contact us.
Sources
Looked up on 14 and 15 November 2026.
- PocketPal AI on GitHub and on the App Store
- Google AI Edge Gallery on GitHub
- Locally on the App Store and LM Link documentation
- SmolChat, ChatterUI, Maid and Enchanted on GitHub
- MLC Chat on the App Store and MLC LLM iOS documentation
- Termux llama-cpp package build script and llama.cpp server README
- Gemma 4 E2B LiteRT-LM card with device benchmarks and Gemma 4 E4B LiteRT-LM files
- LFM3-1.7B model card and GGUF files
- GGUF and MLX file sizes: Qwen3.5-0.8B, Qwen3.5-2B, Qwen3.5-4B, Qwen3.5-9B, LFM2.5-1.2B, Llama 3.2 1B, Llama 3.2 3B, SmolLM3-3B, Phi-4-mini, Gemma 4 E4B GGUF and Gemma 4 E2B MLX 4-bit
- Apple Intelligence and Foundation Models framework
- Android: Gemini Nano
- Ollama FAQ and download page
- Tailscale Serve
Need an assistant your whole organisation can use, on infrastructure you control? See VDF AI Chat, or book a demo.