ExecuServeOn Google Play
ExecuTorch models, served from your phone.
ExecuServe keeps compiled ExecuTorch models loaded on an Android phone and answers the OpenAI and Anthropic APIs, for any app on the phone or, if you allow it, on your network.
Android 12+ on arm64 · Apache-2.0

What it does
- A server, not a chat app
- Your agents, SDKs and tools call the phone the way they call a cloud.
- The KV cache belongs to the server
- Clients resend the conversation, and the server reuses its cache, so follow-ups stay fast.
- Several models, one phone
- Up to three resident, each with its own cache and endpoints.
- Built to stay up
- A foreground service, tested through deep Doze with the screen off.
- A browser chat inside
- A laptop on your network can talk to the phone with nothing installed.
- CPU, GPU and NPU
- The catalog offers each phone only the files its chip can load.
- Private by construction
- No account, no analytics, no crash reporter. Every request needs a key, loopback included.
Measured on a phone
On a POCO X8 Pro Max (Dimensity 9500s), over Wi-Fi, with the minified release build.
| Check | Result |
|---|---|
| Official OpenAI SDK suite | 16 of 16 pass |
| Official Anthropic SDK suite | 8 of 8 pass |
| Edge-case probe (refusals, limits, disconnects) | 28 of 28 pass |
| Screen-off soak on battery, 10 minutes | 20 of 20 answered |
| Warm second turn of a 2,000-token conversation | 0.47 s instead of 8.4 s |
| Qwen3 tool loop, second request | 174 of 205 prompt tokens reused |
| XNNPACK export against llama.cpp, Snapdragon 8 Elite | 1.2 to 1.6× faster decode |
| Qwen3-1.7B, 700-token prompt, Snapdragon 8 Elite Gen 5: first token on the NPU | 0.41 s, against 2.0 s on the CPU and 1.0 s on the GPU |
| Qwen3-1.7B decode on the same phone, CPU / GPU / NPU | 54 / 42 / 25 tokens per second |
| MediaTek NPU, LFM2.5-1.2B, Dimensity | NPU prefill, CPU decode at 39 to 43 tokens per second |
Method and raw results: the POCO X8 Pro Max report. The decode advantage of the XNNPACK export over llama.cpp was measured in OpenWeights, on whose ExecuTorch engine ExecuServe is built.
Quick start
An arm64 phone on Android 12 or later, adb on your computer, and an ExecuTorch export with its tokenizer.
From a clone of the repository, with the app installed:
tools/execuserve --model ~/models/Qwen3-1.7B-8da4w-gptq-2k.pte
export OPENAI_BASE_URL=http://127.0.0.1:8080/v1
export OPENAI_API_KEY=<the key it prints>Then any OpenAI client:
from openai import OpenAI
client = OpenAI() # reads OPENAI_BASE_URL and OPENAI_API_KEY
reply = client.chat.completions.create(
model="qwen3-1.7b",
messages=[{"role": "user", "content": "Hello"}],
)From a clone of the repository, with the app installed:
tools/execuserve --model ~/models/Qwen3-1.7B-8da4w-gptq-2k.pte
export ANTHROPIC_BASE_URL=http://127.0.0.1:8080
export ANTHROPIC_API_KEY=<the key it prints>Then any Anthropic client:
from anthropic import Anthropic
client = Anthropic() # reads ANTHROPIC_BASE_URL and ANTHROPIC_API_KEY
reply = client.messages.create(
model="qwen3-1.7b",
max_tokens=200,
messages=[{"role": "user", "content": "Hello"}],
)The script pushes the model, starts the server, forwards the port over adb, and prints the base URL and key.
Reference
The API
OpenAI's and Anthropic's shapes, error codes and streaming, checked with their official SDKs.
| Route | What it does |
|---|---|
POST /v1/chat/completions | Streaming, tools and tool calls, reasoning, usage with cached tokens |
POST /v1/responses | Items, function calls, typed stream events, previous_response_id |
POST /v1/messages | Anthropic's Messages API: tool_use, thinking blocks, its stream events |
POST /v1/completions | A raw prompt, no template |
GET /v1/models | Installed models with context length, residency and capabilities |
GET /metrics | Prometheus counters, gauges and quantiles |
/models/{id}/v1 | The model, inference and status routes scoped to one model, for hosting several at once |
Clients
| Client | Setting |
|---|---|
| OpenAI SDKs, LangChain, Vercel AI SDK | Base URL http://<phone>:8080/v1, plus the key |
| Anthropic SDKs and agents built on them | Base URL http://<phone>:8080, with no /v1, plus the key |
| Open WebUI | Settings, Connections, OpenAI API, with the URL http://<phone>:8080/v1 |
| codex CLI | A provider with wire_api = "responses" |
| Another app on the same phone | http://127.0.0.1:8080/v1, with a network security config allowing loopback |
Get a model onto the phone
| Way | How |
|---|---|
| In the app | Library, then Browse the catalog, which lists the exports this phone can run |
| From your computer | tools/execuserve --model path/to/Model.pte pushes it with its tokenizer and starts serving |
| Have the phone fetch it | tools/execuserve pull <hf-repo> <file.pte> downloads and verifies it on the phone |
Documentation
| README | Everything the server does, and how to build it |
|---|---|
| Architecture | The design, the HTTP API in full, and edge cases |
| Testing on a phone | From installing to a screen-off soak |
| Results | Measurements on real phones, with dates |
| Privacy policy | What stays on the phone and what leaves |
| Contributing | Setting up, the checks, what a change needs |
| Security | Reporting a vulnerability, and what is in scope |
Open source
Apache-2.0, on GitHub. Next are the iOS app, server-side tools, vision input, and TLS for network mode. Not affiliated with, endorsed by or sponsored by the PyTorch Foundation or Meta.