24 October 2026
Prefill and decode on CPU, GPU and NPU
Every pairing of the three processors on a Snapdragon 8 Elite Gen 5, for nine models in one llama.cpp build, on real multi-turn conversations.
OpenWeightsOn Google Play
Find a model. Run it here.
OpenWeights searches Hugging Face for GGUF and ExecuTorch models, estimates whether one fits your phone before you download it, and runs it on the device. Every token comes from your own hardware: no account, no cloud, no telemetry.
What it does
- Hugging Face is the model list
- Search the Hub for GGUF and ExecuTorch repositories from inside the app. It shows what fits and runs supported architectures on the phone.
- Fit before the download
- The app reads a GGUF's header over HTTP range requests, so it can say what the file needs at your context length, and whether it will run, before you spend a gigabyte.
- Two runtimes, one app
- llama.cpp runs GGUF files and ExecuTorch runs compiled .pte files for eight model families. The app picks by file format, and both ship in the same 28.8 MB release.
- A fast first turn
- The instructions and tool definitions are read into the cache while you type and kept on disk. On the reference phone, a fresh chat's first token came in under one second instead of about 18.5.
- An agent within limits you set
- Seventeen tools and four run modes. The three tools that use the internet each have their own switch, and every tool call is a row in the reply naming what it was given.
- Private by construction
- No accounts, no analytics, no crash reporter, and no backups off the device. What does reach the internet is named: model search and downloads, and the three network tools.
Measured on real phones
- NPU + CPU67.3 s
- NPU only74.9 s
- CPU only148.5 s
- GPU only154.2 s
7 September 2026
Does the exported window matter?
ExecuTorch fixes a model's context window when the model is exported, from 2k to 32k. Whether that choice changes anything but memory.
7 September 2026
A second pass over the window matrix: every cell checked for a failed or noisy run, and three things rerun the same day.
3 and 4 September 2026
Median time to first token and time per output token for five small models on five phone chips, in llama.cpp and ExecuTorch.
Open source
Apache-2.0, on GitHub. Android 12 or later on arm64; an iPhone version is experimental. Models are published by third parties and their licences are their own.