9 of 9
models were fastest with NPU prefill
Newest: 2 to 4 November 2026
Prefill and decode on CPU, GPU and NPU
Which processor should prefill and which should decode: nine models, every pairing, on a Snapdragon 8 Elite Gen 5.
Research.
What we measured, on which hardware, and how. Every study links to its committed results.
42%
of 24 real tasks with Qwen3.6 on an RTX 4090, against 92% on Copilot Pro+ Auto
GGUF, October 2026
When is local Copilot worth it?
The same Copilot CLI agent on the same real tasks: open-weight GGUF models on an RTX 4090 and a MacBook Air M5, against hosted Copilot.
8 agents
at a 4.9 s wait per turn on vLLM, with no errors. At 16, waits reach 16.5 s
Hardware, 23 and 24 September 2026
How many coding agents can one RTX 4090 serve?
Qwen3.8-27B on one rented RTX 4090, through five engine setups, rising load and real coding agents.
Q6_K
is the smallest of ten formats to pass the quality gate
GGUF, 12 September 2026
A 2B model in ten GGUF formats on one A100
Size, speed and how many of its decisions each format keeps, and the same weights behind vLLM.
Over 6 GB
for Qwen3 at 32k, before the first prompt
OpenWeights, 7 September 2026
Does the exported window matter?
Whether the context window fixed at export, from 2k to 32k, changes anything but memory.
About 4×
the median IFEval gain from raising the reply cap to 2048 tokens: 18 to 21 of 30, up from 3 to 6
OpenWeights, 7 September 2026
A 640-token reply cap was cutting off the LFM2.5 2.6B
The model reasons before it answers, so a 640-token cap cut it off. Raising only the cap lifted its scores on every phone. The same day's reruns also tested repeats of one file and a run that had died.
18 of 25
model and chip pairs reached the first token sooner on ExecuTorch
OpenWeights, 3 and 4 September 2026
How fast five small models answer on five phone chips, in llama.cpp and in ExecuTorch.
63 to 71 ms
per token on Inferentia2 from 1 to 32 streams, while throughput grows 26×
ASICs, July and August 2026
Llama 3.1 8B on AWS Trainium and Inferentia
A fine-tune on Trainium, served on Inferentia2, measured beside an MI300X and an H200.
1.84×
the H200's KV cache on an MI300X with 70 GB of weights loaded, from 1.36× the memory
Hardware, July 2026
One AMD Instinct MI300X against one NVIDIA H200, inference and training, on the same vLLM and PyTorch versions.
2.9×
the X2 Elite's GPU prefill on the M5, Gemma 3 1B Q4_0. On the CPU, the X2 Elite leads by 1.9×
Hardware, 27 July 2026
Apple M5 against Snapdragon X2 Elite
A MacBook Air against a Qualcomm reference board on AI work: CPU, GPU and NPU, inference and training, with the same llama.cpp release and byte-identical weights.
About 7×
the GPU matmul of a Dimensity 9500s on a Snapdragon 8 Elite
Hardware, 13 and 27 July 2026
Snapdragon 8 Elite against Dimensity 9500s
What unprivileged code can do for AI on two flagship phone chips: NPU, CPU, GPU and on-device training.