9 of 9
models were fastest with NPU prefill
OpenWeights, 2 to 4 November 2026
Prefill and decode on CPU, GPU and NPU.
Which processor should prefill and which should decode: nine models, every pairing, on a Snapdragon 8 Elite Gen 5.
What we found
Prefill on the NPU, on every model
9.6 to 15.5 times as fast as the CPU and 5.4 to 7.7 times as fast as the GPU. The first token came after a median 81 to 319 ms, against 0.8 to 3.8 s on the CPU.
Decode on the NPU or the CPU, depending on the model
The NPU decoded in the fastest pairing of 6 models and the CPU in Gemma 3 1B and LFM2.5 1.2B. On LFM2.5 2.6B, NPU prefill with CPU decode and with NPU decode finished within 0.4% of each other.
Decode anywhere but the GPU
0 of 9 models were fastest with GPU decode. It was the slowest decoder on 8 models and within 3.5% of the CPU on LFM2.5 2.6B. On Qwen3 4B its median decode step rose from 107 ms at 1,024 to 1,535 tokens of context to 430 ms at 2,048 to 2,559.
Decode is most of the wait
The replies run 401 tokens at the median, and decode took 75% to over 99% of each pairing's time, so the decoder decides more than the prefill processor.
Heat shapes the results
The phone throttles itself in two steps as it heats. Every pairing compared ran under the first; the second comes and goes, and CPU decode gets stuck while it is on. Up to 23% of a CPU-decode pairing's time went to decode steps that took over half a second. Both steps are described under Heat.
The results
The fastest pairing for each model
| Model | Fastest | Its time | Next best | CPU only | Slowest |
|---|---|---|---|---|---|
| Qwen3 0.6B Q4_0 | NPU → NPU | 292 s | NPU → CPU, +8% | +25% | CPU → GPU, +134% |
| Gemma 3 1B Q4_0 | NPU → CPU | 254 s | GPU → CPU, +9% | +25% | CPU → GPU, +137% |
| Llama 3.2 1B Q4_0 | NPU → NPU | 341 s | GPU → CPU, +9% | +26% | CPU → GPU, +54% |
| LFM2.5 1.2B Q4_0, hybrid | NPU → CPU | 261 s | GPU → CPU, +9% | +16% | CPU → GPU, +53% |
| Qwen3 1.7B Q4_0 | NPU → NPU | 447 s | GPU → NPU, +16% | +26% | CPU → GPU, +115% |
| Qwen3 1.7B Q8_0 | NPU → NPU | 619 s | GPU → NPU, +4% | +36% | CPU → GPU, +65% |
| LFM2.5 2.6B Q4_0, hybrid | NPU → CPU | 595 s | NPU → NPU, +0.4% | +27% | CPU → CPU, +27% |
| Llama 3.2 3B Q4_0 | NPU → NPU | 793 s | GPU → NPU, +3% | +26% | CPU → GPU, +66% |
| Qwen3 4B Q4_0 | NPU → NPU | 1,122 s | CPU → NPU, +4% | +22% | CPU → GPU, +174% |
CPU → NPU means prefill on the CPU, decode on the NPU. Times are the total for the same 33 conversation turns; percentages are how much longer than the fastest.
Every pairing, model by model
Total time for the same 33 turns, and how much longer than the model's fastest.
- Rows: prefill
- Columns: decode
- Fastest; darker is faster
- Small number: order it ran in
- Letter: run on another phone
- *: CPU decode got stuck
Qwen3 0.6B Q4_0
Phone A
| prefill decode | CPU | GPU | NPU |
|---|---|---|---|
| CPU | , ran 6th364 s+25% | , ran 4th683 s+134% | , ran 9th347 s+19% |
| GPU | , ran 5th330 s+13% | , ran 2nd653 s+124% | , ran 1st318 s+9% |
| NPU | , ran 7th316 s+8% | , ran 8th629 s+115% | , ran 3rd292 sfastest |
Gemma 3 1B Q4_0
Phone A
| prefill decode | CPU | GPU | NPU |
|---|---|---|---|
| CPU | , ran 9th317 s+25% | , ran 1st603 s+137% | , ran 8th324 s+28% |
| GPU | , ran 6th277 s+9% | , ran 5th594 s+134% | , ran 3rd310 s+22% |
| NPU | , ran 2nd254 sfastest | , ran 4th563 s+122% | , ran 7th288 s+13% |
Llama 3.2 1B Q4_0
Phone A
| prefill decode | CPU | GPU | NPU |
|---|---|---|---|
| CPU | , ran 9th428 s+26% | , ran 7th525 s+54% | , ran 8th389 s+14% |
| GPU | , ran 2nd372 s+9% | , ran 6th498 s+46% | , ran 5th384 s+13% |
| NPU | , ran 4th449 s, including 71 s with decode stuck+32% | , ran 1st474 s+39% | , ran 3rd341 sfastest |
LFM2.5 1.2B Q4_0, hybrid
Phone C
| prefill decode | CPU | GPU | NPU |
|---|---|---|---|
| CPU | , ran 3rd304 s, including 1 s with decode stuck+16% | , ran 5th400 s+53% | , ran 6th362 s+39% |
| GPU | , ran 7th283 s+9% | , ran 2nd381 s+46% | , ran 8th341 s+31% |
| NPU | , ran 1st261 sfastest | , ran 9th354 s+36% | , ran 4th322 s+23% |
Qwen3 1.7B Q4_0
Phone C
| prefill decode | CPU | GPU | NPU |
|---|---|---|---|
| CPU | , ran 2nd564 s+26% | , ran 1st963 s+115% | , ran 9th533 s+19% |
| GPU | , ran 6th624 s, including 61 s with decode stuck+40% | , ran 4th918 s+105% | , ran 7th518 s+16% |
| NPU | , ran 3rd524 s+17% | , ran 8th877 s+96% | , ran 5th447 sfastest |
Qwen3 1.7B Q8_0
Phone B
| prefill decode | CPU | GPU | NPU |
|---|---|---|---|
| CPU | , ran 1st845 s, including 175 s with decode stuck+36% | , ran 4th1,022 s+65% | , ran 6th682 s+10% |
| GPU | , ran 3rd726 s, including 110 s with decode stuck+17% | , ran 2nd991 s+60% | , ran 8th642 s+4% |
| NPU | , ran 9th742 s, including 168 s with decode stuck+20% | , ran 5th957 s+55% | , ran 7th619 sfastest |
LFM2.5 2.6B Q4_0, hybrid
Phone B
| prefill decode | CPU | GPU | NPU |
|---|---|---|---|
| CPU | , ran 9th758 s, including 121 s with decode stuck+27% | , ran 2nd750 s+26% | , ran 8th678 s+14% |
| GPU | , ran 4th701 s, including 113 s with decode stuck+18% | , ran 7th696 s+17% | , ran 5th640 s+8% |
| NPU | , ran 3rd595 s, including 58 s with decode stuckfastest | , ran 1st667 s+12% | , ran 6th598 s+0.4% |
Llama 3.2 3B Q4_0
Phone C
| prefill decode | CPU | GPU | NPU |
|---|---|---|---|
| CPU | , ran 5th995 s, including 63 s with decode stuck+26% | , ran 9th1,313 s+66% | , ran 8th960 s+21% |
| GPU | , ran 1st931 s, including 94 s with decode stuck+17% | , ran 4th1,247 s+57% | , ran 2nd817 s+3% |
| NPU | , ran 7th864 s, including 62 s with decode stuck+9% | , ran 6th1,157 s+46% | , ran 3rd793 sfastest |
Qwen3 4B Q4_0
Phone C; GPU → CPU on phone D
| prefill decode | CPU | GPU | NPU |
|---|---|---|---|
| CPU | , ran 6th1,372 s, including 64 s with decode stuck+22% | , ran 5th3,077 s+174% | , ran 3rd1,163 s+4% |
| GPU | , run on phone D1,406 s, including 257 s with decode stuck+25% | , ran 1st2,939 s+162% | , ran 8th1,233 s+10% |
| NPU | , ran 7th1,398 s, including 256 s with decode stuck+25% | , ran 2nd2,860 s+155% | , ran 4th1,122 sfastest |
Heat: what the phone does
The phone throttles itself in two steps as it heats. These are the stages we saw, what each does to each processor, and what that means for the comparison.
Warming up and cooling down, as observed
| Stage | What the phone does | Temperature |
|---|---|---|
| Idle | No caps on the CPU or GPU | Hottest processor 40 to 55 °C |
| Within 1 to 7 minutes of heavy work | Soft throttle on: CPU and GPU clocks capped at about half. It can flicker on and off before it settles | No single trigger found: the CPU's sensors read anywhere from about 70 to 105 °C when it appeared |
| Warm | Soft throttle stays on. Every pairing compared here started in this state | Hottest processor 58 to 74 °C as each pairing started; the CPU up to 107 °C during |
| Hard throttle on | CPU cores paused, GPU and NPU slowed further, on top of the soft throttle | XO thermistor reaches 78 °C |
| Hard throttle off, usually about a minute later | Back to the soft throttle | XO thermistor below 70 °C |
| Idle again | Soft throttle off within about 10 minutes, the one time this was timed |
Between 70 and 78 °C the hard throttle stays as it was: on if the sensor last reached 78 °C, off if it last fell below 70 °C. The gap keeps it from flickering.
The processors share heat: with only the CPU working, the NPU's sensors read 94 °C; with the NPU doing nearly all the work, the CPU's read 95 °C (Llama 3.2 3B). The XO thermistor warms whichever processor works, so the hard throttle hits all three.
How each processor is throttled
| Soft throttle | Hard throttle | |
|---|---|---|
| CPU | Clock ceilings cut about in half: 6 cores from 3.63 to 1.90 GHz, 2 cores from 4.61 to 2.23 GHzDecode 21% slower (17.1 to 13.6 tok/s), steady | 6 of its 8 cores paused, leaving cores 0 and 1 (going by the switches' names)Decode stuck: 5 to 17 tokens a minute, single steps up to 19 s |
| GPU | Clock capped at 578 MHz, from 1.2 GHzPrefill about half as fast (202 to 105 tok/s) | Slowed furtherNot measured |
| NPU | No visible limit; its clock cannot be readNo slowdown found: an NPU-only pairing repeated within 1% with the soft throttle on and off | SlowedDecode at about half to two thirds of its speed at the median, never stuck |
The soft throttle's CPU and GPU figures compare the same 9 turns of Qwen3 4B on one phone, once with nothing capped and once capped. The NPU's hard-throttle figures compare the same turns run with and without it: two thirds on Qwen3 4B and Llama 3.2 3B, about half on Qwen3 1.7B.
What this means for the comparison
- Every pairing compared started warm
- Before each pairing the benchmark waited, checking every 5 seconds for up to 2 minutes, for all three processors to read 55 °C or less and the soft throttle to lift. If they did, the pairing started cool; if time ran out, it started warm. Only warm starts are compared. At every compared turn's start and end the GPU cap read 578 MHz, and the CPU ceilings read 1.90 and 2.23 GHz on 2,666 of 2,673 turns (lower still on the rest).
- Cool starts were run again
- After the phone sits idle, a run's first pairings can start cool. LFM2.5 1.2B and Qwen3 1.7B were run again, all nine pairings, after a throwaway warm-up pass of the model on the CPU. Qwen3 4B's GPU → CPU was run again on its own, warm, on a fourth phone (D); its matrices mark that cell with a D instead of the order it ran in.
- Order still matters
- Each pairing inherits the heat of the ones before it, and the hard throttle lands on whichever pairing is running: CPU decode gets stuck, NPU decode slows. So the total-time matrices show the order each model's pairings ran in, and these are the results of one shuffled order. Small gaps, such as LFM2.5 2.6B's 0.4%, are within that.
- Compare within a model
- Models ran on different phones and do different work per token, so seconds do not compare across models.
What we readThe temperature, clock and throttle readings behind this, in the phone's own namesShowHide
| Reading | What it is | What it tells us |
|---|---|---|
| Processor temperatures | The hottest of each processor's own sensors: CPU, GPU and NPU | When a pairing may start |
| Clock ceilings | The highest clock the phone currently allows each CPU core group and the GPU | Seeing the soft throttle |
| XO thermistor | A temperature sensor on the board, read by the phone's power-management chip and named for the crystal oscillator (XO) that clocks the chip | It switches the hard throttle |
| Throttle switches | The phone's own settings for pausing CPU cores and slowing the GPU and the NPU | Seeing the hard throttle |
- CPU
- Temperature zones
cpu-0-*andcpu-1-*, clock ceilingscaling_max_freq, core pausesthermal-pause-C0andthermal-pause-3C - GPU
- Temperature zones
gpuss-*, clock capthermal_pwrlevel, further slowdownkgsl - NPU
- Temperature zones
nsphvx-*andnsphmx-*, slowdowncdsp - Board
- The XO thermistor
sys-therm-0, namedpmk8850_xo_thermin the device tree
When CPU decode gets stuckWhat happens during the hard throttle, what it cost each pairing, why we think it happens, whether the OpenWeights app hits it, and what could fix it.ShowHide
What happens
The models' median CPU decode steps took 18 to 83 ms. We call a step stuck when it takes over half a second.
31 of the CPU-decode turns got stuck. In 29 of them decode crawled through one unbroken stretch of 44 to 71 seconds, producing 4 to 20 tokens (single steps up to 19 s), then went back to normal. The other 2 each had one slow step, of 0.7 and 7.3 s.
No NPU decode step took over half a second, and the GPU's slow steps late in long conversations are its normal pace.
| Model | Pairing | Total time | Of which stuck | Longest step | Turns stuck |
|---|---|---|---|---|---|
| Llama 3.2 1B Q4_0 | NPU → CPU | 448.6 s | 71.3 s (16%) | 5.2 s | 1 of 33 |
| LFM2.5 1.2B Q4_0, hybrid | CPU → CPU | 303.5 s | 0.7 s (under 1%) | 0.7 s | 1 of 33 |
| Qwen3 1.7B Q4_0 | GPU → CPU | 624.3 s | 60.5 s (10%) | 8.4 s | 1 of 33 |
| Qwen3 1.7B Q8_0 | CPU → CPU | 845.1 s | 175.0 s (21%) | 13.0 s | 3 of 33 |
| Qwen3 1.7B Q8_0 | GPU → CPU | 725.6 s | 110.2 s (15%) | 8.4 s | 2 of 33 |
| Qwen3 1.7B Q8_0 | NPU → CPU | 742.1 s | 168.1 s (23%) | 12.9 s | 3 of 33 |
| LFM2.5 2.6B Q4_0, hybrid | CPU → CPU | 758.3 s | 121.1 s (16%) | 13.7 s | 2 of 33 |
| LFM2.5 2.6B Q4_0, hybrid | GPU → CPU | 700.6 s | 112.6 s (16%) | 13.2 s | 2 of 33 |
| LFM2.5 2.6B Q4_0, hybrid | NPU → CPU | 595.3 s | 57.6 s (10%) | 12.1 s | 1 of 33 |
| Llama 3.2 3B Q4_0 | CPU → CPU | 995.3 s | 63.4 s (6%) | 12.3 s | 1 of 33 |
| Llama 3.2 3B Q4_0 | GPU → CPU | 930.8 s | 93.9 s (10%) | 9.0 s | 2 of 33 |
| Llama 3.2 3B Q4_0 | NPU → CPU | 864.2 s | 61.5 s (7%) | 11.6 s | 1 of 33 |
| Qwen3 4B Q4_0 | CPU → CPU | 1,371.6 s | 64.5 s (5%) | 13.5 s | 1 of 33 |
| Qwen3 4B Q4_0 | GPU → CPU | 1,406.3 s | 256.9 s (18%) | 19.1 s | 6 of 33 |
| Qwen3 4B Q4_0 | NPU → CPU | 1,398.2 s | 255.8 s (18%) | 18.3 s | 4 of 33 |
The stuck time stays in the totals, because a user would wait through it. A fix would win back less than that share, since the hard throttle would still slow the CPU. If a fix won back the same fraction of its stuck time in every pairing, the fastest would change only on Qwen3 1.7B Q8_0, from NPU → NPU to NPU → CPU, once that fraction passed about 73%.
It happens during the hard throttle
A log of the XO thermistor and the throttle switches every 2 seconds caught six hard throttles in a row. Each time the CPU was decoding, it was stuck for that same minute, to within a few seconds. The time the NPU was decoding, it kept going.
| On | Off | Length | Decoding | CPU decode stuck |
|---|---|---|---|---|
| 18:00:50 | 18:01:51 | 61 s | CPU, Qwen3 1.7B | 18:00:50 to 18:01:51 |
| 18:12:04 | 18:13:29 | 85 s | NPU, Qwen3 1.7B | No; the NPU kept going |
| 18:51:27 | 18:52:26 | 59 s | CPU, Qwen3 4B | 18:51:27 to 18:52:24 |
| 18:55:43 | 18:56:48 | 65 s | CPU, Qwen3 4B | 18:55:48 to 18:56:47 |
| 19:00:31 | 19:01:33 | 62 s | CPU, Qwen3 4B | 19:00:32 to 19:01:33 |
| 19:05:20 | 19:06:27 | 67 s | CPU, Qwen3 4B | 19:05:23 to 19:06:25 |
Local time, 3 November 2026. A fourth phone (D) logged 5 more at the same switch points (78.0 to 78.2 °C on, 69.7 to 69.9 °C off). Earlier runs, logged every 10 seconds, agree: on Qwen3 4B and Llama 3.2 3B every stuck step fell in a turn with the hard throttle on. Hard throttles lasted 0.6 to 13.9 minutes, most of them about one.
Why we think it happens
The CPU decodes with 6 threads, and normally each has a core of its own. llama.cpp computes each token as a chain of operations (874 in Llama 3.2 3B's plan). Many are split across the threads, and the threads wait for each other before the next one starts: hundreds of times per token.
The hard throttle leaves 2 cores, so only 2 of the 6 threads can run at a time. A thread that has finished its part does not give up its core. Unlike a Python thread waiting for the global interpreter lock (GIL), which sleeps, it keeps checking whether the others are done until the scheduler moves it off, while the threads with work left wait for a core.
Each wait can then cost milliseconds, hundreds of times per token: enough to turn milliseconds per token into seconds. This fits the code and the timings, but it has not been proven.
Does the OpenWeights app hit it?
The app sets its decode threads once before each reply, from Android's heat signal: 4 on this phone, 3 when Android reports light or moderate heat, 2 at severe, and at critical or above it does not start the reply.
On this phone Android reported severe (skin 57.8 °C) and emergency (skin 64.1 °C), about 20 minutes apart; skin is Android's estimate of the phone's surface temperature.
A reply already running keeps its threads, so a hard throttle that starts mid-reply can still meet 4. On a phone whose Android signal lags the heat, the app would keep 4 threads throughout.
What could fix it
| Change | Where | Status |
|---|---|---|
| Drop to 2 threads as soon as one decode step takes over half a second, then step back up | App | Not measured |
| Make a waiting thread give up its core after a short check, instead of holding it | llama.cpp | Not measured; must not slow normal decode |
| Decode on the NPU during the hard throttle | App | Not measured; the NPU is slower then too |
| Read how many cores the app may use, before each step | App | Ruled out on this phone: the count stayed at 8 through a logged hard throttle |
How we ran it
- Phones
- Four Snapdragon 8 Elite Gen 5 (SM8850) phones of the same kind (A to D), Android 16, root shell, on AC power in Qualcomm Device Cloud's rack, 2 to 4 November 2026. Each model ran all nine of its pairings on one phone, in one run, except Qwen3 4B's GPU → CPU, run again on phone D because it had started cool.
- Runtime
- llama.cpp b2e5e9b (release b10549) with its CPU, OpenCL (GPU) and Hexagon (NPU) backends in one process. When prefill and decode run on different processors, the conversation state is copied across every turn. The CPU used 6 threads.
- Models
- Qwen3 0.6B, 1.7B and 4B; Gemma 3 1B; Llama 3.2 1B and 3B; LFM2.5 1.2B and 2.6B. All in Q4_0, the 4-bit format all three backends speed up, and Qwen3 1.7B also in Q8_0, an 8-bit format. The LFM2.5 models are hybrids, mixing attention with other layer types, in Liquid AI's own Q4_0 builds.
- Conversations
- Nine from WildChat-1M, UltraChat 200k and Daring-Anteater, 33 turns in all. Each turn's prefill covers only its new tokens, 18 to 1,215 depending on the model; earlier turns are already in the cache.
- Same work in every pairing
- Each turn decodes as many tokens as the dataset's own reply has (401 at the median, up to 1,046, counted by Qwen3), and the next turn continues from the dataset's history, not from what the model wrote. So all nine pairings do exactly the same amount of work, even where their outputs differ.
Limits
- Against the CPU
- 6 threads throughout. Fewer threads might avoid getting stuck in the hard throttle; that is not tested here.
- Against the GPU
- One GPU backend, llama.cpp's OpenCL. Its slowdown on long conversations may be the backend, not the chip.
- The NPU
- One llama.cpp build; newer builds change its NPU backend. NPU decode also uses the CPU for the output layer and the embedding lookup; how much of its time that takes was not measured.
- One format
- Q4_0 for all but one model, the format all three backends speed up. Other CPU formats were not tried.
- Not measured
- Energy, battery drain and memory. A phone in a hand, rather than in a rack on AC power. An ordinary app without root, and a cold start. Contexts past about 4,000 tokens, replies of the model's own length, and continuing a conversation from what the model wrote. Answer quality.
Sources
The tables are generated from the raw results in the OpenWeights repository; the heat readings come from the phone and are recorded in the report.