Five chips, two runtimes.

Median time to first token (TTFT) and time per output token (TPOT), from 60 to 90 benchmark prompts per cell (GSM8K, IFEval, BFCL). Prompts run 19 to 772 tokens, median 92.

Purpose: Show how fast five small models answer on five phone chips, on the two runtimes the app ships, so a reader can pick a model and runtime for a given phone. Time to first token is what a user waits before the reply starts; time per output token is how fast it then streams.
How it was graded: Not graded here: this page is timing only. The same 90 prompts (30 GSM8K, 30 IFEval, 30 BFCL; Gemma 3 skips BFCL, having no tool syntax) ran greedy with thinking off, and the median prefill and decode time of each cell is plotted. The pass rates for these runs are in the repository's benchmark matrix.
How long it ran: One pass per cell on 2026-09-03 and 04. Generation time per cell ran from 6 minutes (LFM2.5-1.2B on ExecuTorch, 8 Elite) to about 100 minutes (SmolLM3 on llama.cpp, Tensor G5); 29 hours of phone time in all, plus loading and Test Lab queueing.

Every model on every chip

Latency map

Each line joins one chip's two runtimes. Lower and left is faster. Both axes are logarithmic.

llama.cpp, GGUFExecuTorch

All models, all chips

The same numbers as a grid, both columns on logarithmic scales. The model picked above is banded.

llama.cpp, GGUFExecuTorch

Limits

Not matched conditions. The Dimensity 9400 ran in hand under a desk fan. The Snapdragon 8 Gen 3 ran on Qualcomm Device Cloud and the 8 Elite, Tensor G5 and Exynos 2400 on Firebase Test Lab, racked, where phones throttle within about 40 seconds.
Two open gaps. Tensor G5 and Exynos 2400 decode at about half speed on llama.cpp against ExecuTorch, cause unknown. SmolLM3 on ExecuTorch prefills at a third of llama.cpp's rate on every chip, which is the compiled model rather than the phones.
Files. llama.cpp ran Q4_K_M GGUFs, Qwen3 at Q8_0. ExecuTorch ran the published XNNPACK INT8/INT4 exports, LFM2.5 as 8da4w and Llama 3.2 as SpinQuant. Runs on 2026-09-03 and 04, single pass, no repeats.

Every number

Medians per cell. Time to first token is shown with the median prefill rate behind it, and time per output token with the decode rate it amounts to.

Chip	llama.cpp, GGUF			ExecuTorch
Chip	TTFT, s	Prefill, tok/s	TPOT, ms	TTFT, s	Prefill, tok/s	TPOT, ms
Gemma 3 1B
Dimensity 9400	1.27	61	42	0.43	164	48
Snapdragon 8 Gen 3	1.02	69	51	0.37	192	27
Snapdragon 8 Elite	0.80	79	28	0.40	167	44
Tensor G5	1.35	56	113	0.48	156	50
Exynos 2400	1.68	45	75	0.45	151	56
LFM2.5 1.2B
Dimensity 9400	0.81	138	31	0.30	321	25
Snapdragon 8 Gen 3	0.73	130	43	0.44	217	27
Snapdragon 8 Elite	0.59	153	27	0.30	288	18
Tensor G5	0.83	112	106	0.32	248	46
Exynos 2400	0.97	92	59	0.42	208	38
Qwen3 1.7B
Dimensity 9400	1.05	113	57	0.76	122	64
Snapdragon 8 Gen 3	0.49	195	74	0.77	120	41
Snapdragon 8 Elite	0.37	266	40	0.75	130	41
Tensor G5	1.17	93	184	0.89	102	94
Exynos 2400	0.89	100	97	0.81	112	67
Llama 3.2 3B
Dimensity 9400	2.39	39	82	1.36	87	82
Snapdragon 8 Gen 3	2.11	43	107	1.82	66	91
Snapdragon 8 Elite	1.70	48	83	1.47	80	63
Tensor G5	2.43	37	180	1.55	71	125
Exynos 2400	3.46	25	151	1.73	64	119
SmolLM3 3B
Dimensity 9400	1.18	42	79	12.69	13	91
Snapdragon 8 Gen 3	1.43	39	101	10.33	14	85
Snapdragon 8 Elite	1.03	47	79	7.18	21	67
Tensor G5	1.86	35	239	15.90	11	108
Exynos 2400	1.79	29	132	12.26	11	121