Over 6 GB

for Qwen3 at 32k, before the first prompt

OpenWeights, 7 September 2026

Does the exported window matter?

Whether the context window fixed at export, from 2k to 32k, changes anything but memory.

What we found

Answers: no consistent effect
Grades moved between windows about as much as between two runs of the same file.
Speed: close, except Qwen3 at 32k
Paired on the Dimensitys, LFM2.5's windows were within 2.4% of each other. Qwen3 at 32k was 5.9% slower to prefill and 4.7% to decode than at 2k.
Memory: the one real cost
Qwen3-1.7B at 32k needs over 6 GB at load, and 12 GB phones kill it.
2k is tight for a model that reasons
At 2k, 14 of LFM2.5-2.6B's 60 GSM8K and IFEval replies ran into the window. Grades barely moved.
Test the two ends first
Compute the KV cost first, run the smallest and largest window per model and chip, and add the middle ones only where those two differ or fail.

The results

Every chart, one condition at a time

Pick a model or condition; every chart follows.

Memory after load

Resident memory of the test process right after each export loaded, Dimensitys, from the fixed-prompt probe. The picked model is drawn solid.

Speed, paired probe

Each window's throughput as a ratio of the model's smallest window, same 929-token prompt, same turn against same turn, six pairs per file. Dot is the median, bar the range. Dimensitys only.

PrefillDecode1.00 means no difference

Grades by window

Prompts passed out of 30 on each set, per chip. Points are joined only where neighbouring windows were both measured; the cloud phones have endpoint measurements only and cannot exclude an effect at 8k or 16k. Hollow markers are one same-file repeat on the same phone, an observed run-to-run reference rather than a noise bound.

DimensitysSnapdragon 8 EliteTensor G5Exynos 2400Second run, same file

Identical replies

How many prompts came back byte-identical to the largest window's reply: the raw token stream, the parsed tool calls (BFCL prompts only) and the shown text. Dashed lines are same-file repeats on the phone of that colour, a reference for observed run-to-run agreement.

DimensitysSnapdragon 8 EliteTensor G5Exynos 2400Same file run twice× fewer prompts than the full set

What else the matrix found

A missing start token: Every compiled model this app had ever run was fed no start-of-text token: the ExecuTorch 1.4.0 runtime adds one only when asked and the app never asked. LFM2.5-1.2B shrugs it off; LFM2.5-2.6B scored 0 of 30 on GSM8K on every chip until the engine wrote the token itself, and every 2.6B cell here is from the fixed build. The 1.2B cells remain BOS-less as measured: comparable among themselves, and a description of that deployed configuration.
A phone that dozes: It throttles instrumentation two to five times with wild variance, so the local runner now wakes the phone and holds it awake.
Two 2.6B conditions: Its own template makes it reason before answering, so under the suite's 640-token cap 16 of 30 GSM8K and 26 of 30 IFEval replies never reached an answer: those rows are cap-censored and kept for their BFCL calls, memory and speed. The rerun at a 2048 cap, GSM8K and IFEval only, gave it room to answer: 23 to 28 of 30 and 18 to 21 of 30 on every phone and window, with its same-file repeat moving grades by 2 and its BFCL calls unchanged.
Qwen3 32k on 12 GB phones: Samsung's Heimdall memory guard killed the test process on the Galaxy S25 Ultra and S24+ after 7 to 12 prompts ("Trigger Global kill before GC, Usage 8.67 GB, Threshold 6 GB" in the logcats). A rerun on the Poco with logcat attached died the same way after 7: MIUI's ActivityManager force-stopped it for "used too many pss resource, pss used 8.65 GB, threshold 6 GB". Only the 16 GB Pixel 10 Pro XL finished. Those cells are marked incomplete and carry no grade or timing.

Every number

The full matrix, as rendered from the graded reports by window_report.py.

LFM2.5 1.2B14 cells, LFM2.5-1.2B-Instruct-8da4wShow

Scrolls sideways; the phone and window column stays put.

Phone and window	File	GSM8K	IFEval	BFCL	Capped	Prefill ms	ms/token	Prefill tok/s	Decode tok/s	Ran at	RSS after load MB	RSS at end MB
D9500s, 2k	796 MB	17/30	15/30	26/30	10/90	360	25.9	254	38.7	2048	not logged	1186
D9500s, 4k	798 MB	16/30	17/30	26/30	12/90	363	26.7	253	37.5	4096	1099	1218
8 Elite, 4k	798 MB	13/30	18/30	28/30	11/90	374	17.7	227	56.4	4096	1104	1207
Tensor G5, 4k	798 MB	13/30	18/30	27/30	14/90	817	64.2	129	15.6	4096	1113	1225
Exynos 2400, 4k	798 MB	13/30	17/30	26/30	11/90	460	34.0	195	29.4	4096	1126	1213
Exynos 2400 repeat, 4k	798 MB	15/30	21/30	27/30	10/90	516	37.5	180	26.7	4096	1126	1217
D9500s, 8k	802 MB	17/30	18/30	26/30	11/90	353	26.1	259	38.3	8192	1226	1333
D9500s, 16k	810 MB	13/30	20/30	27/30	12/90	354	25.9	262	38.6	16384	not logged	993
D9500s repeat, 16k	810 MB	14/30	16/30	28/30	12/90	368	26.2	259	38.2	16384	1416	1528
D9500s, 32k	827 MB	12/30	18/30	27/30	16/90	366	26.3	257	38.0	32768	not logged	1903
D9500s repeat, 32k	827 MB	12/30	15/30	27/30	12/90	392	28.8	237	34.8	32768	1798	1901
8 Elite, 32k	827 MB	16/30	13/30	26/30	11/90	324	16.3	261	61.2	32768	not logged	1880
Tensor G5, 32k	827 MB	16/30	18/30	27/30	13/90	862	79.5	98	12.6	32768	not logged	1896
Exynos 2400, 32k	827 MB	14/30	20/30	25/30	11/90	462	37.1	184	27.0	32768	not logged	1883

Run-to-run control: the same file run twice on the same phone, replies byte-identical.

Window	Phone	Identical replies
4k	Exynos 2400	raw 7/90, calls 23/30, content 11/90
16k	D9500s	raw 13/90, calls 23/30, content 20/90
32k	D9500s	raw 15/90, calls 25/30, content 18/90

Replies identical to the 32k export, per window: raw stream and shown content over the prompts both runs completed, parsed tool calls over the BFCL prompts among them.

Phone	2k	4k	8k	16k
D9500s	raw 12/90, calls 25/30, content 17/90	raw 17/90, calls 26/30, content 18/90	raw 18/90, calls 28/30, content 19/90	raw 12/90, calls 24/30, content 16/90
D9500s repeat	-	-	-	raw 14/90, calls 25/30, content 17/90
8 Elite	-	raw 13/90, calls 26/30, content 15/90	-	-
Tensor G5	-	raw 10/90, calls 24/30, content 14/90	-	-
Exynos 2400	-	raw 16/90, calls 28/30, content 16/90	-	-

LFM2.5 2.6B, 640 cap9 cells, LFM2.5-2.6B-8da4wShow

Scrolls sideways; the phone and window column stays put.

Phone and window	File	GSM8K	IFEval	BFCL	Capped	Prefill ms	ms/token	Prefill tok/s	Decode tok/s	Ran at	RSS after load MB	RSS at end MB
D9500s, 2k	1783 MB	15/30	4/30	24/30	42/90	882	60.3	108	16.6	2048	1979	1958
D9500s repeat, 2k	1783 MB	13/30	6/30	24/30	43/90	872	59.6	110	16.8	2048	1984	1871
8 Elite, 4k	1786 MB	13/30	5/30	22/30	44/90	1197	55.4	72	18.0	4096	2058	2194
Tensor G5, 4k	1786 MB	17/30	6/30	24/30	38/90	1066	92.2	88	10.8	4096	2054	2221
Exynos 2400, 4k	1786 MB	10/30	5/30	23/30	46/90	1142	83.5	81	12.0	4096	2136	2189
D9500s, 32k	1815 MB	13/30	5/30	24/30	45/90	882	59.7	109	16.8	32768	2937	2185
8 Elite, 32k	1815 MB	13/30	5/30	25/30	42/90	1096	49.7	81	20.1	32768	2950	3082
Tensor G5, 32k	1815 MB	19/30	4/30	25/30	37/90	968	81.1	101	12.3	32768	2962	3120
Exynos 2400, 32k	1815 MB	18/30	3/30	25/30	39/90	1085	84.5	80	11.8	32768	3032	3084

Run-to-run control: the same file run twice on the same phone, replies byte-identical.

Window	Phone	Identical replies
2k	D9500s	raw 0/90, calls 24/30, content 29/90

Replies identical to the 32k export, per window: raw stream and shown content over the prompts both runs completed, parsed tool calls over the BFCL prompts among them.

Phone	2k	4k
D9500s	raw 0/90, calls 24/30, content 29/90	-
8 Elite	-	raw 0/90, calls 25/30, content 29/90
Tensor G5	-	raw 0/90, calls 27/30, content 30/90
Exynos 2400	-	raw 0/90, calls 25/30, content 30/90

LFM2.5 2.6B, 2048 cap8 cells, LFM2.5-2.6B-8da4w (cap 2048)Show

Scrolls sideways; the phone and window column stays put.

Phone and window	File	GSM8K	IFEval	BFCL	Capped	Prefill ms	ms/token	Prefill tok/s	Decode tok/s	Ran at	RSS after load MB	RSS at end MB
D9500s, 2k	1783 MB	24/30	21/30	-	14/60	693	62.8	105	15.9	2048	1959	1783
8 Elite, 4k	1786 MB	25/30	19/30	-	11/60	866	51.1	82	19.6	4096	2055	2139
Tensor G5, 4k	1786 MB	23/30	21/30	-	13/60	776	81.9	106	12.2	4096	2055	2188
Exynos 2400, 4k	1786 MB	25/30	21/30	-	13/60	893	87.1	84	11.5	4096	2136	2187
D9500s, 32k	1815 MB	26/30	21/30	-	10/60	621	59.5	113	16.8	32768	2922	2262
8 Elite, 32k	1815 MB	24/30	21/30	-	15/60	982	54.5	80	18.4	32768	2950	2626
Tensor G5, 32k	1815 MB	28/30	18/30	-	12/60	842	102.4	80	9.8	32768	2962	3123
Exynos 2400, 32k	1815 MB	27/30	19/30	-	10/60	924	87.8	82	11.4	32768	3028	3080

Replies identical to the 32k export, per window: raw stream and shown content over the prompts both runs completed, parsed tool calls over the BFCL prompts among them.

Phone	2k	4k
D9500s	raw 0/60, calls 0/0, content 0/60	-
8 Elite	-	raw 0/60, calls 0/0, content 0/60
Tensor G5	-	raw 0/60, calls 0/0, content 0/60
Exynos 2400	-	raw 0/60, calls 0/0, content 0/60

Qwen3 1.7B9 cells, Qwen3-1.7B-8da4wShow

Scrolls sideways; the phone and window column stays put.

Phone and window	File	GSM8K	IFEval	BFCL	Capped	Prefill ms	ms/token	Prefill tok/s	Decode tok/s	Ran at	RSS after load MB	RSS at end MB
D9500s, 2k	1285 MB	20/30	15/30	24/30	8/90	532	45.1	174	22.2	2048	1758	1833
D9500s repeat, 2k	1285 MB	17/30	18/30	24/30	7/90	532	45.7	174	21.9	2048	1766	1900
8 Elite, 2k	1285 MB	18/30	13/30	25/30	12/90	673	41.9	141	23.9	2048	1771	1887
Tensor G5, 2k	1285 MB	23/30	13/30	24/30	8/90	814	89.3	121	11.2	2048	1772	1892
Exynos 2400, 2k	1285 MB	17/30	12/30	23/30	10/90	734	65.9	125	15.2	2048	1838	1886
D9500s, 32k	1348 MB	INCOMPLETE: process ended after 6/90 prompts								32768	6188	5426 (last observed)
8 Elite, 32k	1348 MB	INCOMPLETE: process ended after 12/90 prompts								32768	6127	4836 (last observed)
Tensor G5, 32k	1348 MB	16/30	14/30	25/30	8/90	844	94.0	139	10.6	32768	8499	8602
Exynos 2400, 32k	1348 MB	INCOMPLETE: process ended after 7/90 prompts								32768	6396	6380 (last observed)

Run-to-run control: the same file run twice on the same phone, replies byte-identical.

Window	Phone	Identical replies
2k	D9500s	raw 31/90, calls 30/30, content 31/90

Replies identical to the 32k export, per window: raw stream and shown content over the prompts both runs completed, parsed tool calls over the BFCL prompts among them.

Phone	2k
D9500s	raw 0/6, calls 0/0, content 0/6
8 Elite	raw 0/12, calls 0/0, content 0/12
Tensor G5	raw 29/90, calls 27/30, content 31/90
Exynos 2400	raw 0/7, calls 0/0, content 0/7

Fixed-prompt speed probeDimensitys, 12 files, 929-token prompt, 160-token cap, three turns per load, two interleaved passesShow

Phone cooled below 42 C and awake before each load. The second table pairs each turn with the same turn of the model's smallest window in the same pass.

File	Turns	Prefill tok/s (median, min to max)	Decode tok/s (median, min to max)	Prefill ms, 929 tokens	Resident after load MB
LFM2.5-1.2B-Instruct-8da4w-2k	6	332 (288 to 344)	40.2 (33.7 to 41.0)	2804	1050
LFM2.5-1.2B-Instruct-8da4w-4k	6	340 (330 to 344)	40.4 (36.9 to 41.0)	2738	1098
LFM2.5-1.2B-Instruct-8da4w-8k	6	343 (327 to 345)	40.8 (40.0 to 41.2)	2712	1194
LFM2.5-1.2B-Instruct-8da4w-16k	6	342 (326 to 345)	40.5 (39.8 to 41.2)	2718	1386
LFM2.5-1.2B-Instruct-8da4w-32k	6	343 (329 to 346)	40.5 (39.9 to 40.9)	2712	1768
LFM2.5-2.6B-8da4w-2k	6	149 (142 to 153)	19.0 (16.7 to 19.5)	6254	1984
LFM2.5-2.6B-8da4w-4k	6	148 (135 to 153)	18.1 (16.5 to 19.5)	6264	2046
LFM2.5-2.6B-8da4w-8k	6	149 (140 to 153)	18.2 (16.7 to 19.5)	6234	2172
LFM2.5-2.6B-8da4w-16k	6	149 (138 to 153)	18.4 (16.6 to 19.5)	6226	2427
LFM2.5-2.6B-8da4w-32k	6	149 (139 to 152)	18.1 (16.2 to 19.5)	6240	2936
Qwen3-1.7B-8da4w-2k	6	200 (193 to 206)	18.0 (16.1 to 18.2)	4680	1758
Qwen3-1.7B-8da4w-32k	6	184 (162 to 203)	16.6 (15.1 to 17.6)	5064	6153

File	Prefill vs smallest window (median ratio, min to max)	Decode vs smallest window	Paired turns
LFM2.5-1.2B-Instruct-8da4w-2k	1.000 (1.000 to 1.000)	1.000 (1.000 to 1.000)	6
LFM2.5-1.2B-Instruct-8da4w-4k	1.007 (0.987 to 1.181)	0.999 (0.927 to 1.190)	6
LFM2.5-1.2B-Instruct-8da4w-8k	1.003 (0.995 to 1.198)	1.011 (0.995 to 1.187)	6
LFM2.5-1.2B-Instruct-8da4w-16k	1.008 (0.992 to 1.190)	1.009 (0.988 to 1.187)	6
LFM2.5-1.2B-Instruct-8da4w-32k	1.013 (0.999 to 1.202)	1.006 (0.993 to 1.184)	6
LFM2.5-2.6B-8da4w-2k	1.000 (1.000 to 1.000)	1.000 (1.000 to 1.000)	6
LFM2.5-2.6B-8da4w-4k	0.995 (0.947 to 1.002)	0.985 (0.946 to 1.005)	6
LFM2.5-2.6B-8da4w-8k	0.997 (0.983 to 1.005)	0.991 (0.943 to 1.000)	6
LFM2.5-2.6B-8da4w-16k	1.003 (0.971 to 1.005)	0.997 (0.938 to 1.005)	6
LFM2.5-2.6B-8da4w-32k	0.997 (0.975 to 1.003)	0.976 (0.943 to 1.005)	6
Qwen3-1.7B-8da4w-2k	1.000 (1.000 to 1.000)	1.000 (1.000 to 1.000)	6
Qwen3-1.7B-8da4w-32k	0.941 (0.793 to 0.982)	0.953 (0.835 to 1.012)	6

Prefill ms is the runtime's prefill of the whole prompt, the engine-side part of time to first token. ms/token is decode time over generated tokens; replies differ per cell, so it is not a paired comparison. Capped counts replies that ran to the token cap (640, or 384 for BFCL). RSS is resident memory right after load and at the end of the run.

How we ran it

Why: To decide whether a full sweep of five windows, five models and four chips (about 50 device hours) was worth running. Five windows of LFM2.5-1.2B ran on the Dimensitys; the other model and phone pairs ran the smallest and largest window.
The size of it: The same weights exported at 2k to 32k. 3 models, 4 phones, 23.7 hours.
Grading: 90 prompts per cell: 30 GSM8K (the last number in the reply must equal the reference), 30 IFEval (every instruction in the prompt must be satisfied, checked by the benchmark's own code), 30 BFCL (exactly one call to the right function with the right arguments). Greedy decoding, thinking off, replies capped at 640 tokens (384 for BFCL). The 2.6B was also rerun at a 2048 cap.
Memory per token: LFM2.5 pays 25 to 33 KB per token of window; Qwen3-1.7B pays 224 KB, so its 32k export needs over 6 GB before the first prompt.
Review: Codex reviewed the method twice, before and after the data: same-file repeats, a paired speed probe separate from the task eval, a three-way identity metric counted over BFCL only, memory after load, a second architecture, incomplete cells shown as such, and conclusions bounded by the observed run variation. All are in what you see here.

What was run, cell by cellWindows and repeats per model, the export recipe, and how long each tookShow

LFM2.5-1.2B: Five windows on the Dimensitys, 16k and 32k twice; 4k and 32k on the cloud chips, 4k twice on the Exynos.
LFM2.5-2.6B: 2k and 32k on the Dimensity, 2k twice; 4k and 32k on the cloud chips; then GSM8K and IFEval again at a 2048 cap on every one of those cells.
Qwen3-1.7B: 2k and 32k everywhere, 2k twice on the Dimensity, 32k twice on the Dimensity to catch the kill.
Exports: Every file loaded at its own exported window. Exports use ExecuTorch 1.4.0's recipes: 8-bit dynamic activations, 4-bit weights in groups of 32, 8-bit embeddings, fp32 KV cache, prefill chunk 2048.

How long it ran

Minutes the model spent generating, summed over every run in that cell; 23.7 hours in all. Loading, cooling and Test Lab queueing came on top. Run on 2026-09-07.

Model	Dimensity	8 Elite	Tensor G5	Exynos
LFM2.5-1.2B	63	12	43	35
LFM2.5-2.6B, 640 cap	129	75	128	117
LFM2.5-2.6B, 2048 cap	146	130	213	195
Qwen3-1.7B	33	19	57	25

Limits

Not matched conditions: The Dimensitys ran in hand, cooled below 42 C before each probe load. The 8 Elite, Tensor G5 and Exynos 2400 ran racked on Firebase Test Lab, where phones throttle within about 40 seconds, and had no paired probe: their suite timings are per-cell medians over different replies and do not support a speed claim either way.
Follow-up: The reruns checked every cell of this matrix the same day.

Sources

The LFM2.5-1.2B and LFM2.5-2.6B exports were also published on Hugging Face, each window a file.

The note The tables Raw and graded reports Qwen3-1.7B exports on Hugging Face Source on GitHub

Does the exported window matter?

What we found

The results

Every chart, one condition at a time

Memory after load

Speed, paired probe

Grades by window

Identical replies

What else the matrix found

Every number

How we ran it

How long it ran

Limits

Sources

Keep reading