When the KV Cache Cannot Roll Back: Prefix Reuse on Android.

An agent prompt on a phone can open with about 2,050 tokens of harness, 1,700 of them tool definitions. On a Dimensity 9500s, a 1,036-token prefix takes 9.3 s to prefill on LFM2.5-1.2B and 45 s on SmolLM3-3B; faster decode does nothing to that wait. We solved it with prefix reuse, built twice because our open source app ships two runtimes with different cache constraints.

ExecuTorch's Android API exposes prefill and full reset, but no partial rollback. We prefill the shared head at model load, then feed only the unmatched suffix while the prompt is an exact extension. On our llama.cpp path, LFM2.5's recurrent state cannot rewind without per-token snapshots, so we snapshot the warmed 12 MB state and restore it.

Correctness depends on byte identity. The date is pinned per conversation, history replays the engine's own bytes, changing reasoning mode invalidates the prefix, and truncated replies are never reused. The talk shows the cache rules, the failures and the regression tests; measurements and logs are public.