Prewalk: swap to a cheap model after the first edit (caveman version)
Contents
Coding agent spend most token on reading file, not writing. About 9 of 10. Writing change is cheap part.
Why “plan then hand off” fail
Plan sound smart: big model plan, small model do work.
Big model read whole codebase, write plan. Give plan to small model. Small model start work. Then small model read same files again.
Plan is tiny summary of huge reading. Small model cannot act on summary alone. Must rebuild same understanding, so open same files. Reading paid twice. Handoff make worse.
Prewalk: hand off context, not plan
Keep both model in SAME conversation. Swap at one moment: first real edit.
- Big model explore, write to-do list, make first working edit. Real change, not stub.
- At that edit, swap to small model, same conversation. Delete “explore and plan” instruction.
- Small model inherit all read files, the list, and one working edit. Finish list, no re-reading.
Hand off is whole context, not tiny plan. Small model never rebuild understanding. Understanding already in conversation.
Reported result: big explorer + cheap executor keep ~92% of quality at ~half cost, faster.
Why first edit is right moment
- Quality: first working edit mean hard thinking done. Rest is execution, cheap model fine. Swap earlier = half-formed plan. Swap later = overpay.
- Cost: big model stop in calm confident phase, not stuck desperate phase. Cheap model inherit context that know where it go, less flail.
Plain picture
Senior and junior share one desk. Old way: senior read everything, leave sticky note, go. Junior read note, then read everything again. Prewalk: senior read, write list, make first fix while junior watch, then junior take same seat. Junior already saw everything. Just continue.
One thing people get wrong
Do NOT carry big model memory to small model. Different model, different internal memory shape. Cannot reuse.
So small model read conversation text once from top. That is fast single pass - words already written. NOT same as re-generating exploration by opening files and thinking again (expensive). Prewalk pay one cheap re-read, not full redo. On vLLM, prefix caching make later steps cheap too.
Where you run it
Prewalk not special model or setting. Live in loop you write. Need one thing: chat is stateless - every request carry whole conversation + model name. Cloud API and local server (llama.cpp, vLLM) both work this way.
Because you resend conversation each turn, you can point next turn at different model AND edit conversation first. Hosted black-box agent cannot edit own history. Your own loop can. So prewalk easier on raw local server than inside locked agent.
What me build
Two routes.
- Cloud: big model explore + land first edit, then loop swap model name to cheaper one, delete planning instruction, keep conversation.
- Local: two servers. On first edit, rewrite first message from “explore and plan” to “just execute list”, send rest to small model. One line of code - because you own conversation.
Both share one rule for “first edit”, so swap fire same moment.
Where help, where not
| Situation | Help? | Why |
|---|---|---|
| Deep feature, many files | Yes, lot | Reading dominate; kill double-read |
| Small self-contained change | Not really | Almost no exploration to hand off |
| Many small tasks in parallel | No | Prewalk is one conversation; parallel need separate contexts |
| Two local models similar quality | Thin | Small cost gap |
Sweet spot: single meaty change that force lot of reading before first safe edit.
Maxim
Pay big model once, for reading. Never pay for same reading twice.
Recap
- Agent spend ~90% of token reading, not writing.
- “Plan then hand off” fail: cheap model re-read everything.
- Prewalk keep both model in one conversation, swap at first edit, hand over whole context.
- Cheap model re-read conversation once (fast), not re-generate exploration (expensive).
- Work on cloud and local; easier local, where you edit own conversation.
- Best for deep single feature. Not tiny change, not parallel fan-out.
Sources
- Stencil, Prewalk