Running local LLMs on a laptop
No data center needed. Modern laptop, 16-32 GB RAM, half-decent GPU → handles writing, code review, agent workflows.
Why local
→ Privacy - prompts never leave the machine. → Latency - no network round-trip, no rate limits, no cold starts. → Cost - zero per-token after setup. → Control - pick model, quantization, context window.
Hardware floor
| Component | Minimum | Comfortable |
|---|---|---|
| RAM | 16 GB | 32 GB+ |
| GPU VRAM | 8 GB | 16 GB+ |
| Disk | 20 GB | 100 GB+ |
| OS | Any | Linux/WSL |
Runtimes (2026)
- llama.cpp / llama-server - fastest on CPU, GGUF quantizations, tiny footprint.
- Ollama - easiest onboarding, one-line model pulls, OpenAI-compatible API built in.
- LM Studio - GUI-first, great for non-CLI users.
→ All three speak OpenAI chat-completions API. Any GPT-4 client → works with local model.
Sweet-spot model
Early 2026: Qwen 3.5 14B at Q4_K_M.
- ~9 GB on disk
- ~10 GB VRAM
- Capable for agent tasks
- Fast enough you won’t watch it type
# Ollama example
ollama pull qwen3.5:14b
ollama run qwen3.5:14b "Write a haiku about SQLite."
Next steps
→ Point existing Claude Code / Cursor config at http://localhost:11434/v1.
→ Benchmark a few quants - Q4 usually best speed/quality trade-off.
→ See /experiments/local-agent-benchmarks/ for real numbers.
Lesson: the laptop already has enough silicon. The bottleneck is picking a small enough model and a runtime that gets out of the way.