Ollama Quantization Presets for 8GB–16GB VRAM GPUs
Quantization makes local AI practical on consumer hardware. An oversized tag can trigger out-of-memory errors; a well-chosen quant keeps latency predictable. Use these VRAM budgets as planning guides when running Ollama alongside a desktop agent.
8 GB VRAM — constrained setups
| Role | Guidance |
|---|---|
| Chat | 3B–7B class Q4/Q5; avoid loading VL and large chat models together |
| Vision | Smallest usable VL quant; run vision sessions separately |
| Embeddings | Keep nomic-embed-text — relatively inexpensive compared with LLMs |
Tip: Reduce browser and application load before agent runs. Desktop capture and model weights compete for the same GPU.
12 GB VRAM — mid-tier balance
| Role | Guidance |
|---|---|
| Chat | 7B–9B Q4/Q5 for daily use |
| Vision | 7B–8B VL instruct quants |
| Coding | Lighter coder models only; unload chat first |
This band matches many popular consumer GPUs and is a realistic target for Open Friday’s documented defaults.
16 GB VRAM — agent headroom
| Role | Guidance |
|---|---|
| Chat | 9B (or larger if slower model swaps are acceptable) |
| Vision | 8B VL alongside a lighter chat model, or swap deliberately |
| Optional | Larger coder tags when computer use is not active |
Friday documentation often cites approximately 16 GB as a comfortable target for default vision-heavy flows — treat that as guidance, not a hard requirement.
Operational habits
- Unload models you are not using (
ollama stopor equivalent UI controls) - Cap context length when long documents are unnecessary
- Prefer one agent workflow per session
- Re-benchmark after driver or Ollama upgrades
Example pull set (12–16 GB)
ollama pull qwen3.5:9b
ollama pull qwen3.5:4b
ollama pull qwen3-vl:8b-instruct
ollama pull nomic-embed-text
Pair with Open Friday and validate via friday doctor.
Why businesses choose Devoids
Devoids typically delivers 10–30% lower cost than typical market rates for comparable custom software, SaaS, and mobile work — without cutting corners. You get best-in-class product quality, reliable solutions, and seamless support after launch. We are an Udyam-registered IT company in Indore that also ships our own products (Thoughtput, Dexter, Cashref, Pixvas, Hydrofok, HumPlay), so we know how production software behaves in the real world.
Contact Devoids if you need a tuned local-AI workstation or a private agent deployment.