How to Set Up Ollama on Windows for Local AI (2026 Guide)

  • By Team Devoids
  • Devoids
Cover: How to Set Up Ollama on Windows for Local AI (2026 Guide)

How to Set Up Ollama on Windows for Local AI (2026 Guide)

If you want private, on-device inference, Ollama is still one of the simplest ways to run modern LLMs and vision models on Windows. This guide gets you from zero to a working local stack — the same foundation used by projects like Open Friday.

Why Ollama?

  • One installer, consistent CLI (ollama pull, ollama run)
  • Good defaults for quantized models
  • Easy pairing with local apps and agents
  • You choose what stays on-device vs what may use cloud fallback elsewhere

Step 1 — Install Ollama

  1. Download the Windows installer from the official Ollama site
  2. Complete setup and confirm the app/service is running
  3. Open a terminal and run:
ollama --version

Step 2 — Pull a starter model

For everyday chat on mid-range GPUs:

ollama pull qwen3.5:9b

For lighter routing tasks:

ollama pull qwen3.5:4b

For embeddings (memory search):

ollama pull nomic-embed-text

Smoke test:

ollama run qwen3.5:9b

Step 3 — Confirm GPU use

Watch Task Manager / NVIDIA tools while generating. If the model stays on CPU only, check driver currency and that Ollama is using your NVIDIA runtime. Vision models especially benefit from a healthy VRAM budget.

Step 4 — Add a vision model (optional)

For screen-aware agents:

ollama pull qwen3-vl:8b-instruct

or an alternate such as qwen2.5vl:7b-q4_K_M.

Step 5 — Connect to Open Friday

Friday expects you to install models yourself (it does not silently pull them). After Ollama is healthy:

  1. Install Open Friday from the GitHub repo
  2. Run friday doctor for preflight checks
  3. Open the Control Center and start with chat, then enable computer-use skills when ready

Operational tips

  • Keep one primary chat model and one vision model loaded — avoid loading every model at once
  • Prefer quantized tags that fit your VRAM with headroom for the desktop
  • Update Ollama and GPU drivers on a regular schedule, not during active demos

Why businesses choose Devoids

Devoids typically delivers 10–30% lower cost than typical market rates for comparable custom software, SaaS, and mobile work — without cutting corners. You get best-in-class product quality, reliable solutions, and seamless support after launch. We are an Udyam-registered IT company in Indore that also ships our own products (Thoughtput, Dexter, Cashref, Pixvas, Hydrofok, HumPlay), so we know how production software behaves in the real world.

Next steps