Pick your graphics card and memory. We’ll check 20 popular open models — Llama, Qwen, Mistral, Gemma, DeepSeek and Phi — and tell you what fits, what’s tight, and what to run first. Every result comes with a copy-paste Ollama command.
—
—
—
If weights + KV cache fit in VRAM with headroom, you get full speed. “Tight” still runs, but a longer context or another app can push it over.
Ollama and LM Studio can split a model between GPU and RAM. It works, but expect roughly a quarter of the full-GPU speed.
Q4_K_M is what most local users actually run day to day. Step up to Q8 only if you have VRAM to spare and want the closest-to-original quality.
It takes under a minute to find out — then come back and pick it above.
Ctrl + Shift + Escnvidia-smi and read the MiB totalrocminfo or check glxinfoThree proven price points. Click any card to load that exact machine into the checker above and see its full model list.
Still the cheapest way into serious local AI. Runs every 7–8B model at full speed and 14B models at Q4. Pair with 32 GB RAM.
16 GB plus modern bandwidth makes 27–32B models at Q4 genuinely comfortable, with fast token speeds. The current sweet spot for a new build.
Unified memory is the cheat code for big models: a 64 GB Mac runs 70B at Q4 quietly, with no fans screaming and no separate GPU to buy.
Short answers, no hype.
At Q4_K_M with an 8K context, budget roughly 6 GB for 7–8B, 10 GB for 14B, 22 GB for 32B and 47 GB for 70B — weights plus KV cache plus overhead. Longer context pushes each figure up; Q8 roughly doubles the weights portion.
Yes, slowly. Small models (3–8B at Q4) run on CPU + RAM at a few tokens per second — fine for trying things out or light chat, painful for long documents or coding sessions. A used 12 GB GPU is the single biggest upgrade you can make.
Quantization stores model weights in fewer bits. Q4_K_M is about 40% smaller than Q8_0 with a quality drop most people can’t detect in everyday chat and coding. Below Q4 the drop becomes noticeable; FP16 is the unquantized original and rarely worth it at home.
The KV cache stores the conversation the model is “holding in mind.” It grows with every token of context, and on big models at 128K it can rival the weights themselves. If a model barely doesn’t fit, shortening context is the first fix to try.
Both run the same GGUF models. Ollama is fastest to start (one command, shown on every card here) and best for scripting and apps. LM Studio has a friendlier graphical interface and easier model browsing. Pick either — your fit results above apply to both.
Yes — inference happens entirely on your machine. Prompts and documents never leave your computer, which is the main reason people run local models for sensitive work, coding on private repos, or offline use.