Local LLM VRAM Finder (2026)
Find a practical local AI model for 8GB, 12GB, 16GB, 24GB or 32GB GPUs. Compare published model download sizes, choose by workload, and leave room for context and runtime memory—not just weights.
Illustrative only. Download sizes are not measured peak VRAM; buffers, KV cache and GPU offload vary by runtime.
01 / INTERACTIVE MODEL SELECTOR
Find a local LLM for your GPU
Change the inputs to compare source-linked model candidates. No account or tracking required. The browser reads our locally hosted, source-checked catalog JSON.
Choose a task and context for both search directions. No GPU benchmark or tested compatibility is implied.
Both directions use the same Primary task and Planning context. Reverse lookup never silently changes the workload. Results are screening candidates, not quality rankings.
Loading sourced model catalog…
This calculator uses a conservative planning allowance (2 / 3 / 4.5 / 7 GB for 4K / 8K / 16K / 32K, plus 1 GB for image tasks). These are SXF heuristics, not measured KV-cache sizes for each architecture. Ollama model tag sizes are download sizes. Check actual GPU residency with ollama ps.
Matched model candidates
Select settings to compare published sizes.Enable JavaScript to use the interactive finder. The source-linked 13-row reference snapshot and VRAM picks below remain available without it.
Explore SXF model profiles and AI model comparisons. Exact matching model pages are linked from individual results where SXF has a verified profile.
THE SHORT ANSWER
Which local LLM fits your GPU?
For an 8GB GPU, start with a compact 4B–8B quantized model, such as Qwen3 4B or Qwen3 8B at a modest context length. At 12GB–16GB, the published files for Qwen3 14B, Qwen2.5-Coder 14B and Gemma 3 12B become candidates. With 24GB–32GB, larger quantized options such as Qwen3 30B, Qwen2.5-Coder 32B and Gemma 3 27B are worth evaluating. None of these statements guarantees full-GPU loading at your chosen context length.
This is a hardware-selection guide, not a claim that one model wins all benchmarks. To install a runtime, use our Ollama vs LM Studio vs vLLM guide; for GGUF, AWQ and quantization tradeoffs, see LLM Quantization Explained.
02 / HARDWARE FIRST
Best local LLM candidates by GPU VRAM
Starting points based on published quantized download sizes, not head-to-head performance measurements.
Compact and capable
Qwen3 8B for general chat; Qwen2.5-Coder 7B for code; Qwen3-VL 4B for images. Use short context at first.
Explore 8GB options →Room for 12B–14B
Qwen2.5-Coder 14B or Gemma 3 12B can be candidates with conservative memory settings; avoid assuming huge context fits.
Explore 12GB options →A balanced local setup
Qwen3 14B, Qwen2.5-Coder 14B and Gemma 3 12B provide choices across chat, coding and multimodal tasks.
Explore 16GB options →Larger quantized models
Consider Qwen3 30B, Gemma 3 27B or Qwen2.5-Coder 32B. Long context and concurrency can exceed the remaining headroom.
Explore 24GB options →More space for context
The same larger model files leave more memory for runtime buffers, image inputs and longer context. Still benchmark your workload.
Explore 32GB options →03 / VERIFIED FILE SIZES
Local AI model comparison: download sizes
Tagged files in Ollama's official model registry; sizes rounded as displayed on the cited tag listings, checked 24 October 2026.
| Model tag | Size | Workload | Context caveat | Official source |
|---|---|---|---|---|
qwen3:4b | 2.5 GB | General chat | 4K start recommended | Ollama ↗ |
qwen3:8b | 5.2 GB | Chat, reasoning | 8GB cards: keep context modest | Ollama ↗ |
qwen3:14b | 9.3 GB | Chat, reasoning | Check 12GB headroom carefully | Ollama ↗ |
qwen3:30b | 19 GB | Chat, agents | 24GB cards: limited overhead | Ollama ↗ |
qwen2.5-coder:7b | 4.7 GB | Coding | Good smaller-file candidate | Ollama ↗ |
qwen2.5-coder:14b | 9.0 GB | Coding | 12GB cards: short-context test | Ollama ↗ |
qwen2.5-coder:32b | 20 GB | Coding | Prefer 24GB+ with short context | Ollama ↗ |
gemma3:4b | 3.4 GB | Chat, images | Check vision memory overhead | Ollama ↗ |
gemma3:12b | 8.2 GB | Chat, images | Vision adds image-processing cost | Ollama ↗ |
gemma3:27b | 17 GB | Chat, images | Context may limit full offload | Ollama ↗ |
qwen3-vl:4b | 3.3 GB | Image understanding | Allow extra headroom for images | Ollama ↗ |
qwen3-vl:8b | 6.2 GB | Image understanding | 8GB may require CPU offload | Ollama ↗ |
qwen3-vl:30b | 20 GB | Image understanding | Images and context add overhead | Ollama ↗ |
A 5.2 GB Ollama artifact does not mean 5.2 GB is sufficient GPU memory. The exact tag, architecture, GPU backend, context, number of parallel sessions and offloading determine runtime use. Published sizes can change when maintainers retag models.
04 / USER-CAPTURED GPU EVIDENCE
Measured on a real Tesla T4 session
Reviewed user-submitted evidence, not a performance guarantee or a minimum GPU memory requirement.
Qwen3-VL 4B Instruct — tested image prompt
64.23 generated tokens/second arithmetic mean of three post-warmup Ollama runs (64.3089, 64.2733, 64.1013 tok/s). This is generation-only, calculated as eval_count × 10⁹ / eval_duration_ns (Ollama durations are nanoseconds), not end-to-end image processing or first-response latency.
Vision check: passed this specific fixture. All three measured answers correctly described a centered red square on a blue background. A single simple image does not establish general image-recognition accuracy.
Exact test: qwen3-vl:4b-instruct, quantization Q4_K_M, Ollama 0.40.10, Tesla T4 (15,360 MiB), GPU driver 580.82.07, requested context 4,096, one warmup and three measured trials. The qwen3-vl:4b thinking variant is not verified by this result.
Model digest: 84f7ed8042fa7cdbd902f7dcb7d0370e032095047cc3244179f263b77b5bc318.
Memory numbers mean different things
4,455 MiB: highest observed entire-GPU used memory from 200 ms nvidia-smi sampling; baseline after unloading was 0 MiB. It is not independent model-only VRAM or a confirmed hardware minimum.
3,406.42 MiB: model GPU residency reported by Ollama (/api/ps size_vram); not an independently measured allocation or guaranteed complete residency.
3,337,225,651 bytes: Ollama-reported model file size; not GPU usage. Finder planning allowances elsewhere on this page remain estimates, not measurements.
Evidence status: user-supplied files and SHA-256 checked; source environment not independently attested. Original ZIP is not publicly hosted here, so public reproduction and independent verification remain limited.
This measured example does not change the model-finder's estimated GPU tier, its compatibility labels, or the unapproved public benchmark registry. No universal fit, quality, speed or model-exclusive VRAM is claimed.
05 / ENGINEERING REALITY
How much VRAM does a local LLM actually need?
Your GPU must hold more than model weights. Quantized file size is a useful starting point, but loading a local model also consumes space for execution buffers and a KV cache. A longer context or more simultaneous requests can increase memory demand. Multimodal inputs may add image-processing overhead.
For rough initial planning, this page compares the published Ollama download size + an explicit SXF headroom allowance. It is deliberately not a hardware benchmark. A model shown as a candidate may still spill onto system RAM or refuse to load at your chosen settings.
Why 4-bit vs 8-bit matters
A lower-bit quantized model generally uses fewer bytes for stored weights, but GGUF formats add metadata and different quantizers use different layouts. Q4_K_M, Q8_0, AWQ and GPTQ are not interchangeable in fidelity, compatibility or speed. Read our quantization guide and VRAM calculator before choosing a different variant.
GPU VRAM versus system RAM
On an NVIDIA or AMD discrete GPU, VRAM is separate from system RAM. Some runtimes can offload layers to the CPU, but that usually affects throughput and latency. Apple Silicon uses unified memory, so these discrete-GPU bands should not be interpreted as equivalent Mac RAM requirements.
SXF MEMORY WORKSHEET
Worked example: Qwen3 8B
Ollama lists a 5.2 GB download. With SXF's 3 GB illustrative allowance for 8K context, the worksheet produces 8.2 GB. This does not prove the model cannot run on an 8GB GPU; it flags the need to test a shorter context, offloading or an alternate quantization.
Actual memory use is architecture- and configuration-dependent; the allowance is a screening tool, not a model-specific KV cache formula.
Ollama context-length documentation ↗06 / FROM DECISION TO EXECUTION
Run your selected local model with Ollama
Minimal example for supported desktop platforms. Choose a verified model tag from the table.
- 01
Install Ollama
Follow the official quickstart for your operating system. Download weights only from sources you trust; check model terms before business use.
- 02
Start with a smaller, known tag
For a first coding test on an 8GB GPU, begin with the published 7B coding variant, or use the selector above for your workload.
ollama run qwen2.5-coder:7b - 03
Check what loaded onto the GPU
While the model is running, inspect the processor/offload information. Record the actual memory behavior and compare it against the planning estimate.
ollama ps - 04
Increase context carefully
Start at a modest context length, then evaluate latency, quality and out-of-memory behavior at 8K, 16K or higher. Larger advertised context windows do not mean every GPU can serve them at full length. See the complete local deployment guide for runtime and API settings.
07 / PRACTICAL QUESTIONS
Local LLM GPU & VRAM FAQ
What is the best LLM for an 8GB GPU?
There is no universal performance winner. For a source-backed starting point, Qwen3 4B (2.5 GB published download), Qwen2.5-Coder 7B (4.7 GB) and Qwen3 8B (5.2 GB) cover chat and code. With Qwen3 8B, reduce context or plan for offload if your system does not have enough runtime headroom.
Can a 12GB GPU run a 14B model?
Some 14B quantized artifacts are around 9.0–9.3 GB, leaving only a few GB before buffers, KV cache and OS use. A modest context configuration may work, but it is not guaranteed. The exact tag and runtime matter more than parameter count alone.
Which local coding LLM should I run with 16GB VRAM?
Qwen2.5-Coder 14B is a practical candidate with a published 9.0 GB Ollama download. Test it on representative repository edits and error fixing; a smaller 7B model may still be preferable for latency. These are workload suggestions, not head-to-head benchmark rankings.
Can I run a 30B model on a 24GB GPU?
Some quantized 30B model tags have download sizes around 19–20 GB. They can be candidates for 24GB cards at modest context, but KV cache and runtime buffers reduce available headroom. Long context or concurrency may force partial CPU offloading.
How is an Ollama model file size different from VRAM use?
The tag size describes downloaded model data, while runtime VRAM includes loading format, execution workspace, KV cache and sometimes image processing. Use ollama ps to see whether Ollama places a running model on GPU, CPU or both.
Can I run these models on a laptop or Apple Silicon Mac?
Many local models work on compatible laptops, but laptop GPU TGP, thermals, OS and available shared/system memory matter. Macs have unified memory rather than the dedicated VRAM bands in this page. See the full local inference setup guide for Apple Silicon and CPU-only considerations.
Is 4-bit quantization always the best choice?
No. A smaller quant may enable a bigger model or a larger context, but can affect quality and may behave differently across inference engines. Compare on your own prompts with the same runtime and settings. Learn about GGUF and 4-bit vs 8-bit.
Are these local AI models allowed for commercial use?
Licenses differ. The Qwen3 model card lists Apache-2.0, whereas Google Gemma models use Gemma-specific terms. Check the exact publisher card, associated files and current license before production use. The selector does not grant licensing rights.
SOURCE LOG / NO INVENTED BENCHMARKS
Methodology and primary references
The interactive catalog includes 191 source-linked artifacts as checked on : 178 GGUF single-file variants with integer byte sizes fetched from 18 allowlisted Hugging Face repositories, plus 13 Ollama tags with publisher-rounded download sizes. These are variants and quantizations, not 191 independent model families. The 13-row comparison table above remains a static editorial Ollama reference, not the complete interactive catalog. Source-linked file sizes do not establish a complete multimodal download, runtime compatibility, peak VRAM, inference speed, or model quality. The GPU figures are SXF illustrative file-size-plus-headroom estimates, not benchmarks. Candidate badges describe estimated memory conditions and disclose aggressive quantization or unverified vision assets; they are not quality rankings. The model catalog as a whole has not undergone independent model-by-model GPU-memory, throughput, or quality benchmarking. The isolated Tesla T4 user-supplied Qwen3-VL 4B Instruct image test above is separately reviewed evidence for one exact configuration and fixture, not a benchmark or compatibility proof for the other listed variants. See the catalog source links on each result and the Hugging Face repository index for additional model-card evidence. Individual source licenses and supported runtimes require review before deployment.
KEEP EXPLORING SXF