Home / Calculators / Local AI
Which AI model runs on your computer?
Tell me how much memory your Mac or PC has and I will tell you which open-weights models fit in memory, with which quantization, and what speed range comes out of an illustrative model. Everything is calculated in your browser, with no account and nothing saved.
Methodology: memory is arithmetic (weights + context cache + runtime) using data from the official model cards and config.json files; whether your engine supports each model is not verified here; and speed is a modelled range, illustrative and not calibrated with real tests. Sources and assumptions.
Local AI model calculator
Calculating…
- 1. Memory: arithmetic
- Weights + KV cache + runtime against your usable memory. It is a calculation, but with assumptions (runtime, the Mac's memory fraction, the real size of the file you download).
- 2. Compatibility: not verified here
- A model fitting does not mean your engine (llama.cpp, MLX, Ollama…) already supports its architecture or format. We did not test any combination: check the model card and the engine's release notes.
- 3. Speed: heuristic
- A modelled range based on the spec-sheet bandwidth, with an assumed efficiency of 0.6 and a ±25% band that we set ourselves. It is illustrative: it is not calibrated with real tests and is not a measured margin of error.
Memory needed per model and quantization (8K context).
Weights + 8K context cache in f16 + 0.8 GB of runtime, in GB. It is what has to fit in your VRAM or in your Mac's usable memory.
| Model | Parameters | Q4_K_M | Q5_K_M | Q6_K | Q8_0 | FP16 |
|---|---|---|---|---|---|---|
| Llama 3.2 1B | 1.2 B | 1.7 | 1.8 | 1.9 | 2.2 | 3.3 |
| Llama 3.2 3B | 3.2 B | 3.5 | 3.8 | 4.1 | 4.8 | 7.6 |
| Qwen3.5 4B | 4.7 B | 3.7 | 4.1 | 4.6 | 5.6 | 9.7 |
| Gemma 4 E2B | 5.1 B | 3.8 | 4.3 | 4.8 | 5.9 | 10.4 |
| Gemma 4 E4B | 8.0 B | 5.6 | 6.3 | 7.1 | 8.9 | 15.9 |
| Llama 3.1 8B | 8.0 B | 6.3 | 7.1 | 7.9 | 9.7 | 16.7 |
| DeepSeek-R1-0528 Qwen3 8B | 8.2 B | 6.5 | 7.3 | 8.1 | 10.0 | 17.1 |
| Ministral 3 8B | 8.9 B | 6.9 | 7.7 | 8.6 | 10.6 | 18.4 |
| Qwen3.5 9B | 9.7 B | 6.5 | 7.4 | 8.4 | 10.5 | 19.0 |
| Gemma 4 12B | 12.0 B | 8.0 | 9.1 | 10.3 | 13.0 | 23.5 |
| Ministral 3 14B | 14.0 B | 9.9 | 11.3 | 12.7 | 15.8 | 28.0 |
| Phi-4 14B | 14.7 B | 10.7 | 12.0 | 13.5 | 16.8 | 29.6 |
| gpt-oss 20B | 20.9 B (3.6 B active) | 12.2 MXFP4 native | — | — | — | — |
| Devstral Small 2 24B | 24.0 B | 15.7 | 17.9 | 20.3 | 25.8 | 46.7 |
| Gemma 4 26B A4B | 25.8 B (3.8 B active) | 15.8 | 18.2 | 20.8 | 26.6 | 49.2 |
| Qwen3.8 27B | 27.8 B | 17.1 | 19.7 | 22.5 | 28.7 | 53.0 |
| Gemma 4 31B | 31.3 B | 20.0 | 22.9 | 26.0 | 33.1 | 60.4 |
| DeepSeek-R1-Distill-Qwen 32B | 32.8 B | 21.4 | 24.5 | 27.8 | 35.2 | 63.8 |
| Qwen3.6 35B A3B | 36.0 B (3.0 B active) | 21.4 | 24.8 | 28.4 | 36.5 | 67.9 |
| Llama 3.3 70B | 70.6 B | 43.4 | 50.1 | 57.2 | 73.1 | 134.7 |
| Qwen3-Coder-Next 80B A3B | 79.7 B (3.0 B active) | 46.3 | 53.8 | 61.8 | 79.8 | 149.3 |
| GLM-4.5-Air | 110.5 B (12.0 B active) | 65.1 | 75.5 | 86.6 | 111.5 | 208.0 |
| gpt-oss 120B | 116.8 B (5.1 B active) | 60.1 MXFP4 native | — | — | — | — |
The gpt-oss models are distributed already quantized in MXFP4, which is why they only have one figure. MoE (mixture of experts) models need the memory for all their experts even though they only read the active ones per token.
How it is calculated.
Three pieces of memory and a speed heuristic. Nothing more.
- 1. Model weights
weights = parameters × bits per weight ÷ 8. A model with 8 billion parameters in Q4_K_M (4.89 bits per weight) weighs about 4.6 GB. The bits per weight come from the table in the llama.cpp documentation. - 2. Context cache (KV)
KV = 2 × layers × KV heads × head size × tokens × bytes. It grows with the context you use. Models with hybrid or sliding-window attention (Qwen3.5 onwards, Gemma 4, gpt-oss) only keep a full cache in some of their layers, and the calculator takes that from their config.json. - 3. RuntimeAbout 0.8 GB for compute buffers and the engine itself. Weights + KV + runtime is the memory you need.
- Available memoryOn a Mac, about 70% of unified memory (what macOS gives the GPU by default; if you raise the limit, it is your memory minus 4 GB). On a PC with a GPU, the VRAM; if the model does not fit you can offload to RAM, which is slower. On a PC without a GPU, your RAM minus about 4 GB for the system. Here 1 GB is 1024³ bytes, as with RAM.
- Speed (heuristic)
tokens/s ≈ bandwidth ÷ bytes read per token × 0.6. In a dense model the whole model is read for each token; in a MoE only the active parameters. The 0.6 is our assumption about what fraction of the spec-sheet bandwidth is actually used, not a measurement, and the ±25% is an illustrative band that we chose. That is why we call it a modelled range, not calibrated with real tests.
What matters most: RAM, VRAM or bandwidth.
Capacity decides whether it fits. If the weights plus the cache do not fit in the memory the graphics processor can see, the model does not run fast, or does not run at all. That is why a card's VRAM matters more than its compute power.
Bandwidth decides how fast it goes. Generating each token requires reading the active weights from memory. That is why an RTX 4090 (1008 GB/s) generates much faster than a Mac M4 (120 GB/s) with the same model, as long as it fits in its VRAM.
A Mac's unified memory is a single pool shared by CPU and GPU: there is no need to copy the model between RAM and VRAM, and a Mac with 64 GB or more can load models that no consumer card can handle. In exchange, its bandwidth is lower than a high-end GPU's.
A PC's RAM works for offloading, but it is the bottleneck: dual-channel DDR5 (about 90 GB/s) is several times slower than VRAM. If the model does not fit on the card, a mixture of experts (MoE) usually holds up better because it only touches a fraction of the weights per token.
Frequently asked questions.
- Can I run a 70B model in 16 GB?No. A 70B model in Q4_K_M weighs about 40 GB in weights alone, and the most aggressive quantizations (2-bit) still come to around 20 GB and lose quality. With 16 GB the realistic range is 8 to 14B models. The calculator tells you which ones fit with your setup.
- What is quantization?It means storing the model's weights with fewer bits so they take up less space. The original uses 16 bits per weight (FP16); Q4_K_M uses about 4.9 and Q8_0 about 8.5. Each step saves memory and speeds up reading, in exchange for some quality loss that is barely noticeable at Q8 and usually acceptable at Q4 for chat and code. Below 4 bits it becomes more noticeable.
- Mac or NVIDIA GPU for running local AI?It depends on what you need. An NVIDIA GPU gives more speed per GB, but its VRAM is limited (8 to 32 GB on the cards in the list). A Mac with unified memory can load much larger models because it shares all its memory with the GPU, although at lower bandwidth and therefore fewer tokens per second on the same model. If your model fits in VRAM, the card is faster; if it does not fit, a Mac with plenty of memory may be the only comfortable option.
- What happens if the model does not fit in VRAM?The engine (llama.cpp, for example) can leave some of the layers in system RAM and process them with the CPU. It works, but RAM is much slower than VRAM (dual-channel DDR5 is around 90 GB/s; an RTX 4070, 504 GB/s) and the speed drops to a pace close to RAM's. With MoE models it is better to keep only the experts in RAM, which performs better than splitting layers.
- "Open source" or "open weights"? What license does each model have?Almost all of these models are "open weights": you can download the weights and run them, under the terms of their license. Llama uses Meta's own license, which is not an OSI-approved license. Others use Apache-2.0 or MIT, which are OSI licenses, but that is not enough to call them "open source" under the OSI's AI definition, which also asks for information about the training data and the code. That is why on this page we write "open weights" and show each model's license. Read it before using a model in a product.
- Is running a model locally really private?It can be, but it depends on the tool and how you configure it. If the model runs on your computer and the app is not connected to a remote provider, your messages do not leave it. Many apps, however, mix local models with cloud APIs, web search or telemetry. Check the tool's settings and, if it matters to you, try it with the network disconnected or with a firewall that shows outgoing connections. How to check a repo before installing it.
Methodology and sources.
This page is a modelled estimate, not a lab test: we did not measure any model on any machine. Memory comes from a calculation with data from official model cards (parameters, layers and KV heads from the Hugging Face config.json; bits per weight from the llama-quantize README; bandwidth from each machine's spec sheet). Compatibility with your engine (llama.cpp, MLX…) is not verified. Speed is a modelled, illustrative range, not calibrated with real tests: its efficiency of 0.6 and its ±25% band are our assumptions, not a measured margin of error, and in practice the engine, the driver, the context and the exact file you download all matter. Models with vision may need somewhat more memory for their image encoder. If a figure could not be verified, the model or machine was left out or marked as approximate. Data checked October 29, 2026. Missing a model or a machine? Write to me and I will check it.
Memory bandwidth per machine
Theoretical peak from the spec sheet, in GB/s. The status column says how well backed each figure is.
| Machine | GB/s | Source | Status |
|---|---|---|---|
| Apple M2 | 100 | apple.com/newsroom/2022/06/apple-unveils-m2-with-breakthrough-performance-and-capabilities/ | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| Apple M3 | 100 | support.apple.com/en-us/117735 | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| Apple M4 | 120 | apple.com/newsroom/2024/10/apple-introduces-m4-pro-and-m4-max/ | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| Apple M5 | 153 | apple.com/newsroom/2025/10/apple-unleashes-m5-the-next-big-leap-in-ai-performance-for-apple-silicon/ | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| Apple M2 Pro | 200 | apple.com/newsroom/2023/01/apple-unveils-m2-pro-and-m2-max-next-generation-chips-for-next-level-workflows/ | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| Apple M4 Pro | 273 | apple.com/newsroom/2024/10/apple-introduces-m4-pro-and-m4-max/ | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| Apple M5 Pro | 307 | support.apple.com/126318 | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| Apple M2 Max | 400 | apple.com/newsroom/2023/01/apple-unveils-m2-pro-and-m2-max-next-generation-chips-for-next-level-workflows/ | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| Apple M4 Max (40-core GPU) | 546 | support.apple.com/en-us/121553 | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| Apple M5 Max (40-core GPU) | 614 | support.apple.com/126319 | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| GeForce RTX 3060 12 GB | 360 | nvidia.com/en-us/geforce/news/game-on-you-asked-we-answered-qa/ | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| GeForce RTX 4060 8 GB | 272 | techpowerup.com/review/asus-geforce-rtx-4060-dual-oc/ | Calculated: 17 Gbps × 128 bits ÷ 8, using the data the TechPowerUp review gives. |
| GeForce RTX 4060 Ti 16 GB | 288 | — | approx., no official source |
| GeForce RTX 4070 12 GB | 504 | — | approx., no official source |
| GeForce RTX 4090 24 GB | 1008 | techpowerup.com/review/nvidia-geforce-rtx-4090-founders-edition/ | Third-party review (TechPowerUp), not from the manufacturer; figure checked. |
| GeForce RTX 5060 Ti 16 GB | 448 | nvidia.com/en-us/geforce/graphics-cards/compare/ | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| GeForce RTX 5070 12 GB | 672 | nvidia.com/en-us/geforce/graphics-cards/compare/ | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| GeForce RTX 5070 Ti 16 GB | 896 | nvidia.com/en-us/geforce/graphics-cards/compare/ | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| GeForce RTX 5090 32 GB | 1792 | nvidia.com/en-us/geforce/graphics-cards/compare/ | Manufacturer's spec sheet; figure checked on October 29, 2026. |
| DDR4-3200 (dual channel) | 51.2 | MT/s × 8 bytes × 2 channels | Theoretical peak calculated by us; a single module or channel delivers half. |
| DDR5-5600 (dual channel) | 89.6 | MT/s × 8 bytes × 2 channels | Theoretical peak calculated by us; a single module or channel delivers half. |
| DDR5-6000 (dual channel) | 96 | MT/s × 8 bytes × 2 channels | Theoretical peak calculated by us; a single module or channel delivers half. |
Configuration of each model
Layers, KV heads and head size come from the linked config.json (checked against the calculator's data on October 29, 2026). When the original repository requires accepting a license, a public mirror of the same file is used. Total parameters are the tensor count reported by the Hugging Face API, and the active ones of a MoE are those on its model card.
Assumptions
Bits per weight of each quantization: llama-quantize README (Llama 3.1 8B table; the real size varies a little per model). The rest are our own assumptions, approximate and without an official source: 0.8 GB of runtime, 70% of a Mac's unified memory available to the GPU by default, 4 GB reserved for the system, a 85% threshold to say "comfortable", an efficiency of 0.6 and a ±25% band on speed.