If you've spent any time in local AI communities, you've probably seen this question asked a hundred different ways: "Can my GPU run this model?" "Will 16GB of RAM be enough?" "Why did my download just crash my whole computer?"
The honest answer is that model size and memory requirements aren't as simple as matching gigabytes to parameter counts. Quantization, context length, and the difference between VRAM and system RAM all change the math significantly. This guide breaks down exactly what you need at each model size, so you can stop guessing and start planning a setup that actually works, whether that's a gaming PC, a Mac, or your Android phone connected to a home server via LMSA.
The Quick Answer
If you just want the numbers before the explanation, here's the practical breakdown for the most common Q4 quantization level, which is what most people actually run:
- 7B models: ~5-8GB VRAM (or 8-16GB system RAM for CPU-only)
- 13B models: ~8-12GB VRAM (or 16GB system RAM)
- 30B-34B models: ~16-20GB VRAM (or 32GB system RAM)
- 70B models: ~40-48GB VRAM (or 64GB+ unified memory on Mac)
Those numbers assume 4-bit quantization and a moderate context window. Push the context length up, run a higher-precision quant, or go full FP16, and everything scales up fast. Let's get into why.
The Formula Behind the Numbers
Every LLM memory estimate comes down to one core calculation:
Memory (GB) ≈ Parameters (in billions) × Bytes per parameter × 1.15–1.2 (overhead)
The "bytes per parameter" figure depends entirely on quantization:
- FP16/BF16 (full precision): 2 bytes per parameter
- Q8 (8-bit): 1 byte per parameter
- Q5_K_M: roughly 0.7 bytes per parameter
- Q4_K_M (the most common local setup): roughly 0.5-0.55 bytes per parameter
The extra 15-20% overhead accounts for the KV cache, activations, and runtime memory the inference engine needs on top of the raw model weights. That overhead grows with context length, which is a detail a lot of people miss until their app crashes mid-conversation.
Model Size by Model Size
7B Models: The Accessible Tier
Seven-billion-parameter models like Mistral 7B or Llama 3.1 8B are the entry point most people start with, and for good reason. At Q4 quantization, a 7B model needs roughly 4-6GB of VRAM for the weights alone, and comfortably runs on an 8GB GPU with room left for context. Even CPU-only setups can handle these reasonably well with 8-16GB of system RAM, though you'll trade speed for accessibility.
Cards like the RTX 3060 12GB, RTX 4060, or RTX 5070 handle 7B models without breaking a sweat. On the Mac side, even a base Mac mini M4 with 16GB of unified memory runs 7B models smoothly.
13B Models: The Middle Ground
Thirteen-billion-parameter models (a tier that includes older Llama 2 13B-based models and several newer fine-tunes) need around 8-10GB of VRAM at Q4, pushing closer to 12GB once you factor in a reasonable context window. This is where an 8GB GPU starts to feel tight, and where 12GB cards like the RTX 3060 12GB or RTX 4070 become genuinely useful.
On a Mac, 16-24GB of unified memory handles 13B models comfortably, with an M3 Pro or better delivering 15-22 tokens per second, which feels responsive in normal conversation.
30B-34B Models: Where It Gets Interesting
This mid-tier is often overlooked, but it's become one of the more popular ranges in 2026 because it offers a real jump in reasoning quality without demanding server-grade hardware. Expect to need 16-20GB of VRAM at Q4, which puts it in reach of 16GB and 24GB consumer cards like the RTX 5080 or RTX 4090.
On Apple Silicon, 32GB of unified memory is the realistic minimum, and it runs well.
70B Models: The Big Leagues
This is where things get genuinely demanding. A 70B model at Q4_K_M quantization needs approximately 40-48GB just to hold the weights, KV cache, and runtime overhead, and that's before you push context length up significantly. No single consumer GPU under 32GB will hold that on its own.
Here's what actually works for 70B models in practice:
- RTX 5090 (32GB): The only current consumer GPU that can hold a quantized 70B model almost entirely in VRAM, hitting roughly 40-50 tokens per second. Anything smaller requires offloading layers to system RAM, which tanks performance to 1-3 tokens per second, well below usable for real-time chat.
- Dual GPU setups (two 24GB cards): Technically viable but adds complexity and PCIe overhead that can make it slower than expected.
- Apple Silicon with 64GB+ unified memory: This is genuinely the most practical route for most people. An M4 Max with 64-128GB of unified memory runs a 70B Q4 model at roughly 12-28 tokens per second depending on the exact chip and context length, all without a dedicated GPU or the power draw of a multi-card PC rig.
If you're weighing a hardware purchase specifically to run 70B models, unified memory Macs currently offer the best combination of simplicity and cost relative to a comparable PC build.
VRAM vs. System RAM: Why the Distinction Matters
VRAM (the memory on your graphics card) and system RAM are not interchangeable, and this trips up a lot of people new to local AI. VRAM is significantly faster than system RAM and sits right next to the GPU's processing cores, which is why inference engines try to load as much of the model there as possible.
When a model is too large for your VRAM, the inference engine offloads the excess layers to system RAM, and every token generated has to shuttle data across the PCIe bus, which is dramatically slower than VRAM-to-GPU communication. This is why a 13B model that runs at 25 tokens per second entirely in VRAM can drop to 3-5 tokens per second the moment even a portion of it spills into system RAM.
Apple Silicon sidesteps this problem entirely with unified memory: there's no separate VRAM pool, so the GPU has direct access to the entire memory pool at once. That's part of why Macs punch above their weight for large models despite lacking a discrete GPU.
Don't Forget Context Length
Quantization gets most of the attention, but context length quietly eats memory too, and it scales in a way that surprises people. The KV cache (which stores the model's "memory" of the ongoing conversation) grows with every token in your context window. A 7B model might only need an extra 1-2GB of KV cache at a short 4K context, but stretch that to 32K and you're looking at several additional gigabytes. At the 70B scale, a jump from a 2K to 128K context window can add over 40GB of KV cache alone, easily doubling your total memory requirement.
If you're running long documents through RAG or having extended conversations, budget extra headroom beyond the base model size, not just enough to load the weights.
Practical Recommendations by Budget
Just getting started / budget-conscious: An 8GB GPU (RTX 4060, RTX 5070) or a 16GB Mac handles 7B models well. This is a genuinely good entry point and covers a huge range of use cases, from coding help to general chat.
Mid-range, want quality without breaking the bank: A 12-16GB GPU or a 24-32GB Mac opens up the 13B-30B range, which is where a lot of people find the best balance of speed and capability.
Serious local AI setup: 24GB+ VRAM or 64GB+ unified memory gets you into 70B territory, where output quality starts approaching what you'd get from hosted cloud models, without sending a single token off your network.
Connecting It All to Your Phone
None of this hardware math matters if you're stuck sitting at your desk to use it. Once you've got LM Studio or Ollama running a model that fits your GPU or Mac's memory, LMSA lets you connect from your Android phone over Wi-Fi, or from anywhere using Tailscale, so the model you sized your hardware around becomes something you can actually chat with from the couch, not just the desk it's plugged into.
Whatever tier you land on, from a modest 7B model on an 8GB laptop GPU to a 70B beast on a maxed-out Mac Studio, the underlying principle stays the same: match your quantization and context window to your actual available memory, leave headroom for overhead, and you'll avoid the most common failure mode in local AI, which isn't a bad model choice, it's simply running out of room to hold it.