Skip to content
AI Stack Nest
BeginnerLesson 2 of 3 · 1 min read

Choose a model size that fits your computer

Use real download sizes to pick a model that fits your memory, and learn what quantization means.

Size on disk is a guide

The size of the download is a good first estimate of the memory the model needs. You also need room for your operating system, other programs and the conversation. A common rule of thumb is to keep the model below about half of your total memory.

Real sizes

These sizes come from the Ollama library, read on 2026-11-17: Qwen2.5 0.5B is 398 MB, Gemma 2 2B is 1.6 GB, Mistral 7B is 4.4 GB, Llama 3.1 8B is 4.9 GB, Gemma 2 27B is 16 GB and Llama 3.1 70B is 43 GB.

What is quantization?

The default downloads are quantized. That means the model's numbers are stored with fewer bits, for example four instead of sixteen. The file becomes much smaller and uses less memory, usually with a small drop in quality. It is what makes 7B and 8B models practical on a laptop.

A starting guide

On 8 GB of memory, stay with small models. On 16 GB, 7B to 8B models are a good choice. With more memory you can try larger ones. These are rules of thumb and your processor and graphics card also affect speed.

Try it

The code above compares four models with a 16 GB computer using the half-of-memory rule. Change the ram value to see how the answer changes.

Try it

var ram = 16;
var models = { "Mistral 7B": 4.4, "Llama 3.1 8B": 4.9, "Gemma 2 27B": 16, "Llama 3.1 70B": 43 };
for (var name in models) {
  console.log(name + " (" + models[name] + " GB): " + (models[name] < ram / 2 ? "fits comfortably" : "too big"));
}
Common errors
  • Picking a model as large as your total memory. Leave room for the system and the conversation.

Quick check

Roughly what does 4-bit quantization do?

Quick check

Which size is a sensible first choice on a 16 GB computer?

0 of 3 lessons done in this track