What is a local AI model?
Understand tokens, weights, open-weight models and why you might run a model on your own computer.
Models are files
An AI language model is a very large file full of numbers, called weights, produced by training on large amounts of text. When you ask a question, a program loads those weights into memory and uses them to generate an answer one small piece at a time. Those pieces are called tokens. A token is often part of a word.
Cloud and local
With a cloud assistant, your text travels to the company's servers and the answer comes back. With a local model, the weights are on your own disk and the answer is generated on your own hardware.
Local models are useful when you want privacy, offline use, or no per-request fees. The trade-offs are that you need enough memory and disk space, and the models that fit on a laptop are smaller than the biggest cloud models.
Open-weight is not always open source
Companies publish some models as open-weight, which means you can download them. Licenses differ. For example, Mistral 7B Instruct v0.3 uses Apache 2.0, while Llama 3.1 and Gemma use their own custom terms. Always read the license on the model card before using a model in a product.
What runs the model
You need a program to load the weights and talk to them. Ollama is one such program. It downloads models, runs them, and offers a local API on your computer. In the next lessons you will check how much memory you need and run your first model.
Quick check
What does an open-weight model let you do?
Quick check
What is a token?

