Skip to content
AI Stack Nest
BeginnerLesson 3 of 3 · 1 min read

Run your first model with Ollama

Install Ollama, run a model, check whether it uses your GPU and learn the commands you will use every day.

Install Ollama

Download Ollama from the official site, ollama.com/download, and follow the guide for your system: Windows, macOS or Linux. Each guide lists the requirements from the official documentation.

Run a model

Open a terminal and run:

ollama run llama3.1

The first time, Ollama downloads the model, about 4.9 GB for the 8B version. When the >>> prompt appears, type a question. Type /bye to leave.

Commands you will use every day

  • ollama list shows installed models.
  • ollama ps shows loaded models and whether they use the GPU or CPU.
  • ollama stop MODEL unloads a model from memory.
  • ollama rm MODEL deletes a model to free disk space.

Good to know

Ollama listens on http://localhost:11434 and by default only accepts connections from your own computer. A model stays loaded for five minutes after the last request, then it is unloaded. The default context length is 4096 tokens.

For the full command list, see the Ollama cheat sheet. If something goes wrong, check the troubleshooting section of your system's guide.

Quick check

Which command shows whether a model runs on the GPU or CPU?

Quick check

How do you leave an Ollama chat?

0 of 3 lessons done in this track