Skip to content
AI Stack Nest
1 min read · updated 17 Nov 2026

Build a private chat assistant on your own computer

Run a chat assistant that works without sending your text to a cloud AI service, using Ollama and a small open model.

Steps

  1. Install Ollama for your system
  2. Pull a 7-8B model such as Llama 3.1 8B
  3. Chat in the terminal to check that it works
  4. Call the local API from your own script

What you will build

By the end you will have an AI assistant that runs completely on your computer. The model is stored on your disk, answers are generated on your own hardware, and you can talk to it from the terminal or from your own code. This is useful for practising with AI, for working with text you would rather not upload, and for learning how model APIs work.

You need a computer with at least 8 GB of memory, about 10 GB of free disk space, and an internet connection for the one-time model download. A computer with a supported GPU will be faster, but it is not required.

Walkthrough

  1. Install Ollama. Follow the guide for your system: Windows, macOS or Linux.
  2. Download a model. Run ollama pull llama3.1. The download is about 4.9 GB.
  3. Chat. Run ollama run llama3.1, ask a question and type /bye when you are done.
  4. Use the API. Ollama serves a local API on port 11434. This request asks the model a question and prints the whole answer at once:
curl http://localhost:11434/api/chat -d '{"model":"llama3.1","messages":[{"role":"user","content":"Explain what a token is in one sentence."}],"stream":false}'
  1. Check resource use. Run ollama ps to see whether the model is on the GPU or the CPU.

Next steps

Try a different model from the models page, compare answers on the same question, and read the Ollama cheat sheet for the settings that control context length and memory use.

Models used here

Not sure your computer can run a model? Use the will-it-run checker.