Build a private chat assistant on your own computer
Run a chat assistant that works without sending your text to a cloud AI service, using Ollama and a small open model.
Steps
- Install Ollama for your system
- Pull a 7-8B model such as Llama 3.1 8B
- Chat in the terminal to check that it works
- Call the local API from your own script
What you will build
By the end you will have an AI assistant that runs completely on your computer. The model is stored on your disk, answers are generated on your own hardware, and you can talk to it from the terminal or from your own code. This is useful for practising with AI, for working with text you would rather not upload, and for learning how model APIs work.
You need a computer with at least 8 GB of memory, about 10 GB of free disk space, and an internet connection for the one-time model download. A computer with a supported GPU will be faster, but it is not required.
Walkthrough
- Install Ollama. Follow the guide for your system: Windows, macOS or Linux.
- Download a model. Run
ollama pull llama3.1. The download is about 4.9 GB. - Chat. Run
ollama run llama3.1, ask a question and type/byewhen you are done. - Use the API. Ollama serves a local API on port 11434. This request asks the model a question and prints the whole answer at once:
curl http://localhost:11434/api/chat -d '{"model":"llama3.1","messages":[{"role":"user","content":"Explain what a token is in one sentence."}],"stream":false}'- Check resource use. Run
ollama psto see whether the model is on the GPU or the CPU.
Next steps
Try a different model from the models page, compare answers on the same question, and read the Ollama cheat sheet for the settings that control context length and memory use.
Models used here
Not sure your computer can run a model? Use the will-it-run checker.

