Ollama Tutorial 2026: Install, Configure, and Avoid the Three Silent Traps

This Ollama tutorial takes you from zero to a local language model you can use in the terminal, over an API and from your own scripts. Ollama is a free, open-source tool (MIT license) that downloads language models to your own computer or server, runs them and exposes them through a local interface. One command to install, one command to run, done. That is the pitch.

The pitch is even true. What it leaves out: Ollama ships with a few defaults that will hand you wrong answers without a single error message. On October 29, 2026 we installed Ollama 0.40.13, released the evening before, on our own server and ran every step of this tutorial ourselves, with measurements. Three things stood out that almost no guide mentions. They are in here, with numbers.

If you first want to know whether local AI is worth it for you at all, what hardware you need and how Ollama compares to LM Studio and llama.cpp, we covered that in depth in our article on running AI locally. This article is the workbench that goes with it: how do you actually use Ollama well once you’ve decided to?

Video: “Learn Ollama in 10 Minutes” by proflead (January 2026), a compact visual introduction. Version numbers in the video are older now, the workflow is the same.

Ollama tutorial in short: the key points first

• Installation: a one-liner on Linux, done in 8.9 seconds on our server. A regular installer on macOS and Windows.
• First model: ollama run qwen3.5:4b downloads about 3.3 GB and opens a chat. The download took us 25 seconds.
• Trap 1, context length: Ollama defaults to a 4,096-token context. Longer inputs get cut off at the front without warning. In our test that made the model answer a question about a long text plainly wrong.
• Trap 2, thinking: many current models “think” first by default. For small questions that made them over 100 times slower, but sometimes also more correct. You want to control this on purpose.
• Trap 3, concurrency: simultaneous requests are processed one after another by default. Four parallel requests took 9.8 seconds instead of 2.5.
• API: runs automatically on 127.0.0.1:11434, including an OpenAI-compatible format. Existing scripts often only need a new base URL.
• Security: the API has no authentication. Never expose port 11434 to the internet unprotected.

What is Ollama, and what actually happens when it runs?

At its core, Ollama is packaging around an inference engine. Run ollama ps and the RUNNER column shows llamacpp for normal GGUF models: llama.cpp does the actual number crunching, Ollama handles everything around it. That includes:

• Model management like a package manager: pull, list, rm, with versions and shared file layers.
• A background service (ollama serve) that loads models into memory on demand and unloads them after a while.
• An HTTP API in its own format plus an OpenAI-compatible one.
• Chat templates, so you don’t have to care which special tokens a given model expects for “user” and “assistant”.

The project moves fast. On the day of our test the GitHub repository had around 185,000 stars and the library at ollama.com/library listed 264 models. For comparison: on August 1, 2026, when we wrote our local AI article, it was 232 models and version 0.32.5. Eight version jumps in three months. That is why this tutorial explains the stable fundamentals and only names specific models as dated examples.

One point belongs right at the start because it matters for privacy: Ollama now also offers cloud models. About 20 library entries are marked that way. They carry cloud in the name and do not run on your machine but on Ollama’s servers. If you use Ollama because “my data stays with me”, check which model you are actually starting.

Installing Ollama on Linux, macOS and Windows

Linux (including servers)

On Linux the official one-liner is all you need:

curl -fsSL https://ollama.com/install.sh | sh

If you don’t like piping scripts from the internet into your shell (fair), download it and read it first. The script was 455 lines long for us and does four main things: it downloads the right archive, creates an ollama system user, adds it to the render and video groups (for GPUs) and sets up a systemd service.

curl -fsSL https://ollama.com/install.sh -o ollama-install.sh
less ollama-install.sh
sh ollama-install.sh

On our server the whole thing took 8.9 seconds. Afterwards the script prints:

>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.

The warning is correct for us, our server has no GPU. That is explicitly not a blocker, as the measurements below show.

One observation worth knowing: we had installed Ollama on this server before and deliberately disabled the service afterwards. The install script re-enabled and started it during the update without asking. If you switched the service off for a reason (say, the server needs that RAM for other things), you’ll have to do it again after every update. Updating, by the way, is the exact same command as installing.

Check that everything is running:

ollama --version
systemctl status ollama
ss -tlnp | grep 11434

On our machine the service listened on 127.0.0.1:11434, local only. That is the right default.

If you’re new to Linux services: our guide to creating a systemd service explains what’s behind systemctl and helps with the configuration further down.

macOS

On macOS you download the app from ollama.com/download, drag it to Applications and start it. It installs the ollama command for the terminal and sits as a small icon in the menu bar. On Apple Silicon (M1 and later), Ollama automatically uses unified memory as GPU memory, which makes Macs surprisingly good at running local models. We did not test the macOS installer ourselves for this article.

Windows

Windows has an installer on the same page. After installation ollama is available in PowerShell and the command prompt, and the service runs in the background. NVIDIA and AMD GPUs are detected if current drivers are installed. We didn’t test the Windows installer ourselves either, but usage afterwards is identical to Linux.

Docker

If you run everything in containers, use the official ollama/ollama image. A volume for the models is essential, otherwise you re-download them after every restart:

docker run -d --name ollama -v ollama:/root/.ollama -p 127.0.0.1:11434:11434 ollama/ollama

Note the 127.0.0.1: in front of the port. Without it, Docker publishes the port on every network interface and often bypasses your firewall in the process. More on that in our Docker Compose examples.

Your first model: download, run, chat

Download a model

ollama pull qwen3.5:4b

The part before the colon is the model family, the part after it is the “tag”, usually the size in billions of parameters. Downloading 3.3 GB took us 25 seconds on a fast server connection. At home on 100 Mbit/s, expect more like five minutes.

ollama show tells you what’s inside a model:

$ ollama show qwen3.5:4b
  Model
    architecture        qwen35
    parameters          4.2B
    context length      262144
    quantization        Q4_K_M
  Capabilities
    completion
    vision
    tools
    thinking
        levels     false, true
        default    true

Two lines in there matter more than they look. context length 262144 means the model could handle over 260,000 tokens of context. Ollama only uses a fraction of that by default, more on that in a moment. And thinking default true means the model reasons before answering unless told otherwise. That has consequences too.

Chat in the terminal

ollama run qwen3.5:4b

A prompt appears. /bye ends the chat, /? lists all commands. The most useful ones:

/set parameter num_ctx 16384   # more context for this session
/set think false               # turn thinking off
/show info                     # model info

For numbers, start with ollama run qwen3.5:4b --verbose. After each answer you’ll see how many tokens per second the model produced.

Which model should I pick?

Honest answer: it changes monthly. A few rules of thumb stay stable, though.

• File size is roughly RAM usage. A 3.3 GB model used 3.1 GB of memory with a small context and 3.7 GB with a large one. Plan for 20 to 30 percent headroom.
• Start small. A model with 3 to 5 billion parameters runs on almost any machine with 8 GB of RAM and shows you whether you like the approach.
• Q4_K_M is the default and the right size-versus-quality trade-off for most purposes. Ollama picks it automatically for most models.
• Specialist models for specialist jobs. There are dedicated coder models for programming, and embedding models for search and databases (see below).
• Very small models make things up. In our August test, a 0.8-billion-parameter model confidently answered a question about quantization completely wrong. Fast is not the same as useful.

How much RAM a server needs in general, including AI workloads, is broken down in How much RAM does a server need?.

Essential commands at a glance

ollama ps is the most important command when something’s off. Here’s what ours looked like:

NAME          SIZE      PROCESSOR    CONTEXT    RUNNER      UNTIL
qwen3.5:4b    3.1 GB    100% CPU     4096       llamacpp    4 minutes from now

If PROCESSOR says 100% CPU even though you have a GPU, the model doesn’t fit in video memory. If it says something like 40%/60% CPU/GPU, part of it runs on the CPU and it’ll be slower accordingly. And the CONTEXT column leads straight to the first trap.

Trap 1: Ollama silently truncates long inputs

This is the most important section of this tutorial. By default Ollama uses a context window of 4,096 tokens. That’s what the official FAQ says, and what ollama ps showed us. A token is roughly a word fragment; in English one word averages a bit over one token. So 4,096 tokens is about six to eight pages of text, prompt and answer combined.

What happens when your input is longer? We tested it. Our text began with “SECRET WORD: Blackberry.”, followed by about 12,600 tokens of filler, ending with the question: “What is the SECRET WORD from the beginning?”

(Our test prompt was in German, “Brombeere”; the effect is language-independent.)

With the default setting, Ollama processed only 2,050 of more than 12,000 tokens. It threw away the rest, including the secret word at the start. There was no error message, no HTTP error, no warning in the response. The model honestly answered that it saw no secret word, because in the part it received, there wasn’t one. The only clue is prompt_eval_count: 2050 in the response, and nobody reads that.

In everyday use this means: if you have Ollama summarize documents, analyze logs or reply to long emails with default settings, you get answers about texts the model never fully saw. That’s especially treacherous because the answers sound plausible.

How to set the context properly

There are four ways, depending on where you use Ollama:

1. Per request in the API:

{"model": "qwen3.5:4b", "prompt": "...", "options": {"num_ctx": 16384}}

2. In the interactive chat: /set parameter num_ctx 16384

3. Permanently for the whole server via an environment variable (Linux with systemd):

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=16384"

sudo systemctl restart ollama

After that, ollama ps showed CONTEXT 16384 for us even without specifying it in the request.

4. Baked into your own model variant with a Modelfile (see the next section).

What more context costs

More context isn’t free. We measured two costs:

• Memory: with 4,096 tokens the model used 3.1 GB, with 16,384 tokens 3.7 GB. With bigger models the difference grows accordingly.
• Time: our server read the 12,645 tokens at about 160 tokens per second, which took 79 seconds before the first word of the answer appeared. A GPU does this many times faster. On a CPU, prompt processing is the real bottleneck, not generation speed.

The sensible rule: set the context as large as your longest real inputs, plus room for the answer. Not blindly to the maximum, that just eats memory.

Trap 2: thinking makes models slow, but sometimes right

Many current models are reasoning models. Before the actual answer, they write out an internal chain of thought. In Ollama, this is on by default for supported models; ollama show says default true.

We asked a classic trick question (in German): how many r’s are in the word “Erdbeerkuchen” (strawberry cake)? Just the number. The correct answer is 2 (Erdbeerkuchen).

A neat illustration of why there’s no blanket recommendation. With thinking, the model reasoned through more than 1,100 characters, took 120 times longer and got it right. Without thinking, the answer came instantly and was wrong.

Rule of thumb from our practice:

• Thinking off for summarizing, rephrasing, translating, classifying and simple extraction. Reasoning adds little there and costs a lot of time.
• Thinking on for math, logic, multi-step tasks, and anything where a wrong result is expensive.

How to toggle it:

ollama run qwen3.5:4b --think=false        # terminal
ollama run qwen3.5:4b --hidethinking       # thinks, but doesn't show it

{"model": "qwen3.5:4b", "prompt": "...", "think": false}

In the API, the reasoning ends up in its own field (thinking), separate from the answer (response). If your application suddenly waits forever for answers after switching to a new model, thinking is the first suspect.

Custom model variants with a Modelfile

A Modelfile is to Ollama what a Dockerfile is to Docker: a small text file that turns an existing model into your own variant with fixed settings. It’s the cleanest way to defuse Trap 1 permanently and give a model a fixed role.

FROM qwen3.5:4b
PARAMETER num_ctx 16384
PARAMETER temperature 0.3
SYSTEM """You are a terse Linux admin helper. Answer in at most 3 sentences."""

ollama create admin-helper -f Modelfile
ollama run admin-helper

Creating it took 0.25 seconds and used no extra disk space: the model directory stayed at 3.4 GB. Ollama stores models in layers like Docker images, and your variant just references the existing weights. So you can create as many variants for different tasks as you like.

The most important instructions:

With FROM ./my-model.gguf you can also bring in models from Hugging Face that aren’t in the Ollama library.

The Ollama API: use it from scripts and programs

Ollama’s real value isn’t the chat, it’s the API that’s already running. It comes in two formats.

The native API

curl http://127.0.0.1:11434/api/generate -d '{
  "model": "qwen3.5:4b",
  "prompt": "Explain in three sentences what a reverse proxy is.",
  "stream": false,
  "think": false
}'

For conversations with history, use /api/chat with a list of messages. Besides the text, the response contains metrics you can use for your own benchmarks:

• load_duration: time to load the model (only significant on the first call, 5.1 seconds for us)
• prompt_eval_count: how many input tokens were actually processed (this is where you catch Trap 1!)
• eval_count and eval_duration: tokens generated and time in nanoseconds

Short answers ran at 12 to 37 tokens per second for us, depending on how busy the server was with other work. Under load it halved. That’s the honest part of CPU inference on a server that does other things too: the speed fluctuates. People read about 5 to 8 tokens per second, so it’s still plenty for chat.

Our Python measurement script deliberately uses no extra libraries:

import json, urllib.request

def ask(prompt, model="qwen3.5:4b", ctx=16384):
    body = {"model": model, "prompt": prompt, "stream": False,
            "think": False, "options": {"num_ctx": ctx}}
    req = urllib.request.Request("http://127.0.0.1:11434/api/generate",
          data=json.dumps(body).encode(), headers={"Content-Type": "application/json"})
    d = json.load(urllib.request.urlopen(req, timeout=600))
    return d["response"], d["prompt_eval_count"]

answer, read = ask("Summarize: ...")
print(answer, f"({read} tokens read)")

The OpenAI-compatible API

Under /v1/, Ollama also speaks the OpenAI API format. Handy, because nearly every tool and library understands it:

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:11434/v1", api_key="ollama")
r = client.chat.completions.create(model="qwen3.5:4b",
        messages=[{"role": "user", "content": "Just say: Hello"}])
print(r.choices[0].message.content)

The API key is just a placeholder. We tested with an arbitrary value and the request was answered without complaint (4.7 seconds including model load). Which also means: the API checks no key. Whoever can reach the port can do anything. More on that below.

Ollama with coding tools

New in recent versions is ollama launch. It starts tools like Claude Code, Codex, OpenCode, Cline or VS Code with an Ollama model as the brain. In version 0.40.13 the list of supported integrations had grown to 18. Convenient, but don’t expect miracles from a 4B model on a CPU. We measured how the big cloud tools perform in Claude Code vs Cursor. For serious programming work, local models need a lot of GPU memory.

Video: “How to Use Ollama in VSCode (2026)” by Tutorial vx, showing the editor integration step by step.

Embeddings with Ollama: search and RAG

Besides text models, Ollama can run embedding models. These don’t produce text; they turn a text into a list of numbers, a vector. Similar texts get similar vectors. Semantic search and RAG build on this (Retrieval Augmented Generation: the model gets the matching passages from your own documents before answering).

ollama pull nomic-embed-text
curl http://127.0.0.1:11434/api/embed -d '{
  "model": "nomic-embed-text",
  "input": ["How do I reset my password?", "I forgot my password"]
}'

At 274 MB the model is tiny and downloaded in 5.9 seconds. Turning three sentences into vectors of 768 numbers each took 0.54 seconds. So far, so good. Then we measured how well it captures meaning. The closer the value to 1, the more similar the model considers two sentences:

In English the model separates cleanly: 0.79 versus 0.34. In German both values are almost level; the model can barely tell a matching topic from a completely unrelated one. The prefixes the vendor recommends (search_query: and search_document:) didn’t change that much either (0.615 versus 0.527).

The lesson: the embedding model featured in countless Ollama tutorials is fine for English and a poor choice for German or mixed-language content. For multilingual content, test a multilingual model such as bge-m3 (about 1.2 GB). We didn’t measure that one for this article, so run the same test with your own example sentences before building search on it. Three sentences are enough to expose an unusable model.

Running Ollama as a service on a server

On a server, Ollama is a perfectly normal systemd service. All settings go through environment variables, which you set in an override file with sudo systemctl edit ollama. That keeps them intact across updates: we checked the install script, and it rewrites only the main unit file /etc/systemd/system/ollama.service.

The key settings

An example for a server that processes documents and should keep the model ready:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=16384"
Environment="OLLAMA_KEEP_ALIVE=30m"
Environment="OLLAMA_MODELS=/srv/ollama/models"

If you change the storage location, the ollama user needs write access there (sudo chown -R ollama:ollama /srv/ollama).

Trap 3: concurrent requests wait in line

We sent four identical requests at the same time. The result:

The individual times give it away: the requests were processed one after another, and the fourth waited almost ten seconds. Each one ran at normal speed (31 to 37 tokens per second); they were just queued. That’s OLLAMA_NUM_PARALLEL=1 at work.

For a personal assistant this doesn’t matter. But if you put Ollama behind a website, a team tool or an automation that fires many requests at once, you should know. More concurrency costs memory: per the documentation, RAM use scales with OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH. Four parallel requests with a 16,384-token context need considerably more memory than one. On a CPU without a GPU, more concurrency also helps little because the requests share the same cores.

Security: port 11434 does not belong on the internet

The Ollama API has no user management and no passwords. Anyone who can reach it can use, download and delete models and keep your server busy with compute. Security researchers regularly find thousands of Ollama servers sitting wide open in search engines for exposed services, usually because someone set OLLAMA_HOST=0.0.0.0 to reach it from their laptop.

Our recommendation, from convenient to secure:

1. Keep it on 127.0.0.1 and connect from your own machine through an SSH tunnel: ssh -L 11434:127.0.0.1:11434 user@server. Ollama is then reachable on your laptop at localhost:11434, encrypted, with no open port. How SSH works is covered in What is SSH?.
2. For multiple users, put a reverse proxy like Caddy or nginx in front, with HTTPS and authentication (basic auth or an upstream login). Ollama itself stays on 127.0.0.1.
3. On a home network, OLLAMA_HOST=0.0.0.0 can be acceptable, but only if your router doesn’t forward the port and the firewall opens it to your local network only.

A Docker reminder: -p 11434:11434 without 127.0.0.1: in front publishes the port to the world, even if your firewall supposedly blocks it.

Common problems and fixes

“Error: could not connect to ollama server”. The service isn’t running. On Linux, start it with sudo systemctl start ollama; on macOS and Windows, open the app. We got exactly this message ourselves because we had disabled the service weeks earlier.

Answers ignore the start of my text. Trap 1: context too small. Raise num_ctx and check prompt_eval_count in the API response.

The model takes forever for short answers. Trap 2: thinking is on. Turn it off with think: false or --think=false when the task doesn’t need it.

The first answer is slow, all later ones fast. Normal. On the first call Ollama loads the model into memory (5.1 seconds for us). After five minutes without requests it’s unloaded again. Extend that with OLLAMA_KEEP_ALIVE.

Everything runs on the CPU despite a GPU. Check ollama ps. Usually the model doesn’t fully fit in video memory, or the driver is missing. Pick a smaller model, a smaller quantization or less context.

The disk is filling up. Models are big and stay until you delete them. ollama list shows the sizes, ollama rm cleans up. On Linux they live in /usr/share/ollama/.ollama/models by default.

Updating and uninstalling Ollama

Update on Linux: just run the install script again. For us that went from 0.32.5 to 0.40.13 in 8.9 seconds. The script rewrites the service’s main unit file, so your settings belong in the override file from systemctl edit. Remember that the script re-enables the service. On macOS and Windows, the app notifies you of updates itself.

Update models: run ollama pull NAME again. Only changed layers are downloaded.

Uninstall on Linux:

sudo systemctl disable --now ollama
sudo rm /etc/systemd/system/ollama.service
sudo rm -r /usr/local/lib/ollama /usr/local/bin/ollama
sudo rm -r /usr/share/ollama        # also deletes all models
sudo userdel ollama && sudo groupdel ollama

After our tests we deleted all models and disabled the service again so it doesn’t hold memory on our production server when we don’t need it.

FAQ

Is Ollama free?

Yes. Ollama itself is open source under the MIT license and costs nothing, including for commercial use. The licenses of individual models are separate and vary; some allow commercial use only with restrictions. Only the optional cloud models cost money.

Does Ollama work offline?

Yes, once the models are downloaded. You need internet for installation and pull; after that everything runs offline. The exception: models with cloud in the name always run on Ollama’s servers.

Do I need a GPU for Ollama?

No. Our test server has none, and small models still ran at 12 to 37 tokens per second, faster than you can read. A GPU mainly speeds up reading long inputs enormously and allows bigger models. On Apple Silicon Macs, regular memory doubles as GPU memory.

How much RAM do I need for Ollama?

About as much as the model’s size, plus 20 to 30 percent headroom, plus your operating system. For a 4B model, 8 GB total RAM is a good starting point; for 12 to 14 billion parameters, more like 16 GB; for 27B and up, 32 GB or a GPU with lots of memory. More context and more parallel requests increase the requirement.

Where does Ollama store models?

On Linux with the official script in /usr/share/ollama/.ollama/models; on macOS and Windows in your user directory under .ollama/models. The OLLAMA_MODELS variable lets you choose another location, such as a larger disk.

Is my data private with Ollama?

With local models, yes: processing happens entirely on your device, no request goes to anyone else’s server. That doesn’t apply to models with cloud in the name. And it only holds as long as you don’t expose the API to the network, because it has no authentication.

How do I change the context length in Ollama?

Per request with "options": {"num_ctx": 16384}, in the chat with /set parameter num_ctx 16384, for the whole server with the OLLAMA_CONTEXT_LENGTH environment variable, or permanently in a Modelfile with PARAMETER num_ctx 16384. Check the current length in the CONTEXT column of ollama ps.

Can I use Ollama with the OpenAI library?

Yes. Set the base URL to http://127.0.0.1:11434/v1 and any API key. Chat completions, embeddings and model listing work, and many tools built for OpenAI run directly against local models this way.

What’s the difference between Ollama and LM Studio?

Ollama is a command-line tool with a background service and API, ideal for servers and automation. LM Studio is a graphical app for trying out and comparing models. Both use llama.cpp under the hood. The full comparison is in our article on running AI locally.

How do I access Ollama from another computer?

Safest is an SSH tunnel (ssh -L 11434:127.0.0.1:11434 user@server), so Ollama stays local-only. Alternatively set OLLAMA_HOST=0.0.0.0, but only on a protected home network or behind a reverse proxy with authentication. Never unprotected on the internet.

Why does Ollama do worse in other languages than in English?

Most models were trained mostly on English text. Larger models compensate well, small ones noticeably less. It was most striking in our embedding test: the popular nomic-embed-text separated topics cleanly in English but barely at all in German. Always test models with examples in your own language.

Conclusion: easy to start, important to understand

Ollama keeps its promise. Installed in under ten seconds, first model loaded in half a minute, an API that runs immediately and even speaks the OpenAI format. There is no more convenient tool for local AI.

The convenience has a price, and its name is defaults. The 4,096-token context window cuts off long texts without warning, and exactly where local AI is most useful (your own documents, logs, long texts) it then delivers plausible, wrong answers. Thinking can make an answer right or a hundred times slower, and you should decide which you need. Concurrent requests queue up, and the API trusts anyone who can reach it.

None of this is a reason against Ollama. It’s a reason not to skip these five minutes of configuration: set the context length to fit, choose thinking deliberately, keep an eye on prompt_eval_count, keep the port local. Then Ollama is exactly what it wants to be: a tool that just works.