Install Ollama on Linux: Run AI Locally

ChatGPT, Gemini, and Claude demonstrate what modern AI can do. But why must your AI run in the cloud? Ollama runs Large Language Models on a Linux PC, workstation, or server. According to Ollama, prompts and responses from locally run models are not sent to Ollama; optional cloud models are a separate feature.

At its simplest, installation takes one command. This tutorial explains how to install Ollama on Linux, configure it, and use your first local AI model.

What is Ollama?

Ollama is a runtime for Large Language Models (LLMs). It downloads and manages models, runs them on CPU or GPU, provides a command-line interface and exposes a local REST API. The current CLI can also configure and launch supported coding tools and development environments.

Why run AI locally?

1. Data stays local

Your own system processes prompts and responses. This is useful for internal code, confidential documents, knowledge bases, RAG systems, and labs. Local operation is not automatically compliant with privacy law: data, model licenses, access controls, and procedures still require assessment.

2. No per-request local API fees

Once a model is downloaded, local inference has no conventional per-token API charge. You supply the CPU, memory, storage, electricity, and possibly GPU.

3. No permanent cloud connection

An existing local model can generally run offline. New models, updates, and optional cloud features still need internet access.

4. Direct application integration

Ollama exposes its API at http://localhost:11434/api by default.

Linux requirements

Official packages are available for AMD64/x86-64 and ARM64. A dedicated GPU is optional. Model size, quantization, and context length determine RAM and VRAM requirements, so begin with a smaller model on modest hardware.

Step 1: Update Ubuntu or Debian

sudo apt update
sudo apt upgrade -y
sudo apt install curl -y
uname -m

Common architecture results are x86_64 and aarch64.

Step 2: Install Ollama on Linux

curl -fsSL https://ollama.com/install.sh | sh
ollama -v

Inspect downloaded installation scripts before running them, particularly on managed systems. This is the command in the official Linux guide.

Step 3: Check the Ollama service

sudo systemctl status ollama
sudo systemctl start ollama
sudo systemctl enable ollama

Ollama recommends a startup service on suitable systemd systems.

Step 4: Run your first model

ollama run gemma3

Ollama downloads the model if necessary, then opens an interactive prompt. Try: Explain Docker in five simple sentences.

Install and use Codex CLI as a coding agent on Linux (Read article)

Download and manage models

# Download
ollama pull gemma3
# Run
ollama run gemma3
# List local models
ollama ls
# Show running models and CPU/GPU allocation
ollama ps
# Remove
ollama rm gemma3

These commands are documented in the official CLI reference. ollama ps also reports CPU, GPU, or mixed loading.

Use Ollama with an NVIDIA GPU

Support depends on the GPU generation, compute capability, and driver. Check the driver and model allocation with:

nvidia-smi
ollama ps

100% GPU means the entire model is in GPU memory. Consult the current Ollama hardware list.

Use Ollama with an AMD GPU

The documented ROCm route on Linux currently requires ROCm v7. Vulkan provides additional support. Check the current compatibility list first; manual installations also offer a separate ROCm archive.

Run Ollama without a GPU

Models can run entirely on CPU and system memory. This is slower but often sufficient for tests, smaller models, and simple automations.

Use the Ollama API

curl http://localhost:11434/api/generate -d '{
  "model": "gemma3",
  "prompt": "Explain Linux in three short sentences.",
  "stream": false
}'

This supports Python and JavaScript applications, internal chatbots, document analysis, RAG, and automation. Ollama provides official Python and JavaScript libraries.

How MCP connects AI systems to tools and data (Read article)

Does Ollama really run locally?

Yes, when you use a local model. To enforce local-only mode, disable Ollama cloud features:

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_NO_CLOUD=1"

sudo systemctl daemon-reload
sudo systemctl restart ollama

Cloud models and related cloud features are then unavailable.

Change the model storage path

Linux models are stored in /usr/share/ollama/.ollama/models by default. To use a larger drive:

sudo mkdir -p /data/ollama-models
sudo chown -R ollama:ollama /data/ollama-models
sudo systemctl edit ollama

[Service]
Environment="OLLAMA_MODELS=/data/ollama-models"

sudo systemctl daemon-reload
sudo systemctl restart ollama

Expose Ollama on your network

Ollama binds to 127.0.0.1:11434 by default. On a protected internal network, a systemd override can use:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Never expose port 11434 directly to the internet. The local API does not require authentication. Use firewall rules, a VPN, TLS, a reverse proxy, authentication, access controls, and network segmentation. The loopback default is normally best for a single workstation.

Review AI output and system-administration risks critically (Read article)

View logs and update Ollama

journalctl -e -u ollama
journalctl -u ollama -f
sudo systemctl status ollama
curl -fsSL https://ollama.com/install.sh | sh
ollama -v

Manual Linux installation

Administrators who avoid curl | sh can install the official archive:

curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst \
  | sudo tar x -C /usr
ollama serve

Separate ARM64 and AMD/ROCm archives are available. A dedicated systemd service is recommended on servers.

Context length and memory

A larger context window allows the model to consider more information at once but consumes more memory. ollama ps reports the loaded model's context length and memory allocation. This matters especially for long documents, coding repositories, and agents.

Who is Ollama for?

• Individuals: an accessible introduction to local models.
• Developers: a local API for custom applications.
• Administrators: a central service inside a protected network.
• Businesses: deliberate processing on owned infrastructure.
• Agent developers: local models for tools and workflows.

Ollama or cloud AI?

Cloud AI excels with very large models, elastic scaling, and minimal local hardware. Ollama excels in data control, offline use, development, owned infrastructure, local API integration, and predictable hardware cost. A deliberately designed hybrid setup is often practical.

Capabilities and limits of modern AI agents in 2026 (Read article)

FAQ: Ollama on Linux

Can I install Ollama on Ubuntu or Debian?

Yes. Ollama provides official Linux packages for AMD64 and ARM64.

Does Ollama require a graphics card?

No. Models run on CPU too, although a compatible GPU is usually much faster.

Does Ollama run entirely locally?

Local models run locally. Set OLLAMA_NO_CLOUD=1 to disable cloud features as well.

Which port does Ollama use?

Port 11434, bound to 127.0.0.1 by default.

Does the local API require authentication?

Ollama says no authentication is required for localhost:11434, so never expose it publicly without protection.

Where are models stored?

By default in /usr/share/ollama/.ollama/models; OLLAMA_MODELS changes the location.

Conclusion

With curl -fsSL https://ollama.com/install.sh | sh and ollama run gemma3, a Linux machine quickly becomes a local AI platform. The API supports custom programs, knowledge bases, and automations, while OLLAMA_NO_CLOUD=1 enables intentional local-only operation.

Sources and currency

Original article date: August 9, 2026; fact-check and revision: August 13, 2026. Sources are Ollama's official documentation for Linux and systemd, the CLI, the API, GPU support, local mode and configuration, and authentication. The undocumented gemma4 in the supplied draft was corrected to the currently documented gemma3.