How to Install Ollama in an LXC Container on Proxmox VE (Run Local AI Chat Models for Beginners)

Introduction

You've probably heard someone in a homelab forum mention running "their own AI" instead of paying for ChatGPT or Claude. That's not as complicated as it sounds, and you don't need a rack of GPUs to try it. If you already run Proxmox VE, you have everything you need to spin up a small, isolated space and get a local language model answering questions in about fifteen minutes.

This guide walks through installing Ollama — a free tool for running open-source AI language models on your own hardware — inside an LXC container on Proxmox VE. No cloud account, no API key, no data leaving your network. We'll keep the container small on purpose, because most homelab boxes don't have a beefy GPU sitting around, and the smaller open models run just fine on CPU alone.

What You Will Learn

By the end of this tutorial you'll have a working, self-contained AI chatbot running on your own Proxmox host. Specifically, you'll learn:

• What Ollama is and why people run it instead of a hosted chatbot
• Why an LXC container is a good fit for this, and when a full VM makes more sense instead
• How to create a lightweight LXC container in Proxmox VE from scratch
• How to install Ollama inside it and download a small language model
• How to chat with the model from the command line, and how to reach it from another computer on your network
• The errors people run into most often, and the settings worth changing right after install

What Is This Feature?

Ollama is a small piece of software that downloads open-source AI language models and runs them locally, then exposes a simple command line and a local web API you can talk to. Think of it as the engine room — it doesn't have a fancy chat window built in by default, but it does the actual work of loading a model into memory and generating responses. Once it's running, you type a question and get an answer, all computed on your own CPU (or GPU, if you have one to pass through).

An LXC container is Proxmox's lightweight virtualization option. Instead of emulating a whole virtual computer the way a VM does, an LXC container shares the host's Linux kernel and just isolates the processes, filesystem, and network inside a sandbox. That means it starts in a second or two, uses a fraction of the RAM a VM would need for the same job, and feels — from the inside — exactly like a normal, minimal Linux server. For something like Ollama, which is just a single Go binary with no special hardware driver requirements on CPU, a container is a much cheaper way to isolate it than a full VM.

You'll also see the terms privileged and unprivileged container come up. An unprivileged container maps root inside the container to an ordinary, low-permission user on the actual Proxmox host, so if something inside the container gets compromised, it can't easily touch the host. Proxmox defaults to unprivileged, and there's no reason to change that for this project — Ollama doesn't need any elevated access to the host.

Why Would You Use It?

The honest answer is: privacy, cost, and curiosity, roughly in that order. Everything you type stays on hardware you own. There's no per-message bill, no rate limit tied to a subscription tier, and no risk of a provider changing the model underneath you. It's also just a genuinely fun way to learn how these tools work under the hood.

It's not a straight swap for a hosted assistant like ChatGPT or Claude, though — I'd be doing you a disservice if I pretended otherwise. The small models that run comfortably on CPU are noticeably less capable at complex reasoning, long documents, or coding help than the big hosted models. Where they genuinely shine is quick, private tasks: rewriting an email, summarizing a short note, answering a general knowledge question, or just tinkering with prompts to see how a model behaves. If you need serious reasoning power, you'll want either a GPU passed through to a bigger model, or you'll keep a hosted assistant around for the hard stuff. Running Ollama locally and using a hosted assistant aren't mutually exclusive — plenty of homelabbers do both.

Isolating it in its own LXC container also means if you decide later you don't want it anymore, or you want to try a completely different setup, you delete one container and it's gone — nothing lingering on your Proxmox host itself.

Prerequisites

Before you start, make sure you have:

• A working Proxmox VE 9.x host (this also works fine on 8.x — the container creation steps haven't changed)
• At least 4 GB of free RAM on the host to dedicate to the container, and roughly 16 GB of free disk space
• A Debian or Ubuntu container template downloaded (we'll cover this in the steps below if you haven't done it yet)
• Basic comfort typing commands over SSH or the Proxmox console — you don't need to be a Linux expert, just able to copy and paste carefully

A GPU is not required. This tutorial is written entirely around CPU-only inference, since that's what most homelab hardware actually has available.

Step-by-Step Tutorial

Step 1: Download a container template (if you haven't already)

Proxmox needs a base operating system image to build the container from. In the web interface, click your node, then local (or whichever storage holds templates) in the left sidebar, then CT Templates, and finally the Templates button. Search for debian-13-standard and download it. Debian 13 ("Trixie") is what current Proxmox VE releases ship container templates for, and it's a solid, boring, low-maintenance choice for something like this.

Prefer the command line? SSH into your Proxmox host and run:

pveam update
pveam available --section system | grep debian-13
pveam download local debian-13-standard_13.1-2_amd64.tar.zst

The exact version number in that filename (13.1-2 here) changes as Proxmox rebuilds templates over time, so run the pveam available command yourself and use whatever filename it actually shows you.

Step 2: Create the container

In the web UI, click Create CT in the top right. Walk through the wizard with these settings:

• General: pick a hostname like ollama, set a root password, and leave Unprivileged container checked
• Template: select the debian-13-standard template you just downloaded
• Disk: 16 GB is comfortable for the OS plus one or two small models
• CPU: 2 cores is a reasonable starting point
• Memory: 4096 MB, with swap left at the default
• Network: DHCP is fine to start, or set a static IP if that's your usual setup

Prefer the CLI? This does roughly the same thing in one command:

pct create 200 local:vztmpl/debian-13-standard_13.1-2_amd64.tar.zst \
  --hostname ollama --cores 2 --memory 4096 --swap 512 \
  --rootfs local-lvm:16 --net0 name=eth0,bridge=vmbr0,ip=dhcp \
  --unprivileged 1 --features nesting=1

Adjust 200 to whatever container ID is free on your system, and swap local-lvm for whatever storage you actually use for container disks. The nesting=1 feature isn't required for Ollama itself, but it's worth turning on now in case you later want to run something like Open WebUI in Docker inside the same container — adding it after the fact means editing the container's config again.

Start the container once it's created, either with the Start button in the GUI or pct start 200 from the shell.

Step 3: Get a shell inside the container

Click the container in the Proxmox web UI, then Console, and log in as root with the password you set. Or, from the Proxmox host's own shell, run:

pct enter 200

Either way, you should now be sitting inside a fresh, minimal Debian system.

Step 4: Update the container and install Ollama

Run a quick update first, then install Ollama using its official install script:

apt update && apt upgrade -y
curl -fsSL https://ollama.com/install.sh | sh

That script detects your CPU, downloads the right Ollama binary, and sets up a systemd service so it starts automatically. It takes a minute or two on most connections. When it finishes, check that the service came up cleanly:

systemctl status ollama
ollama -v

If systemctl status shows "active (running)" in green, you're good to move on.

Step 5: Pull a small model and try it

This is the part that actually feels like magic the first time you do it. Pull a compact model — llama3.2:1b is a good starting point since it's small enough to run comfortably on 2 CPU cores with no GPU:

ollama pull llama3.2:1b

That download is roughly a gigabyte and a half, so give it a minute depending on your connection. Once it's done, chat with it directly:

ollama run llama3.2:1b

You'll land in an interactive prompt. Ask it something simple — "explain what a container is in one paragraph" is a fine test — and watch it respond right there in your terminal, computed entirely on your Proxmox host. Type /bye to exit when you're done. If your hardware has some headroom left over, llama3.2:3b or qwen2.5:3b are noticeably sharper while still being CPU-friendly on most modern homelab boards, just slower per response.

Step 6: Reach it from other devices on your network

By default, Ollama only listens on 127.0.0.1 — meaning only the container itself can talk to it, not your laptop or another VM. To open it up to your LAN, edit the systemd service:

systemctl edit ollama

In the editor that opens, add these lines:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Save, then reload and restart the service:

systemctl daemon-reload
systemctl restart ollama

Now any device on your network can reach it at http://<container-ip>:11434. One honest warning here: Ollama has no built-in login or authentication. Anyone who can reach that port can use your model and burn your CPU cycles. Fine on a trusted home LAN behind your router — not something you want exposed to the internet without a reverse proxy and some form of access control in front of it.

Commands Explained

Command	What it does
pveam update	Refreshes Proxmox's list of downloadable container templates from the network
pveam download	Downloads a specific container template to local storage
pct create	Creates a new LXC container from a template with the options you specify
pct enter	Drops you straight into a root shell inside a running container from the host
curl -fsSL https://ollama.com/install.sh | sh	Downloads and runs Ollama's official install script, which sets up the binary and a systemd service
ollama pull <model>	Downloads a model from Ollama's library without running it yet
ollama run <model>	Downloads the model if needed, then opens an interactive chat with it
systemctl edit ollama	Opens (and creates, if needed) a config override file for the Ollama service, without touching the original unit file

Common Errors

A few things trip up almost everyone the first time:

• "command not found: ollama" right after install. Usually means the install script didn't finish, or you're in a fresh shell session that hasn't picked up the updated PATH. Log out and back in, or just run pct enter again.
• The model download seems frozen. It's rarely actually frozen — a 1–2 GB download over a slow connection just looks that way. Give it a few minutes before assuming something's wrong.
• "connection refused" when hitting the API from another machine. This almost always means you skipped the OLLAMA_HOST step above, and it's still bound to localhost only.
• Container feels sluggish or the model takes forever to respond. Two CPU cores and 4 GB RAM is a floor, not a guarantee of speed. Larger models on CPU-only hardware are slow by nature — that's the model doing real work, not a misconfiguration.

Troubleshooting

If Ollama won't start at all, check the service logs directly rather than guessing:

journalctl -u ollama -n 50 --no-pager

That usually points straight at the problem — a port already in use, a permissions issue, or a corrupted download. If the container itself seems out of memory, run free -h inside it while a model is loaded. If you're seeing the OOM killer terminate the process, the fix isn't more troubleshooting — it's just giving the container more RAM in its Proxmox resource settings, or picking a smaller model.

If the container won't even start from Proxmox, check pct status 200 and the task log in the Proxmox web UI. Nine times out of ten with a fresh container, it's a storage or network bridge that doesn't match what you selected during creation.

Best Practices

A few habits that will save you a headache later:

• Keep this container dedicated to Ollama. Don't pile a web server, a database, and Ollama into the same box — separate containers are cheap on Proxmox, and isolation is most of the point of doing this at all.
• Take a snapshot right after you've got Ollama working. If you break something experimenting with models or configs later, you'll be glad you can roll back in seconds instead of rebuilding from scratch.
• Don't expose port 11434 directly to the internet. If you want remote access, put a reverse proxy with authentication in front of it instead.
• Start small. It's tempting to immediately pull the biggest model you can find — resist that until you know how your specific hardware handles the small ones first.

Frequently Asked Questions

Do I need a GPU for this?
No. Every step here runs on CPU only. A GPU will make responses faster, but it's not required to get a working setup.

Why an LXC container instead of a VM?
Ollama is a plain Linux binary with no special hardware needs on CPU, so a container gives you the same isolation with a fraction of the overhead. Use a VM instead only if you specifically need GPU passthrough, since that's more mature on full VMs.

Which model should I start with?
llama3.2:1b is the friendliest starting point on modest hardware. Move up to a 3B model once you've confirmed the basics work and you want sharper answers.

Can I use this instead of ChatGPT or Claude?
For quick, private, low-stakes tasks, sure. For serious reasoning, long documents, or coding help, the small local models still lag well behind the big hosted ones — treat this as a complement, not a replacement.

Is my data actually private?
Yes, in the sense that nothing leaves your network by default. Just remember that opening the API to your LAN means anyone on that LAN can use it too, so treat it like any other unauthenticated service on your network.

Conclusion

What you've got now is a small, disposable Linux container that happens to be running a real AI model, fully under your control. That's genuinely worth appreciating — a few years ago this wasn't something you could casually do on a homelab box over a lunch break.

From here, the natural next steps are trying a couple of different models to see how they compare, or looking into a proper chat interface like Open WebUI if the command line isn't your thing (that's a Docker-based project, so it's a good excuse to turn on nesting for this container and explore it separately). Either way, you've already done the part that trips people up the most: getting Ollama running cleanly inside Proxmox in the first place.