Install Ollama on your VPS
Infraa installs Ollama on your VPS as a pinned, CPU-only build of the official release — run open models like Llama, Gemma, and Qwen behind a simple HTTP API (including an OpenAI-compatible endpoint), with your prompts and data never leaving the box. The artifact is verified against a checked-in SHA-256, runs as a dedicated non-root system user, and binds to localhost only. Be clear-eyed about the hardware: inference runs on the CPU, so it suits small models and background jobs, not fast interactive chat.
- Versions
- Ollama 0.31.1 — Infraa's CPU-only repack of the official release (GPU libraries stripped), verified by SHA-256
- Works on
- Ubuntu 22.04, Ubuntu 24.04, and Debian 12
- Package source
- Pinned artifact (CPU-only repack of the official release)
What does the Ollama install do?
- Creates a dedicated, non-login system user (infraa-ollama) — Ollama never runs as root.
- Downloads the pinned CPU-only build (~31 MB instead of the official 1.4 GB GPU bundle) into /opt/infraa/apps/ollama, verifying it against a SHA-256 checksum that ships in the reviewed catalog.
- Installs a systemd unit (infraa-ollama.service) that runs ollama serve bound to 127.0.0.1 on your chosen port, with models stored in /var/lib/infraa-ollama.
- Enables and starts the service, then verifies it's listening — it starts with no models; you pull your first one with a single API call.
How do I install Ollama with Infraa?
Connect your server
Run the Infraa install one-liner (curl install.infraa.sh | sh) on your VPS. The agent registers over an outbound connection in about 30 seconds — no inbound ports, no SSH keys to share.
Pick the service
Choose it from the one-click catalog in the dashboard, or just ask in chat. Infraa turns your request into a concrete, reviewable plan.
Review and approve
Read exactly what will run, then approve. The agent executes only the approved, cryptographically signed steps and streams progress live.
What happens after the install?
Infraa checks Ollama every 60 seconds — the service unit and your chosen port. Pull a model over SSH (curl the localhost API) to get started; begin with a 1–3B model and only go bigger if your RAM allows — a quantized 7–8B model wants ~8 GB of RAM and generates roughly 1–10 tokens per second on typical VPS CPUs. The Ollama API has no authentication, so it stays localhost-only; to expose it, install the nginx module and put access control at the proxy.
Ollama on Infraa — common questions
01How fast is Ollama on a VPS without a GPU?
Honest answer: slow for interactive use. This is a CPU-only install — a quantized 7–8B model generates roughly 1–10 tokens per second on typical VPS CPUs, which works for background jobs, summaries, and low-volume internal tools but frustrates live chat. Small models (1–3B) are considerably faster. There is no GPU support in this build; if you need fast inference, use a GPU host or a hosted API.
02How much RAM and disk does it need?
RAM is the hard limit: a quantized 7–8B model wants ~8 GB, while 1–3B models run in 2–4 GB — if the model doesn't fit in RAM it will crawl or fail. Models are large downloads (a 7–8B is ~4–5 GB) stored under /var/lib/infraa-ollama, so budget 10 GB+ of free disk.
03Is the Ollama API exposed to the internet?
No, and that's deliberate: the Ollama API has no authentication, so anyone who could reach it could run and download models on your server. The unit binds to 127.0.0.1 on your chosen port (11434 by default). To expose it publicly with a domain and TLS, install the nginx module, point a managed reverse-proxy vhost with Let's Encrypt at the port, and restrict access at the proxy — for example with basic auth or an IP allowlist.
04What exactly is Infraa's CPU-only build?
The official Ollama release for Linux is a 1.4–1.5 GB archive, most of which is GPU libraries (CUDA/Vulkan) that a GPU-less VPS never loads. Infraa re-packages the official release bytes with those GPU directories removed — the binary itself is unmodified — using a reproducible script, and pins the result by SHA-256 in the reviewed catalog. The agent verifies the checksum before anything is installed.
Connect a server in 30 seconds, then install Ollama from the catalog or just ask in chat.
