Local AI & Private Cloud
run it in-house, govern it
This track teaches you how to keep AI workloads inside your own infrastructure and obey data‑residency rules. It is aimed at research groups, small‑to‑medium enterprises, or institutes that need to decide where their data lives and still run modern models. After completing the chapters you will be able to install Docker on macOS, Linux, and Windows; evaluate and apply a governance checklist for private AI; use EU‑hosted inference endpoints that comply with GDPR; run open‑weight models locally so no data leaves your machine; set up a self‑hosted private cloud with Dokploy either on Hetzner or on‑premises; and finally deploy applications using Docker containers, images, and volumes. Start with the Docker installation chapters for the operating system you use (macOS, Linux, Windows) to get comfortable with containers. Then move to "Private AI for your org" and "EU‑sovereign inference" to understand governance and compliant model access. Follow with "Local & Private Models" before tackling the self‑hosting sections (Docker on Linux, Self‑host on a private cloud, Self‑hosting with Dokploy). End with "Deploy & Run Applications with Docker" to tie everything together. If you are short on time, skip the OS‑specific Docker chapters you do not need and go straight from the governance checklist to self‑hosting with Dokploy.
When your own hardware isn't enough, an inference provider runs open-weight models for you and serves them over an API — but "the cloud" is not one jurisdiction. IONOS AI Model Hub, Scaleway and OVHcloud are EU companies running EU data centers: the same OpenAI-compatible base-URL swap as any US provider, but GDPR-native by default, with no US CLOUD Act exposure to explain to a data-protection officer. Mistral Medium is the EU-hosted flagship model to pick if you want quality, not just jurisdiction. Renting an EU GPU (e.g. via Hetzner) to self-host runs roughly a third of the equivalent hyperscaler price. Groq stays in this chapter for one reason: it is the fastest hosted option, US-based, and the explicit "convenience over data-residency" contrast — reach for it when speed matters more than where the data sits, never as the default for institutional data.
- 12.1 Private AI for your org: buy, build & govern The data-residency spectrum and the governance checklist for an institution 2026-08-06 17
- 12.2 EU-sovereign inference GDPR-native, EU-hosted model access — before you reach for a US convenience API 2026-08-06 80
- 12.3 Local & Private Models Run AI on your own machine — nothing leaves it 2026-08-06 249
- 12.4 Self-host on a private cloud Stand up a host you control — Dokploy on Hetzner or on-prem — and promote to it safely 2026-08-06 10
- 12.5 Self-hosting with Dokploy on Hetzner Your own git-push deploy platform — on a €8.49/month box you control 2026-08-06 25
- 12.6 Deploy & Run Applications with Docker overview Containers, images, and volumes — the mental model behind every local AI tool you install 2026-08-06 19
- 12.6.1 Docker on macOS Install Docker Desktop on Mac — Apple Silicon vs Intel, and your first container 2026-08-06 65
- 12.6.2 Docker on Linux Install Docker Engine on Linux — package manager, permissions, and your first container 2026-08-06 84
- 12.6.3 Docker on Windows Install Docker Desktop on Windows — WSL2, troubleshooting, and your first container 2026-08-06 43
- 12.7 Serving a model to your whole team Why the tool that got you started stops at user two 2026-08-13 0
- 12.8 The LLM gateway One key, many models, and a budget that actually holds 2026-08-13 30
- 12.9 Running untrusted AI code safely Your agent writes code. Do not run it here. 2026-08-13 11
- 12.10 Quantization as an engineering decision Somebody already chose your model's precision. It should be you. 2026-08-13 22
- 12.11 Many models, one GPU The first wall every home lab and small server hits 2026-08-13 0
- 12.12 LiteLLM Hands-on: stand up your own gateway, then break it on purpose 2026-08-13 12