Designer / Operator · 2025 – present
Production AI Infrastructure — Self-Hosted
The self-hosted infrastructure my AI work runs on: a multi-host Proxmox cluster serving local LLM inference to a production agent fleet and two operating businesses, daily.
The problem
Serious agentic work needs serious inference: long contexts, predictable latency, and data that stays on hardware you control. Cloud APIs are the right tool for some jobs — and the wrong dependency for a practice built on sovereignty and always-on operations.
The build
- Built a multi-host Proxmox cluster on enterprise servers (Lenovo, Supermicro EPYC, Dell PowerEdge) with an RTX 3090 GPU pool.
- Serve local LLM inference with vLLM: tensor-parallel serving, FP8 KV cache, 262K-token context, speculative decoding, and prefix caching.
- Connected the cluster with 40GbE networking and RDMA for low-latency GPU and cluster traffic; run container workloads on k3s (lightweight Kubernetes).
- Secured remote access and network segmentation with Tailscale — the same in-VPC / on-prem pattern deployed for consulting clients.
The evidence
- This infrastructure serves the Prismatic agent fleet and two operating businesses every day — production load, not a hobby lab.
- 262K-token context windows make long-horizon agent tasks practical on local models.
- Owning the stack end to end means inference cost, privacy, and availability are engineering decisions, not vendor terms.
Most consultants rent their AI. I run mine. The homelab is a production environment — enterprise servers, a GPU pool, high-speed interconnect, and local inference tuned for the long-context, long-horizon work agent fleets actually do. It is also the reference deployment for the in-VPC pattern I install for BeyondSaaS clients: AI capacity that lives inside the owner’s walls, under the owner’s control.