[MS]
Resume ↓

AI Infrastructure

Applied infrastructure, not a hobby list

A self-hosted AI stack built the same way I build everything else: local first, with an explicit policy for what's allowed to leave the building. It runs on the same Proxmox cluster as the rest of the lab. The diagram below is the whole flow; each section after it goes one level deeper.

Request flow: Mac Studio hardware runs local models, served through a LiteLLM gateway that routes sensitive workloads to local inference only, and escalates non-sensitive requests to external APIs. Both paths feed Open WebUI, which powers Wazuh rule tuning and the Hermes agent framework. Mac Studio M2 Ultra · 128GB unified memory · oMLX inference Local Models Qwen3.6-27B · Qwen3-Coder-Next Gemma 4 · embedding models LiteLLM Gateway Routing + virtual keys, running as a Proxmox LXC Local Inference Sensitive workloads never leave the cluster External APIs Escalation path only no sensitive data, ever Open WebUI User-facing interface Wazuh rule tuning Hermes agents

Hardware & Inference

Where the models actually run

PRIMARY DRIVER

Qwen3.6-27B

The default model for day-to-day reasoning and drafting.

CODING

Qwen3-Coder-Next

Swapped in for anything code-shaped.

GENERAL

Gemma 4

A second opinion, run alongside Qwen for comparison.

RETRIEVAL

Embedding models

Backing search and retrieval across the self-hosted stack.

Serving & Routing

A gateway with a policy, not just a proxy

The gateway itself is one piece. The policy behind it is the part worth a case study.

For anyone whose data I touch

Sensitive-data AI routing policy

A rule, enforced by software I control, that keeps anything private or client-related from ever reaching an outside AI company.

Self-hosted
  • LiteLLM
  • Data Governance
  • Proxmox LXC

Interface & Tools

The rest of the stack

Open WebUI
SearXNG
jina-reranker-v3
Playwright fetcher
Flux.1 Dev + ComfyUI
Kokoro

Applied to Security

Pointed at the SIEM, not just chat

The most useful application so far is boring on purpose: using the local models to draft tuned local_rules.xml configurations from noisy Wazuh alert output — a step toward alert-triage, SOC-copilot style tooling, running on the same Wazuh deployment covered in Experience.

Agent Orchestration

A self-hosted agent layer with memory

Alongside the chat interface, I built and deployed Hermes, a self-hosted messaging gateway and agent framework running in a Proxmox LXC, with Hindsight handling memory persistence so context carries between sessions instead of resetting every time. Same governance rules apply: sensitive routing stays local.