Deployment · Hardening · Operations

Self-Hosted AI API Gateway
Deployment & Operations

I deploy, harden and operate self-hosted AI API gateways for teams that want one controlled entry point for their LLM traffic — with their own keys, their own logs, and their own infrastructure.

Hire me on Upwork See the live demo

deploy · verify · hand over
# every deployment ends with proof, not promises
$ deployctl preflight --remote gateway-host
ok  ports 80/443 free · docker 27.x · 4GB free
$ deployctl apply --plan plan.json
ok  container healthy · bound 127.0.0.1 only
$ deployctl verify --target gateway-host
ok  TLS A · HSTS · headers 6/6 · logs redacted
# handover: runbook + backups + your keys, not mine

The problem

Keys everywhere

Provider API keys pasted into notebooks, CI variables and desktop apps. Nobody knows which key is used where, or what it would cost to rotate one.

No spend visibility

The invoice arrives at month end. Which team, which feature, which runaway script spent the budget? The provider dashboard cannot tell you.

Provider lock-in at the code level

Every service hardcodes one vendor SDK. Switching models — or surviving an outage — means a code change and a deploy.

What a gateway fixes

A self-hosted gateway (open-source projects such as LiteLLM, one-api or Sub2API-class deployments) gives you one OpenAI-compatible endpoint in front of every upstream you use. Your services get virtual keys you can create, cap and revoke; upstream keys stay in one vault. You get per-key usage, rate limits, model routing and failover — on your own server.

What I deliver

Deployment

Docker Compose or systemd deployment, reverse proxy with HTTPS and HSTS, non-root containers, a documented directory layout, and an upgrade path that has been tested, not guessed.

Hardening

Key vaulting with restricted permissions, admin plane separated from the data plane, security headers, fail2ban, log redaction so secrets never land in plaintext logs.

Operations

Health checks and alerting, usage dashboards, backup and restore drills with a written runbook, quota policies per team and per key.

Handover

You own everything at the end: infrastructure, documentation, and a recorded walkthrough. No dependency on me to keep it running.

Scope boundaries, stated up front

I deploy gateways that route traffic under your provider accounts and API keys (or keys your organisation is licensed to use). I do not build or operate systems for reselling consumer subscriptions as APIs, sharing or pooling account quotas, bypassing provider rate limits, or extracting credentials. See the compliance page for the full matrix.