inference

Running Gemma 4 on a Self-Hosted vLLM Server from Antigravity — A Practical Guide to Building High-Throughput Inference

Gemma 4 runs beautifully in Ollama for a single user. The moment a handful of teammates start hitting the same endpoint, throughput collapses. This guide walks you through replacing Ollama with a self-hosted vLLM server, wiring it into Antigravity as a custom endpoint, choosing a quantization level with real numbers, and hardening the stack with rate limits and LoRA routing — end to end.