The first thing that tripped me up after wiring Gemma 4 into Antigravity as a local model was simple: the official repo lists Q4_K_M, Q5_K_M, Q8_0, and fp16 side by side, but which one should I actually run on my Mac? The README says "Q4_K_M is the balanced choice" and stops there. After a few days on a 16GB M2 Mac, I learned that "balanced" assumes a server-shaped workload — not the way a solo developer uses Antigravity.
What follows is a week of measurements running each Gemma 4 quant alongside Antigravity, doing real work: code completion, doc generation, and summarization. Short version: for solo development through Antigravity, Q5_K_M is the practical pick. Below is why.
Why Compare on Three Axes Instead of Benchmarks
Most quantization comparisons stop at MMLU or HumanEval scores. Useful, but they miss the texture of running a local model alongside an editor like Antigravity all day. The three axes that actually mattered to me:
- Speed (tokens/sec) — slow completions break my flow
- Memory (RSS) — can it coexist with Chrome, Slack, and Antigravity itself?
- Output quality — does the code it writes actually compile and match my project's idioms?
Benchmarks are a reasonable starting point, but on a personal machine, the headroom for "everything else" matters as much as raw inference speed.
Numbers From an M2 Mac (16GB), llama.cpp Backend
I ran a llama.cpp server, hooked it up to Antigravity through an OpenAI-compatible MCP endpoint, and ran the same five-prompt suite per quant. Hardware: Apple M2 / 16GB / macOS 14.5, llama.cpp v0.6.x.
- Q4_K_M (~4.4GB) — 38 tokens/sec, RSS 5.2GB. Code generation passes ~80% of the time, but type inference falls apart on more complex refactors.
- Q5_K_M (~5.4GB) — 32 tokens/sec, RSS 6.3GB. Code generation passes >90%, and JSDoc / comment generation reads noticeably more natural than Q4.
- Q8_0 (~8.2GB) — 20 tokens/sec, RSS 9.1GB. Quality is essentially indistinguishable from fp16, but opening Antigravity + Chrome together pushes the system into swap.
- fp16 (~15.3GB) — 12 tokens/sec, RSS over 16GB. Not viable on a 16GB Mac; the OS itself stalls under memory pressure.
Q4_K_M is the fastest on paper, but I still ended up on Q5_K_M. Here's why.
Why I Settled on Q5_K_M — Stability Over Raw Speed
When you use code completion in Antigravity, every wrong suggestion costs you the time to undo and redo. Saving 6 tokens/sec on Q4_K_M doesn't help if the output is subtly off — that's a net loss.
Concrete example. I asked the model to generate a Zod schema in TypeScript. Q4_K_M occasionally produced z.email() (Zod v4 syntax) when my project is on v3, breaking the file. Q5_K_M produced the correct z.string().email() on every run with the same prompt. Across dozens of completions a day, that drop in "almost-but-not-quite" suggestions matters far more than the tokens/sec gap.
# The llama.cpp server command I actually use (Q5_K_M)
./llama-server \
-m ~/models/gemma-4-9b-it-Q5_K_M.gguf \
--port 8080 \
--ctx-size 8192 \
--threads 8 \
--n-gpu-layers 99 \
--host 127.0.0.1
# Expected output: HTTP server listening on 127.0.0.1:8080
# RSS settles around 5.8GB at startup, ~6.3GB during inferenceWith the server up, point Antigravity's MCP config at http://127.0.0.1:8080/v1/chat/completions as an OpenAI-compatible endpoint and your in-editor completions flip over to local Gemma 4.
Antigravity-Side Settings That Actually Matter
The quant choice is half the story. The settings on the Antigravity side affect feel just as much.
First, cap max_tokens around 2048. The default is sometimes 4096, but local models accumulate small errors as outputs get longer, and longer outputs feel slower too. For completion-style work, 1024–2048 is plenty.
Second, drop temperature to 0.2–0.3. At the default 0.7, Q5_K_M sneaks in slightly noisy suggestions; cooler settings keep code generation tighter.
// Antigravity MCP config snippet (settings.json)
{
"mcpServers": {
"gemma4-local": {
"url": "http://127.0.0.1:8080/v1/chat/completions",
"model": "gemma-4-9b-it-Q5_K_M",
"maxTokens": 2048,
"temperature": 0.25,
"stop": ["\n```\n", "</answer>"]
}
}
}
// Effect: tighter completions, fewer trailing explanationsThe stop tokens are a small but real win — they prevent Gemma 4 from drifting into "by the way, here's a quick explanation of the code…" territory after the answer is already written.
A Detail Most Comparisons Miss: Context Window vs Memory Pressure
Quantization isn't the only knob. The --ctx-size flag on llama.cpp also competes for the same RAM budget. I started with 16384 thinking "more context can't hurt," but on Q5_K_M with a 16GB Mac it pushed RSS past 9GB and the system started thrashing whenever I had a video call running.
Dropping --ctx-size to 8192 brought RSS back down to ~6.3GB and didn't hurt my actual usage at all — most of my prompts to Antigravity are under 4k tokens anyway. The lesson I keep relearning: tune for the prompts you actually send, not the worst case you imagine.
If your workflow does send long contexts (say, asking the model to read a whole file), 12288 is a reasonable middle ground on Q5_K_M without sliding into swap territory.
What I Use Q4_K_M For Anyway
I'm not anti-Q4. There are two situations where I still prefer it:
- On the road, on battery — Q4_K_M draws less power and runs cooler. On a long flight with no outlet, the extra hour of battery is worth the slight quality dip.
- Bulk doc generation — when generating many short README sections in batch, the speed advantage compounds and the per-output errors are easier to spot in review than during real-time completion.
For interactive editor work, though, Q5_K_M wins on a daily basis.
Don't Take "Q4_K_M Is Recommended" at Face Value
The official quantization guidance is, I suspect, written from a server-shaped perspective: many concurrent users, scarce GPU VRAM, multiple models sharing a card. In that world, Q4_K_M absolutely earns its keep.
A solo developer's Mac is a different shape. With 16GB you have nearly 10GB free even with Q5_K_M loaded, plus Antigravity, a browser, and Slack still humming along. The "balanced" recommendation shifts when the deployment context shifts — an obvious thing, perhaps, but worth confirming with your own measurements before committing.
I spent three days following the official advice on Q4_K_M. The cumulative drag of "almost-right" completions added up. After switching to Q5_K_M the undo count visibly dropped, and the difference has held over a week of daily use.
Related Reading
- Gemma 4 mobile memory budget design patterns
- Fixing Gemma 4 OOM in Antigravity
- Gemma 4 local development setup
What to Try Next
If you're also running Gemma 4 through Antigravity on a 16GB Mac, download the Q5_K_M weights, start llama-server with the command above, and use it for half a day of code completion. Compare how often you have to undo a suggestion versus Q4_K_M on your own codebase — that's the fastest way to verify the choice for yourself.
I'm still watching how this changes once Gemma 5 lands, but the small conviction I'd share for now is this: a week of real use beats a benchmark scoreboard when you're picking a quantization to live with.