Antigravity × Ollama: Gemma 4 in Your Editor

Pointing Antigravity at Ollama

In Antigravity's config (antigravity.config.json or the matching env vars), declare Ollama as an OpenAI-compatible provider:

{
  "llm_providers": [
    {
      "name": "local-gemma",
      "type": "openai_compatible",
      "base_url": "http://localhost:11434/v1",
      "api_key": "ollama",
      "default_model": "gemma3:12b"
    },
    {
      "name": "cloud-gemini",
      "type": "google_gemini",
      "api_key": "YOUR_GEMINI_API_KEY",
      "default_model": "gemini-2.5-pro"
    }
  ],
  "routing": {
    "agents": {
      "doc-summarizer": "local-gemma",
      "code-generator": "cloud-gemini"
    }
  }
}

The api_key is set to the literal string "ollama" on purpose. Ollama doesn't authenticate, but many OpenAI-compatible clients refuse to send a request when the key is empty. Any non-empty string works; "ollama" is a convention.

Routing Per-Agent

The snippet above sends a summarizer agent to Ollama and a code generator to Gemini. That split reflects what I reach for in practice:

• Keep local: summaries of sensitive data, first-pass filtering of large logs, cost-bound batch jobs, work done on spotty or offline networks
• Send to cloud: code generation where accuracy matters, translations that need multilingual nuance, answers that benefit from up-to-date information

"All local" and "all cloud" are both wrong defaults. Split by task.

Sizing the Model From the Memory You Actually Have

What size fits your hardware determines whether the local route is tolerable or painful. Here's my working sense:

One note on tags before you go further: later sections use names like gemma4:9b, but registry tags shift between generations. Run ollama list and ollama show <tag> to confirm the exact tag and parameter count on your machine before wiring anything up. I keep the tag out of config files entirely and expose it through an environment variable such as ANTIGRAVITY_LOCAL_MODEL, so a generation change means editing exactly one place.

On an M2 Max 64GB, gemma3:27b lands around 15–20 tokens/s and gemma3:12b hits about 40. On an RTX 4090 Linux box, 12B goes 60–80 tokens/s.

Quantization matters too. q4_0 is the default; q5_K_M improves quality at some memory cost. My working baseline is 12b q5_K_M, sizing up or down based on available VRAM.

Fallback Routing — Slow Failure Hurts More Than Clean Failure

The main reason to keep a local LLM in the loop is insurance for when the cloud side becomes unavailable. But after running this setup for a few months as a solo indie developer, the failure that actually cost me time was not the one I had planned for. It was the opposite direction: local Ollama stopped answering, and my router kept sending traffic to it anyway.

Model loading. Memory pressure pushing the box into swap. Hitting the OLLAMA_MAX_LOADED_MODELS ceiling so another model gets evicted mid-request. In all three cases Ollama still accepts the connection — it just never replies. An error you can branch on. Silence you cannot.

On the Antigravity side you declare the fallback target roughly like this:

{
  "routing": {
    "default": "cloud-gemini",
    "fallback": "local-gemma",
    "fallback_triggers": ["network_error", "rate_limit", "api_error"]
  }
}

As a declaration that is enough. What it actually does under load, though, is not visible from the file itself.

So I measured a naive fallback against one that remembers failures. The target is not Ollama itself but a local stub that returns OpenAI-compatible responses, because what I wanted to see is not inference speed — it is how many times, and how many milliseconds, the router keeps paying to the broken side. Local was pinned to two failure modes ("returns 503 instantly" and "accepts the connection and goes quiet"), cloud to one healthy mode answering in 40ms.

The naive version

import json, time, urllib.request
 
def call(url, timeout):
    req = urllib.request.Request(
        url, data=json.dumps({"messages": []}).encode(),
        headers={"content-type": "application/json"})
    with urllib.request.urlopen(req, timeout=timeout) as r:
        return json.load(r)
 
class Naive:
    """Try local every time; fall through to cloud on any failure."""
    def __init__(self, local, cloud, timeout):
        self.l, self.c, self.t = local, cloud, timeout
 
    def send(self):
        try:
            return call(self.l, self.t), "local"
        except Exception:
            return call(self.c, self.t), "cloud"

This is roughly what the routing JSON above does under the hood. It is functionally correct — requests still come back when local is dead.

The version that remembers

class Breaker:
    """Open after fail_max consecutive failures; skip local entirely for cooldown seconds."""
    def __init__(self, local, cloud, timeout, fail_max=2, cooldown=2.0):
        self.l, self.c, self.t = local, cloud, timeout
        self.fail_max, self.cooldown = fail_max, cooldown
        self.fails, self.open_until = 0, 0.0
        self.skipped = 0
 
    def send(self):
        if time.monotonic() < self.open_until:
            self.skipped += 1                    # straight to cloud, no local attempt
            return call(self.c, self.t), "cloud"
        try:
            r = call(self.l, self.t)
            self.fails = 0                       # one success clears the memory
            return r, "local"
        except Exception:
            self.fails += 1
            if self.fails >= self.fail_max:
                self.open_until = time.monotonic() + self.cooldown
            return call(self.c, self.t), "cloud"

The only difference is that it remembers recent failures and stops probing for a while.

Fast failure and slow failure split the results

Twelve requests per condition, local timeout fixed at 0.5s.

When the error comes back immediately, the two are indistinguishable — 0.6ms apart. Testing only that condition and concluding "we're fine" is exactly what I had been doing.

The gap opens when nothing comes back. The naive router pays the full 0.5s timeout on every single request, 4.3x total. And it buys nothing with that time: it is re-purchasing the same fact — local is down — twelve times over.

If an agent turn makes five LLM calls, that 0.5s becomes 2.5s of dead time per turn. Some of what I had been experiencing as "Antigravity feels sluggish today" was living right here.

A shorter cooldown costs more while local is down

So what should cooldown be? I ran requests continuously for an 8-second window and counted how many completed, with local staying silent throughout.

A 0.5s cooldown completed 41% fewer requests than a 5.0s one. Probing frequently to catch recovery early means each probe sits through a timeout.

What you buy with that is recovery speed. At cooldown 2.0s, I measured the delay between local actually recovering and the router's first successful local call four times: 0.32 / 0.95 / 1.63 / 2.26 seconds (median 1.29s). Probe timing and recovery timing are independent, so the delay spreads evenly across the interval. Set cooldown to n seconds and recovery detection lags by n/2 on average, n at worst. That's the whole trade.

Looking at those numbers, I settled on 2 seconds. Loading a local model takes several seconds at best, so probing at 0.5s intervals mostly arrives too early to matter. Stretching to 5 seconds, on the other hand, made the pause after restarting ollama serve noticeable enough to bother me. Two seconds is where those two pressures met.

One more thing: I set the local timeout distinctly shorter than the cloud one. Local exists because it is fast. If it makes me wait longer than the cloud, there is no reason to route there at all. My box gives up on local at 0.5s and gives the cloud its normal budget.

Coming back to that JSON config above: fallback_triggers lists network_error, rate_limit, and api_error. All three are failures where the other side hands you something. The failure where nothing comes back is not on the list at all. What you can express in a config file and what actually happens in production are different sets — measuring is what made that land for me.

Operational Gotchas Nobody Warns You About

Context length: Ollama's default is 2048 or 4096 tokens. Antigravity agents with long histories get truncated silently. Set OLLAMA_CONTEXT_LENGTH, or add num_ctx 16384 to the Modelfile. Memory grows with context length — keep an eye on it.

Cold starts: Ollama loads a model into memory on the first request, so the initial response can be 10–30 seconds late. OLLAMA_KEEP_ALIVE=-1 pins the model in memory for the session, which feels dramatically snappier in interactive use.

Parallel agent memory pressure: When Antigravity runs many agents concurrently, Ollama may try to load several models simultaneously and blow past your RAM budget. Cap concurrent loads with OLLAMA_MAX_LOADED_MODELS, and design so that critical agents share a single model rather than each having its own.

Gemma 4 Itself — Improvements Over Earlier Generations

Before getting deeper into the integration, it helps to understand what's actually different about Gemma 4. Knowing context length, strengths, and weaknesses up front makes model sizing and task routing easier later on.

Gemma 4 delivers substantial improvements:

• Architecture: More efficient 7B/9B variants (20% faster inference than Gemma 3)
• Accuracy: 8.5% higher on benchmarks (especially code generation and JSON parsing)
• Multimodal: Now accepts text AND images (Gemma 3 was text-only)
• Context Length: Expanded from 32K to 128K tokens
• Localization: Japanese language performance improved 3.2x with expanded training data
• Latency: Runs in 2-5 seconds on modern hardware

The spike in "gemma 4 antigravity" search queries reflects strong demand from developers seeking efficient, privacy-preserving AI solutions.

Steering Local/Cloud Switching with AGENTS.md

Antigravity's agent configuration (AGENTS.md) lets you set per-agent rules for which model to use. Sensitive data processing can be forced to local, offline mode can kick in when the network can't be reached — operational requirements can drive routing.

Dropping an AGENTS.md file at the workspace root lets Antigravity route between models per task instead of picking one globally.

# Agent Configuration
 
### Default Model
Use `gemini-3-pro` for planning and complex reasoning.
 
### Privacy-Sensitive Tasks
Use `gemma4-local` for:
- Files under `private/`
- Anything containing customer data
- First-pass drafts (cloud model reviews before ship)
 
### Offline Mode
If network is unavailable, fall back to `gemma4-local` for all tasks.

The hybrid pattern — sensitive work local, heavy reasoning cloud — is what I've found actually sustainable. Fully-local is rarely practical; fully-cloud misses the privacy and cost wins.

Inference Performance Tuning

Local inference speed varies dramatically with CPU/GPU resource management. Here are the adjustment points that matter for getting practical response times.

The default 9B quantization is fine on most hardware, but three tweaks matter:

Quantization level. Ollama defaults to Q4_K_M. On Apple Silicon M3+ or a machine with spare VRAM, pull q5_K_M or q6_K for better quality at almost the same speed:

ollama pull gemma4:9b-q5_K_M

Shrink the context window to what you actually need. The model ceiling (128K tokens on Gemma 4) and the num_ctx Ollama actually allocates are different numbers. The default is 2048 or 4096, and pushing it to the ceiling grows the KV cache proportionally, which costs memory and inference time directly. I measure the real history length of the tasks I run, then set num_ctx to roughly 1.5x that.

Keep the model warm. OLLAMA_KEEP_ALIVE=30m prevents Ollama from unloading the model after five idle minutes. Cold loads cost a few seconds each time; this eliminates that tax for frequent callers.

Practical Development Workflows — Where Antigravity × Local Gemma 4 Pays Off

The typical workflows for Antigravity development with local LLMs in the loop, and how Gemma 4 actually behaves in each scenario.

Pattern 1: Local by Default, Cloud for Complex Decisions

Handle the high-frequency, lower-complexity parts of development locally (completions, fixes, test generation), and reach for cloud models only when the task genuinely demands it.

# Tasks well-suited to Gemma 4 locally
 
# 1. Code completion
def parse_config(file_path: str) -> dict:
    """Load a configuration file and return a dict."""
    # ← Antigravity (Gemma 4) fills this in on-device
 
# 2. Docstring generation
# ← Point at the function, ask for a docstring → local inference
 
# 3. Bug fixes from error messages
# ← Paste the traceback, get a suggested fix → local inference

# Signals to switch to a cloud model
- Discussing project-wide architecture across many files
- Planning a refactor spanning 10+ files
- Security threat modeling (higher stakes, worth the cloud cost)
- API design review before a public release

Pattern 2: Fully Air-Gapped Workflow for Confidential Projects

For projects where code absolutely cannot leave the machine, set Antigravity to local-only mode via environment variables:

# .env for a confidential project
ANTIGRAVITY_MODEL=local
ANTIGRAVITY_ALLOW_CLOUD=false
ANTIGRAVITY_LOCAL_ENDPOINT=http://localhost:11434/v1
ANTIGRAVITY_LOCAL_MODEL=gemma4:26b
 
# With ALLOW_CLOUD=false, Antigravity won't attempt any external requests
# regardless of what operation is performed

Pattern 3: Offline Development (Travel, Flights, Remote Sites)

Download models before going offline. Antigravity detects connectivity loss and falls back to the local model automatically.

# Before going offline
ollama pull gemma4:e2b   # Essential for basic development
ollama pull gemma4:e4b   # If disk space allows
 
# During offline work: Antigravity routes automatically to the local model

Gemma 4 Coding Performance Benchmarks

The first thing most teams ask about a local LLM is code generation quality. Here are Gemma 4's numbers on standard benchmarks (GSM8K, HumanEval, MBPP), reconciled against actual usage feel.

Measured using Gemma 4 E4B on M3 Max MacBook Pro with 64 GB RAM:

Python code completion: ~2.1s average (vs Claude 3.5 Sonnet: ~85% quality, ~1.3× faster)

TypeScript function implementation: ~4.3s average (vs Claude 3.5 Sonnet: ~80% quality, ~0.9× speed)

Bug diagnosis and fix: ~3.8s average (vs Claude 3.5 Sonnet: ~75% quality, ~1.1× faster)

Documentation generation: ~5.2s average (vs Claude 3.5 Sonnet: ~88% quality, ~1.2× faster)

For daily coding assistance, Gemma 4 E4B produces usable-to-good output across most tasks. Given that the API cost drops to zero, using it as the default model and reserving cloud AI for high-stakes decisions is a sound strategy.

Where Gemma 4 Wins, Where Gemma 4 Loses — Task Routing Heuristics

In a hybrid local/cloud setup, the key decision is "what stays local." Here are the areas where Gemma 4 reliably holds up, and the areas where you should probably route to the cloud.

Gemma 4 is an impressively capable open-weight model, but it has real limitations worth knowing.

Where it excels: Code generation across 140 languages, short-context understanding and modification, docstring and comment generation, and reading Japanese technical documentation. For all of these it performs close to hosted models.

Where cloud models still win: Very long contexts (understanding 10,000+ lines of code as a whole), knowledge of frameworks released after the training cutoff, and nuanced architectural judgment calls where the reasoning chain is long and ambiguous.

This makes a local-first, cloud-fallback hybrid the most practical approach for most development environments — local Gemma 4 for routine work, cloud for the tasks that genuinely benefit from it.

Pitfalls I Hit in Real Use

Local LLM integration has more "designed-fine-but-broke-in-practice" cases than most setups. Here are the operational issues that come up most often, and what fixed them.

Three lessons the tutorials miss:

Japanese (and other non-English) quality is noticeably worse. Gemma 4 9B is English-trained at its core. For polished long-form Japanese generation, either 27B or a cloud model is the honest answer. Short structured tasks are fine.

Tool calling is flaky at 9B. Local LLMs across the board struggle with complex tool-call JSON. If you define more than a couple of tools, expect occasional malformed arguments. Reserve local models for tasks with one or two well-specified tools; leave the many-tool orchestration to cloud models.

Laptop thermals are a real constraint. Running 9B continuously on battery gets hot and dies in 2–3 hours. If this is a daily driver, plug in or run Ollama on a separate always-on box.

Stop Trusting My Numbers — Measure TTFT and tok/s on Your Own Box

Every speed figure above comes from my hardware. As an indie developer I have only a couple of machines to test on, and moving the same setup between them changed the feel enough to surprise me. Two machines with the same chip will diverge based on memory bandwidth and thermal headroom, and raising num_ctx slows everything down regardless. After redoing this exercise a few times, my conclusion is simple: reading someone else's benchmark is slower than running your own.

There are two numbers worth capturing. TTFT (time to first token) and tok/s (generation throughput). For inline completion in the editor, perceived speed is almost entirely TTFT; tok/s only starts to matter once you're generating long output. Collapsing both into a single "fast or slow" judgment is how model selection goes wrong.

Ollama's OpenAI-compatible endpoint supports streaming, so the standard library is enough to capture both.

# bench_ollama.py — measure TTFT and tok/s per model
import json
import time
import urllib.request
 
ENDPOINT = "http://localhost:11434/v1/chat/completions"
PROMPT = "Add a docstring to this function:\ndef parse_config(path):\n    ..."
 
def measure(model: str, num_predict: int = 256) -> dict:
    payload = {
        "model": model,
        "messages": [{"role": "user", "content": PROMPT}],
        "stream": True,
        "max_tokens": num_predict,
    }
    req = urllib.request.Request(
        ENDPOINT,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json", "Authorization": "Bearer ollama"},
    )
 
    started = time.perf_counter()
    first_token_at = None
    tokens = 0
 
    with urllib.request.urlopen(req) as res:
        for raw in res:
            line = raw.decode("utf-8").strip()
            if not line.startswith("data: "):
                continue
            body = line[len("data: "):]
            if body == "[DONE]":
                break
            delta = json.loads(body)["choices"][0]["delta"]
            if not delta.get("content"):
                continue
            if first_token_at is None:
                first_token_at = time.perf_counter()
            tokens += 1
 
    finished = time.perf_counter()
    ttft = (first_token_at or finished) - started
    gen_seconds = finished - (first_token_at or started)
    return {
        "model": model,
        "ttft_sec": round(ttft, 2),
        "tokens": tokens,
        "tok_per_sec": round(tokens / gen_seconds, 1) if gen_seconds > 0 else 0.0,
    }
 
if __name__ == "__main__":
    for tag in ("gemma3:4b", "gemma3:12b", "gemma3:27b"):
        # Discard the first call — it includes model load time — and record the second.
        measure(tag, num_predict=32)
        print(measure(tag))

The deliberate double call is the important part. The first request carries the entire model-load cost, so recording it yields a TTFT in the tens of seconds that tells you nothing. The number you actually live with day to day is the one measured with OLLAMA_KEEP_ALIVE already holding the model in memory.

Token counts here approximate by counting stream chunks. That isn't an exact tokenizer count, but it's more than accurate enough for comparing models against each other.

Turning the Numbers Into a Decision

Here are the thresholds I apply once the measurements are in.

Building this table removed nearly all the hesitation from model selection. The biggest shift was that I stopped choosing the larger model on tok/s alone. On my M3 Max, 12B and 27B differ by roughly 2× in throughput — but by more than 3× in TTFT. For completion work, 12B feels distinctly faster, and I would never have caught that inversion without separating the two measurements.

One more thing: run the benchmark both plugged in and on battery. Laptop power management can cut tok/s nearly in half, which means a result you validated at your desk may not reproduce on the road.

Next Step

Once the pipe is stable, the interesting design question is: which agents should be local-only by contract? Good candidates are code review of proprietary repos, internal doc summarization, and personal note structuring — work where you can commit to "zero bytes leave this machine."

We cover Antigravity's multi-agent architecture in more depth in Advanced Multi-Agent Orchestration. A follow-up on local+cloud hybrid topologies is in the works.