Antigravity × Ollama: Gemma 4 in Your Editor

Pointing Antigravity at Ollama

In Antigravity's config (antigravity.config.json or the matching env vars), declare Ollama as an OpenAI-compatible provider:

{
  "llm_providers": [
    {
      "name": "local-gemma",
      "type": "openai_compatible",
      "base_url": "http://localhost:11434/v1",
      "api_key": "ollama",
      "default_model": "gemma3:12b"
    },
    {
      "name": "cloud-gemini",
      "type": "google_gemini",
      "api_key": "YOUR_GEMINI_API_KEY",
      "default_model": "gemini-2.5-pro"
    }
  ],
  "routing": {
    "agents": {
      "doc-summarizer": "local-gemma",
      "code-generator": "cloud-gemini"
    }
  }
}

The api_key is set to the literal string "ollama" on purpose. Ollama doesn't authenticate, but many OpenAI-compatible clients refuse to send a request when the key is empty. Any non-empty string works; "ollama" is a convention.

Routing Per-Agent

The snippet above sends a summarizer agent to Ollama and a code generator to Gemini. That split reflects what I reach for in practice:

• Keep local: summaries of sensitive data, first-pass filtering of large logs, cost-bound batch jobs, work done on spotty or offline networks
• Send to cloud: code generation where accuracy matters, translations that need multilingual nuance, answers that benefit from up-to-date information

"All local" and "all cloud" are both wrong defaults. Split by task.

The Day Nothing Could Reach 127.0.0.1:11434

curl http://localhost:11434/api/tags returned clean JSON from my terminal, and the agent running inside a container got connection refused against the same address. I spent most of an evening auditing the Antigravity side of that. The problem was on the listening end.

Ollama binds 127.0.0.1 on port 11434 by default (Ollama FAQ). It's closed to the loopback interface, so only the same machine can reach it. A Docker container, a WSL2 guest, another box on the LAN — the moment your topology includes any of those, the connection is refused. What base_url says in your config file has nothing to do with it at that point.

OLLAMA_HOST decides what gets opened, and there's a trap worth spelling out. Typing OLLAMA_HOST=0.0.0.0 ollama serve in a terminal does not reach an Ollama that is already resident as the macOS app or as a systemd service. Handing an environment variable to a resident process takes a per-platform ritual.

Verify from the machine that needs to call it. Verifying locally proves nothing here.

# run this from the caller: inside the container, or on the other box
curl -s --max-time 3 http://192.0.2.10:11434/api/tags | head -c 200

And then a line I keep. Opening a route is not the same as granting permission. Ollama has no authentication of its own — that's precisely why a dummy string in the api_key field works. Binding to 0.0.0.0 means anyone on that network can call your model. My rule is to leave the default untouched whenever the whole setup lives on one machine, and to put Nginx or a Cloudflare Tunnel in front on the rare occasions I need external access, so that authentication lives in that layer instead.

One more thing if you do put a proxy in front. The FAQ's reverse proxy example specifies proxy_set_header Host localhost:11434 for a reason: Ollama inspects the Host header it receives, and passing yours through untouched gets the request rejected. The ngrok flag --host-header="localhost:11434" and cloudflared's --http-host-header="localhost:11434" exist for the same reason.

If the caller is a browser extension or a web UI, that's a separate problem from routing — you'll need OLLAMA_ORIGINS. Only 127.0.0.1 and 0.0.0.0 are allowed as cross-origin sources by default, so patterns like chrome-extension://* have to be added explicitly. Opening the network path alone won't fix that one.

Sizing the Model From the Memory You Actually Have

What size fits your hardware determines whether the local route is tolerable or painful. Here's my working sense:

One note on tags before you go further: later sections use names like gemma4:e4b and gemma4:26b. An earlier version of this page said gemma4:9b; I could not find that tag. Registry tags shift between generations. Run ollama list and ollama show <tag> to confirm the exact tag and parameter count on your machine before wiring anything up. I keep the tag out of config files entirely and expose it through an environment variable such as ANTIGRAVITY_LOCAL_MODEL, so a generation change means editing exactly one place.

On an M2 Max 64GB, gemma3:27b lands around 15–20 tokens/s and gemma3:12b hits about 40. On an RTX 4090 Linux box, 12B goes 60–80 tokens/s.

Quantization matters too. q4_0 is the default; q5_K_M improves quality at some memory cost. My working baseline is 12b q5_K_M, sizing up or down based on available VRAM.

Fallback Routing — Slow Failure Hurts More Than Clean Failure

The main reason to keep a local LLM in the loop is insurance for when the cloud side becomes unavailable. But after running this setup for a few months as a solo indie developer, the failure that actually cost me time was not the one I had planned for. It was the opposite direction: local Ollama stopped answering, and my router kept sending traffic to it anyway.

Model loading. Memory pressure pushing the box into swap. Hitting the OLLAMA_MAX_LOADED_MODELS ceiling so another model gets evicted mid-request. In all three cases Ollama still accepts the connection — it just never replies. An error you can branch on. Silence you cannot.

On the Antigravity side you declare the fallback target roughly like this:

{
  "routing": {
    "default": "cloud-gemini",
    "fallback": "local-gemma",
    "fallback_triggers": ["network_error", "rate_limit", "api_error"]
  }
}

As a declaration that is enough. What it actually does under load, though, is not visible from the file itself.

So I measured a naive fallback against one that remembers failures. The target is not Ollama itself but a local stub that returns OpenAI-compatible responses, because what I wanted to see is not inference speed — it is how many times, and how many milliseconds, the router keeps paying to the broken side. Local was pinned to two failure modes ("returns 503 instantly" and "accepts the connection and goes quiet"), cloud to one healthy mode answering in 40ms.

The naive version

import json, time, urllib.request
 
def call(url, timeout):
    req = urllib.request.Request(
        url, data=json.dumps({"messages": []}).encode(),
        headers={"content-type": "application/json"})
    with urllib.request.urlopen(req, timeout=timeout) as r:
        return json.load(r)
 
class Naive:
    """Try local every time; fall through to cloud on any failure."""
    def __init__(self, local, cloud, timeout):
        self.l, self.c, self.t = local, cloud, timeout
 
    def send(self):
        try:
            return call(self.l, self.t), "local"
        except Exception:
            return call(self.c, self.t), "cloud"

This is roughly what the routing JSON above does under the hood. It is functionally correct — requests still come back when local is dead.

The version that remembers

class Breaker:
    """Open after fail_max consecutive failures; skip local entirely for cooldown seconds."""
    def __init__(self, local, cloud, timeout, fail_max=2, cooldown=2.0):
        self.l, self.c, self.t = local, cloud, timeout
        self.fail_max, self.cooldown = fail_max, cooldown
        self.fails, self.open_until = 0, 0.0
        self.skipped = 0
 
    def send(self):
        if time.monotonic() < self.open_until:
            self.skipped += 1                    # straight to cloud, no local attempt
            return call(self.c, self.t), "cloud"
        try:
            r = call(self.l, self.t)
            self.fails = 0                       # one success clears the memory
            return r, "local"
        except Exception:
            self.fails += 1
            if self.fails >= self.fail_max:
                self.open_until = time.monotonic() + self.cooldown
            return call(self.c, self.t), "cloud"

The only difference is that it remembers recent failures and stops probing for a while.

Fast failure and slow failure split the results

Twelve requests per condition, local timeout fixed at 0.5s.

When the error comes back immediately, the two are indistinguishable — 0.6ms apart. Testing only that condition and concluding "we're fine" is exactly what I had been doing.

The gap opens when nothing comes back. The naive router pays the full 0.5s timeout on every single request, 4.3x total. And it buys nothing with that time: it is re-purchasing the same fact — local is down — twelve times over.

If an agent turn makes five LLM calls, that 0.5s becomes 2.5s of dead time per turn. Some of what I had been experiencing as "Antigravity feels sluggish today" was living right here.

A shorter cooldown costs more while local is down

So what should cooldown be? I ran requests continuously for an 8-second window and counted how many completed, with local staying silent throughout.

A 0.5s cooldown completed 41% fewer requests than a 5.0s one. Probing frequently to catch recovery early means each probe sits through a timeout.

What you buy with that is recovery speed. At cooldown 2.0s, I measured the delay between local actually recovering and the router's first successful local call four times: 0.32 / 0.95 / 1.63 / 2.26 seconds (median 1.29s). Probe timing and recovery timing are independent, so the delay spreads evenly across the interval. Set cooldown to n seconds and recovery detection lags by n/2 on average, n at worst. That's the whole trade.

Looking at those numbers, I settled on 2 seconds. Loading a local model takes several seconds at best, so probing at 0.5s intervals mostly arrives too early to matter. Stretching to 5 seconds, on the other hand, made the pause after restarting ollama serve noticeable enough to bother me. Two seconds is where those two pressures met.

One more thing: I set the local timeout distinctly shorter than the cloud one. Local exists because it is fast. If it makes me wait longer than the cloud, there is no reason to route there at all. My box gives up on local at 0.5s and gives the cloud its normal budget.

Coming back to that JSON config above: fallback_triggers lists network_error, rate_limit, and api_error. All three are failures where the other side hands you something. The failure where nothing comes back is not on the list at all. What you can express in a config file and what actually happens in production are different sets — measuring is what made that land for me.

Operational Gotchas Nobody Warns You About

Context length: Ollama's default context window is 4096 tokens (Ollama FAQ). Antigravity agents with long histories get truncated silently. Set OLLAMA_CONTEXT_LENGTH, or add num_ctx 16384 to the Modelfile. Memory grows with context length — keep an eye on it.

Cold starts: Ollama loads a model into memory on the first request, so the initial response can be 10–30 seconds late. OLLAMA_KEEP_ALIVE=-1 pins the model in memory for the session, which feels dramatically snappier in interactive use.

Parallel agent memory pressure: When Antigravity runs many agents concurrently, Ollama may try to load several models simultaneously and blow past your RAM budget. Cap concurrent loads with OLLAMA_MAX_LOADED_MODELS, and design so that critical agents share a single model rather than each having its own.

When sharing one model still blows up — num_ctx multiplies by parallelism

I took the advice above, pointed every agent at the same model, and still fell over the moment the third agent started. My assumption was that one loaded model meant one fixed memory cost. It doesn't.

Ollama serves concurrent requests to a single model up to OLLAMA_NUM_PARALLEL, and the RAM it needs scales with OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH. In the documentation's own example, a 2K context served at 4-way parallelism costs the same as holding one 8K context (Ollama FAQ).

Later in this article I suggest setting num_ctx to about 1.5x your real history length. That estimate assumes a parallelism of 1. Raise the parallelism and the same multiplication applies.

So the order matters: lower num_ctx first, then add agents. On days when I want more agents running, I halve num_ctx and check that conversations still aren't being truncated before I touch the parallelism at all. If nothing gets cut, the extra slots are free to take.

There's a second lever. OLLAMA_KV_CACHE_TYPE=q8_0 quantizes the K/V cache to 8-bit, which costs roughly half the memory of the default f16 with a precision loss that is usually not noticeable. It assumes Flash Attention is active, and it's a global setting rather than a per-model one.

Weight quantization buys you model quality; K/V cache quantization buys you history length. While I had those two filed under the same heading, every time I wanted a longer context I dropped down a model size to pay for it. Separating them is what let me keep 12B and stretch the history instead.

ollama ps is the fastest way to see where you actually stand. If the PROCESSOR column has shifted from 100% GPU to something like 48%/52% CPU/GPU, the model no longer fits in VRAM and part of it has spilled into system memory. That boundary is usually where tok/s falls off, and checking that one line before you start editing settings saves a lot of detours.

Gemma 4 Itself — Improvements Over Earlier Generations

Before getting deeper into the integration, it helps to understand what's actually different about Gemma 4. Knowing context length, strengths, and weaknesses up front makes model sizing and task routing easier later on.

Here is what actually changed from Gemma 3, in the directions that affect how you'd wire it up:

• Architecture: More work per parameter, so smaller variants cover jobs that used to need a size up — which widens what your memory budget can reach
• Multimodal: It accepts images alongside text
• Context length: Longer than the previous generation, so more conversation histories fit without pruning
• Localization: Broader training data made non-English output easier to work with than it was

You'll notice I'm not quoting percentages here. I've picked a model off headline benchmark deltas more than once and had to redo the choice a week later, because the number that moved wasn't the number my workload cared about. Published benchmarks narrow the shortlist; your own measurements make the call. There's a short script later in this article for measuring TTFT and tok/s on your own box. When it's time to actually pick a size, use those numbers instead of mine.

Steering Local/Cloud Switching with AGENTS.md

Antigravity's agent configuration (AGENTS.md) lets you set per-agent rules for which model to use. Sensitive data processing can be forced to local, offline mode can kick in when the network can't be reached — operational requirements can drive routing.

Dropping an AGENTS.md file at the workspace root lets Antigravity route between models per task instead of picking one globally.

# Agent Configuration
 
### Default Model
Use `gemini-3-pro` for planning and complex reasoning.
 
### Privacy-Sensitive Tasks
Use `gemma4-local` for:
- Files under `private/`
- Anything containing customer data
- First-pass drafts (cloud model reviews before ship)
 
### Offline Mode
If network is unavailable, fall back to `gemma4-local` for all tasks.

The hybrid pattern — sensitive work local, heavy reasoning cloud — is what I've found actually sustainable. Fully-local is rarely practical; fully-cloud misses the privacy and cost wins.

Inference Performance Tuning

Local inference speed varies dramatically with CPU/GPU resource management. Here are the adjustment points that matter for getting practical response times.

Out of the box, larger models can feel slower than you expect. Three things are worth touching:

Quantization level. The official library's default tag is usually a 4-bit quantization. On Apple Silicon M3+ or a machine with spare memory, a higher-precision quantization may be worth trying. Tag names differ per model, so confirm one exists before you pull it instead of guessing:

ollama show gemma4:e4b   # check parameter count and quantization of the default tag
# Pull other quantizations only if they appear in the Tags list on the model page
ollama pull gemma4:e4b

Memory use goes up for certain; the speed change depends on the machine. Run the measurement script near the end of this page before and after.

Shrink the context window to what you actually need. The model ceiling and the num_ctx Ollama actually allocates are different numbers. The latter defaults to 4096, and pushing it up to the model ceiling grows the KV cache proportionally, which costs memory and inference time directly. I measure the real history length of the tasks I run, then set num_ctx to roughly 1.5x that.

Keep the model warm. OLLAMA_KEEP_ALIVE=30m prevents Ollama from unloading the model after five idle minutes. Cold loads cost a few seconds each time; this eliminates that tax for frequent callers.

Practical Development Workflows — Where Antigravity × Local Gemma 4 Pays Off

The typical workflows for Antigravity development with local LLMs in the loop, and how Gemma 4 actually behaves in each scenario.

Pattern 1: Local by Default, Cloud for Complex Decisions

Handle the high-frequency, lower-complexity parts of development locally (completions, fixes, test generation), and reach for cloud models only when the task genuinely demands it.

# Tasks well-suited to Gemma 4 locally
 
# 1. Code completion
def parse_config(file_path: str) -> dict:
    """Load a configuration file and return a dict."""
    # ← Antigravity (Gemma 4) fills this in on-device
 
# 2. Docstring generation
# ← Point at the function, ask for a docstring → local inference
 
# 3. Bug fixes from error messages
# ← Paste the traceback, get a suggested fix → local inference

# Signals to switch to a cloud model
- Discussing project-wide architecture across many files
- Planning a refactor spanning 10+ files
- Security threat modeling (higher stakes, worth the cloud cost)
- API design review before a public release

Pattern 2: Fully Air-Gapped Workflow for Confidential Projects

For projects where code absolutely cannot leave the machine, set Antigravity to local-only mode via environment variables:

# .env for a confidential project
ANTIGRAVITY_MODEL=local
ANTIGRAVITY_ALLOW_CLOUD=false
ANTIGRAVITY_LOCAL_ENDPOINT=http://localhost:11434/v1
ANTIGRAVITY_LOCAL_MODEL=gemma4:26b
 
# With ALLOW_CLOUD=false, Antigravity won't attempt any external requests
# regardless of what operation is performed

Pattern 3: Offline Development (Travel, Flights, Remote Sites)

Download models before going offline. Antigravity detects connectivity loss and falls back to the local model automatically.

# Before going offline
ollama pull gemma4:e2b   # Essential for basic development
ollama pull gemma4:e4b   # If disk space allows
 
# During offline work: Antigravity routes automatically to the local model

Gemma 4 Coding Quality — Check by Task, Not by Percentage

An earlier version of this page carried per-task average seconds and a "quality %" against a cloud model. I couldn't attach a reproducible procedure to those numbers, so I removed them. I had no grounds for stating quality as a percentage.

What I use instead is a handful of checks I ran on my own repositories for about a week:

Speed changes with the machine, quality changes with the task. Measure both once on your own repository and you'll have better evidence than anyone's table. On cost, simply moving everyday drafting to local frees your cloud quota for the decisions that need it.

Where Gemma 4 Wins, Where Gemma 4 Loses — Task Routing Heuristics

In a hybrid local/cloud setup, the key decision is "what stays local." Here are the areas where Gemma 4 reliably holds up, and the areas where you should probably route to the cloud.

Gemma 4 is a capable open-weight model, but it has real limitations worth knowing.

Where it excels: Code generation in mainstream languages such as Python, TypeScript, Go and Rust, short-context understanding and modification, docstring and comment generation, and reading Japanese technical documentation. These are the tasks I hand over without a second thought.

Where cloud models still win: Very long contexts (understanding 10,000+ lines of code as a whole), knowledge of frameworks released after the training cutoff, and nuanced architectural judgment calls where the reasoning chain is long and ambiguous.

This makes a local-first, cloud-fallback hybrid the most practical approach for most development environments — local Gemma 4 for routine work, cloud for the tasks that genuinely benefit from it.

Pitfalls I Hit in Real Use

Local LLM integration has more "designed-fine-but-broke-in-practice" cases than most setups. Here are the operational issues that come up most often, and what fixed them.

Three lessons the tutorials miss:

Long-form Japanese suffers at small sizes. With a small model like E4B, honorific consistency and technical-term translation slipped compared with a large cloud model. For polished long-form Japanese, 26B or above, or a cloud model, is the honest answer. Short structured tasks are fine.

Tool calling can be flaky on small models. Local LLMs across the board struggle with complex tool-call JSON. If you define more than a couple of tools, expect occasional malformed arguments. Reserve local models for tasks with one or two well-specified tools; leave the many-tool orchestration to cloud models.

Laptop thermals are a real constraint. Running E4B or 26B continuously on battery gets hot and dies in 2–3 hours. If this is a daily driver, plug in or run Ollama on a separate always-on box.

Stop Trusting My Numbers — Measure TTFT and tok/s on Your Own Box

Every speed figure above comes from my hardware. As an indie developer I have only a couple of machines to test on, and moving the same setup between them changed the feel enough to surprise me. Two machines with the same chip will diverge based on memory bandwidth and thermal headroom, and raising num_ctx slows everything down regardless. After redoing this exercise a few times, my conclusion is simple: reading someone else's benchmark is slower than running your own.

There are two numbers worth capturing. TTFT (time to first token) and tok/s (generation throughput). For inline completion in the editor, perceived speed is almost entirely TTFT; tok/s only starts to matter once you're generating long output. Collapsing both into a single "fast or slow" judgment is how model selection goes wrong.

Ollama's OpenAI-compatible endpoint supports streaming, so the standard library is enough to capture both.

# bench_ollama.py — measure TTFT and tok/s per model
import json
import time
import urllib.request
 
ENDPOINT = "http://localhost:11434/v1/chat/completions"
PROMPT = "Add a docstring to this function:\ndef parse_config(path):\n    ..."
 
def measure(model: str, num_predict: int = 256) -> dict:
    payload = {
        "model": model,
        "messages": [{"role": "user", "content": PROMPT}],
        "stream": True,
        "max_tokens": num_predict,
    }
    req = urllib.request.Request(
        ENDPOINT,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json", "Authorization": "Bearer ollama"},
    )
 
    started = time.perf_counter()
    first_token_at = None
    tokens = 0
 
    with urllib.request.urlopen(req) as res:
        for raw in res:
            line = raw.decode("utf-8").strip()
            if not line.startswith("data: "):
                continue
            body = line[len("data: "):]
            if body == "[DONE]":
                break
            delta = json.loads(body)["choices"][0]["delta"]
            if not delta.get("content"):
                continue
            if first_token_at is None:
                first_token_at = time.perf_counter()
            tokens += 1
 
    finished = time.perf_counter()
    ttft = (first_token_at or finished) - started
    gen_seconds = finished - (first_token_at or started)
    return {
        "model": model,
        "ttft_sec": round(ttft, 2),
        "tokens": tokens,
        "tok_per_sec": round(tokens / gen_seconds, 1) if gen_seconds > 0 else 0.0,
    }
 
if __name__ == "__main__":
    for tag in ("gemma3:4b", "gemma3:12b", "gemma3:27b"):
        # Discard the first call — it includes model load time — and record the second.
        measure(tag, num_predict=32)
        print(measure(tag))

The deliberate double call is the important part. The first request carries the entire model-load cost, so recording it yields a TTFT in the tens of seconds that tells you nothing. The number you actually live with day to day is the one measured with OLLAMA_KEEP_ALIVE already holding the model in memory.

Token counts here approximate by counting stream chunks. That isn't an exact tokenizer count, but it's more than accurate enough for comparing models against each other.

Turning the Numbers Into a Decision

Here are the thresholds I apply once the measurements are in.

Building this table removed nearly all the hesitation from model selection. The biggest shift was that I stopped choosing the larger model on tok/s alone. On my M3 Max, 12B and 27B differ by roughly 2× in throughput — but by more than 3× in TTFT. For completion work, 12B feels distinctly faster, and I would never have caught that inversion without separating the two measurements.

One more thing: run the benchmark both plugged in and on battery. Laptop power management can cut tok/s nearly in half, which means a result you validated at your desk may not reproduce on the road.

Next Step

Once the pipe is stable, the interesting design question is: which agents should be local-only by contract? Good candidates are code review of proprietary repos, internal doc summarization, and personal note structuring — work where you can commit to "zero bytes leave this machine."

We cover Antigravity's multi-agent architecture in more depth in Advanced Multi-Agent Orchestration. A follow-up on local+cloud hybrid topologies is in the works.