Articles/Agents & Manager

◈ Agents & Manager/2026-04-09Advanced

Antigravity × Gemma 4: Building Production AI Agents with Local LLMs

A complete guide to running Gemma 4 in Antigravity and building production-grade AI agents. Covers model selection, Ollama setup, AgentKit 2.0 integration, and multi-agent scaling.

Antigravity³⁵⁶ Gemma 4²² local LLM¹⁴ AI agents²⁴ AgentKit¹⁷ multi-agent⁵⁰

✦ Premium Article

The first request of the morning took almost thirty seconds. No error, no timeout, nothing in the logs — just silence, then a normal answer. Ollama had quietly unloaded the model after a long idle period, and the load time was riding on top of that one request. The inference was never the fragile part. The operations around it were.

Gemma 4 has made local models a realistic choice rather than a weekend experiment. Coding and reasoning quality at the 9B tier is good enough for real work, and from Antigravity's side that means you now have an inference engine you can hammer without watching a billing meter.

What follows covers variant selection, Ollama setup, tool use and memory with AgentKit 2.0, multi-agent orchestration, and — the part most guides skip — what breaks once the thing is running all day. Getting a single call to work is well documented. Keeping it working is where the interesting problems live.

The Gemma 4 Landscape: Variants and Antigravity Compatibility

Gemma 4, from Google DeepMind, distinguishes itself through a strategic variant approach. There are three main flavors:

Gemma 4 2B

Parameters: 2 billion
Memory: 4GB (quantized) to 6GB (full precision)
Speed: Extremely fast per token
Best for: Lightweight edge devices, real-time responses, prototyping

Gemma 4 9B

Parameters: 9 billion
Memory: 8GB (quantized) to 18GB (full precision)
Speed: Fast and balanced
Best for: General-purpose agents, most production environments

Gemma 4 27B

Parameters: 27 billion
Memory: 16GB (quantized) to 54GB (full precision)
Speed: Slower, but highest accuracy
Best for: Complex reasoning tasks, specialized domain agents

Antigravity Synergy

Antigravity was built with local LLMs in mind:

Zero API costs: No cloud dependency, no per-token billing
Fine-grained inference control: Adjust temperature, context length, beam search parameters directly
Integrated agent framework: AgentKit 2.0 bakes in Tool Use and Memory management
Multi-model management: Run multiple Gemma 4 variants simultaneously through Ollama

Antigravity's orchestration layer makes switching between models, scaling agents, and managing state trivial.

Setting Up Gemma 4 with Ollama

Before we code, let's get Gemma 4 installed locally.

Verify Ollama Installation

ollama --version

If not installed, grab it from ollama.com.

Pull Gemma 4 Models

# Gemma 4 9B (recommended: good balance)
ollama pull gemma4:9b
 
# Or the lightweight 2B
ollama pull gemma4:2b
 
# Or the powerful 27B
ollama pull gemma4:27b

First pull takes a few minutes to hours depending on model size and internet speed. Gemma 4 9B is about 9GB.

Quantization Variants

By default, you get a quantized version (typically q4_0 or q5_K_M). If you have spare RAM, request higher precision:

# Finer quantization (≈18GB, 8GB RAM recommended)
ollama pull gemma4:9b-q5_K_M
 
# Full precision (≈34GB, 24GB RAM required — not recommended)
ollama pull gemma4:9b-fp16

Trade-off table:

Quantization	File Size	RAM	Accuracy	Use Case
q2_K	~5GB	6GB	Low	Ultra-light edge
q3_K	~6GB	8GB	Low–Med	Edge AI
q4_0	~9GB	12GB	Medium	Standard prod
q5_K_M	~11GB	14GB	Med–High	High-accuracy
fp16	~18GB	28GB	Highest	Research

Start the Ollama Server

ollama serve

In another terminal:

ollama list

Expected output:

NAME              ID              SIZE      MODIFIED
gemma4:9b         a1b2c3d4e5f6    9.5 GB    2 minutes ago

Verify the API

curl -X POST http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4:9b",
    "prompt": "What is AI?",
    "stream": false
  }'

You'll get back a JSON response with the generated text. Good — you're ready for Antigravity.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦Choose the right Gemma 4 variant (2B/9B/27B) and configure it optimally in Antigravity

✦Build a production agent using Ollama + Antigravity + AgentKit 2.0 integration

✦Scale with multi-agent parallelism and cost optimization strategies for Gemma 4

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Connecting Antigravity to Gemma 4

Now let's wire up your Antigravity project.

Environment Variables

Create .env.local in your project root:

NEXT_PUBLIC_LLM_PROVIDER=ollama
NEXT_PUBLIC_OLLAMA_BASE_URL=http://localhost:11434
NEXT_PUBLIC_OLLAMA_MODEL=gemma4:9b
 
# Optional: inference parameters
NEXT_PUBLIC_OLLAMA_TEMPERATURE=0.7
NEXT_PUBLIC_OLLAMA_TOP_P=0.9
NEXT_PUBLIC_OLLAMA_TOP_K=40
NEXT_PUBLIC_OLLAMA_NUM_PREDICT=2048

Parameter meanings:

TEMPERATURE: Output diversity (0 = deterministic, 1 = creative)
TOP_P: Cumulative probability for nucleus sampling
TOP_K: Sample only from top-K tokens
NUM_PREDICT: Max generation length

Configuration File

For finer control, create antigravity.config.ts:

export const llmConfig = {
  provider: "ollama",
  baseUrl: process.env.NEXT_PUBLIC_OLLAMA_BASE_URL || "http://localhost:11434",
  model: process.env.NEXT_PUBLIC_OLLAMA_MODEL || "gemma4:9b",
  temperature: parseFloat(process.env.NEXT_PUBLIC_OLLAMA_TEMPERATURE || "0.7"),
  topP: parseFloat(process.env.NEXT_PUBLIC_OLLAMA_TOP_P || "0.9"),
  topK: parseInt(process.env.NEXT_PUBLIC_OLLAMA_TOP_K || "40", 10),
  numPredict: parseInt(process.env.NEXT_PUBLIC_OLLAMA_NUM_PREDICT || "2048", 10),
};
 
export const agentConfig = {
  maxIterations: 10,
  verbose: true,
  timeout: 30000,
};

Restart and Test

npm run dev

Check browser console (F12) for connection logs:

[Antigravity] Connected to Ollama at http://localhost:11434
[Antigravity] Model loaded: gemma4:9b

Building Your First Agent: Simple Query Responder

Let's start with a basic agent that answers questions.

Agent Skeleton

// src/agents/BasicGemma4Agent.ts
import { OllamaClient } from "@ollama/sdk";
 
const client = new OllamaClient({
  baseUrl: "http://localhost:11434",
});
 
interface AgentState {
  input: string;
  history: Array<{ role: string; content: string }>;
}
 
export async function basicAgent(input: string): Promise<string> {
  const state: AgentState = {
    input,
    history: [],
  };
 
  state.history.push({
    role: "user",
    content: input,
  });
 
  const response = await client.generate({
    model: "gemma4:9b",
    prompt: buildPrompt(state.history),
    temperature: 0.7,
  });
 
  state.history.push({
    role: "assistant",
    content: response.response,
  });
 
  return response.response;
}
 
function buildPrompt(history: Array<{ role: string; content: string }>): string {
  let prompt = "You are a helpful AI assistant.\n\n";
  for (const msg of history) {
    if (msg.role === "user") {
      prompt += `User: ${msg.content}\n`;
    } else {
      prompt += `Assistant: ${msg.content}\n`;
    }
  }
  prompt += "Assistant:";
  return prompt;
}

Usage

const result = await basicAgent("Explain quantum computing in 3 sentences.");
console.log(result);

Output:

Quantum computing harnesses quantum mechanics principles like superposition and entanglement to process information fundamentally differently than classical computers. Unlike classical bits (0 or 1), quantum bits (qubits) can exist in multiple states simultaneously, allowing quantum computers to explore many solutions in parallel. This makes them potentially exponentially faster for certain problems like cryptography and molecular simulation.

AgentKit 2.0: Tool Integration and Memory

Production agents need more than raw language modeling. They need tools (web search, database queries, calculations) and persistent memory. That's AgentKit 2.0.

Defining Tools

// src/agents/tools.ts
import { Tool } from "@antigravity/agentkit";
 
export const searchTool: Tool = {
  name: "web_search",
  description: "Search the web for information",
  inputSchema: {
    type: "object",
    properties: {
      query: {
        type: "string",
        description: "Search query",
      },
    },
    required: ["query"],
  },
  execute: async (params: { query: string }) => {
    const response = await fetch(`https://api.example.com/search`, {
      method: "POST",
      headers: { "Authorization": `Bearer ${process.env.SEARCH_API_KEY}` },
      body: JSON.stringify({ q: params.query }),
    });
    return await response.json();
  },
};
 
export const databaseQueryTool: Tool = {
  name: "query_database",
  description: "Query the application database",
  inputSchema: {
    type: "object",
    properties: {
      sql: {
        type: "string",
        description: "SELECT-only SQL query",
      },
    },
    required: ["sql"],
  },
  execute: async (params: { sql: string }) => {
    const db = getDBConnection();
    return await db.query(params.sql);
  },
};
 
export const calculateTool: Tool = {
  name: "calculate",
  description: "Perform math",
  inputSchema: {
    type: "object",
    properties: {
      expression: {
        type: "string",
        description: "Math expression like '2 + 2'",
      },
    },
    required: ["expression"],
  },
  execute: async (params: { expression: string }) => {
    return { result: safeEval(params.expression) };
  },
};

The Agent Loop

// src/agents/GemmaAgentWithTools.ts
import { Agent, AgentState, Tool } from "@antigravity/agentkit";
import { searchTool, databaseQueryTool, calculateTool } from "./tools";
 
export class GemmaProductionAgent {
  private agent: Agent;
 
  constructor() {
    this.agent = new Agent({
      model: "gemma4:9b",
      tools: [searchTool, databaseQueryTool, calculateTool],
      maxIterations: 10,
      verbose: true,
    });
  }
 
  async run(input: string): Promise<string> {
    const state: AgentState = {
      input,
      messages: [],
      toolCalls: [],
      scratchpad: "",
    };
 
    for (let i = 0; i < this.agent.maxIterations; i++) {
      const response = await this.agent.step(state);
 
      if (response.action === "finish") {
        return response.thought;
      }
 
      if (response.action === "tool") {
        const tool = this.agent.tools.find(
          (t) => t.name === response.toolName
        );
        if (!tool) {
          state.scratchpad += `Tool ${response.toolName} not found.\n`;
          continue;
        }
 
        const toolResult = await tool.execute(response.toolInput);
        state.messages.push({
          role: "tool",
          toolName: response.toolName,
          result: toolResult,
        });
 
        state.scratchpad += `Tool: ${response.toolName}\nResult: ${JSON.stringify(toolResult)}\n`;
      }
    }
 
    throw new Error("Max iterations reached");
  }
}

Using the Agent

const agent = new GemmaProductionAgent();
const result = await agent.run(
  "What's the current price of gold and how much is 5 kg worth?"
);
console.log(result);

The agent automatically:

Recognizes it needs current data
Calls web_search for gold prices
Uses calculate to compute 5kg worth
Returns a complete answer

Multi-Agent Systems: Parallel Execution at Scale

One agent can't handle multiple simultaneous tasks efficiently. Time for multi-agent orchestration.

Manager-Subordinate Architecture

// src/agents/MultiAgentOrchestrator.ts
import { GemmaProductionAgent } from "./GemmaProductionAgent";
 
interface Task {
  id: string;
  description: string;
  priority: "high" | "medium" | "low";
}
 
export class MultiAgentOrchestrator {
  private agents: Map<string, GemmaProductionAgent> = new Map();
  private taskQueue: Task[] = [];
 
  constructor(numAgents: number = 3) {
    for (let i = 0; i < numAgents; i++) {
      this.agents.set(`agent-${i}`, new GemmaProductionAgent());
    }
  }
 
  async delegateTask(task: Task): Promise<string> {
    this.taskQueue.push(task);
    const agent = Array.from(this.agents.values())[0];
    return await agent.run(task.description);
  }
 
  async processBatch(tasks: Task[]): Promise<Map<string, string>> {
    const results = new Map<string, string>();
 
    const promises = tasks.map(async (task) => {
      const agentArray = Array.from(this.agents.values());
      const agent = agentArray[Math.floor(Math.random() * agentArray.length)];
      const result = await agent.run(task.description);
      results.set(task.id, result);
    });
 
    await Promise.all(promises);
    return results;
  }
}

Batch Processing Example

const orchestrator = new MultiAgentOrchestrator(3); // 3 parallel agents
 
const tasks: Task[] = [
  {
    id: "task-1",
    description: "Analyze Q1 customer feedback trends",
    priority: "high",
  },
  {
    id: "task-2",
    description: "Generate README for the REST API",
    priority: "medium",
  },
  {
    id: "task-3",
    description: "Summarize recent machine learning papers",
    priority: "low",
  },
];
 
const results = await orchestrator.processBatch(tasks);
 
results.forEach((result, taskId) => {
  console.log(`${taskId}: ${result}`);
});

Performance Notes

Memory: Each agent loads a full Gemma 4 model. 3 agents of 9B = ~36GB RAM
CPU: Parallel inference is CPU-intensive. Configure thread affinity if needed
Latency: Total time is bottlenecked by the slowest task

Production Optimization: Quantization, Context, and Caching

Real-world deployment requires careful tuning.

Quantization Strategy by Use Case

Use Case	Quantization	Temperature	Relative throughput
Real-time chat	q4_0	0.5	1.0 (baseline)
Analysis tasks	q5_K_M	0.7	~0.7x
Code generation	q4_K_M	0.6	~0.9x
Document summarization	q4_0	0.3	1.0 (baseline)

Absolute token rates vary enormously by hardware, so treat these as ratios rather than numbers to quote. Measure your own baseline once and you'll never need anyone else's benchmark:

ollama run gemma4:9b --verbose "Summarize the following in three bullet points: ..."

The eval rate line is your generation speed. What actually drives design decisions is the ratio — how much slower 27B is than 9B on your machine. Once you know that, you can decide on paper which tasks earn the bigger model.

Context Window Optimization

export const llmConfig = {
  model: "gemma4:9b",
  contextLength: 4096, // Limit to 4K (default 8K)
  slidingWindow: true,
  windowSize: 2048, // Keep only latest 2K tokens
};

Response Caching

import NodeCache from "node-cache";
import { createHash } from "crypto";
 
const cache = new NodeCache({ stdTTL: 3600 });
 
async function cachedGenerate(prompt: string): Promise<string> {
  const cacheKey = createHash("sha256").update(prompt).digest("hex");
 
  const cached = cache.get(cacheKey);
  if (cached) return cached as string;
 
  const response = await ollama.generate({ model: "gemma4:9b", prompt });
  cache.set(cacheKey, response.response);
 
  return response.response;
}

What Breaks Once It's Always On

Everything above works fine when you run it by hand. Things start failing once an agent sits there all day and people poke it at unpredictable intervals. Three problems show up every single time.

1. The first request after idle is absurdly slow

Ollama unloads models after a period of inactivity. The next request pays the full load cost — several seconds at 9B, long enough to look like a hang at 27B. Worse, it never registers as an error. Your monitoring just says "sometimes slow."

Set keep_alive explicitly, per request:

await fetch(`${OLLAMA_BASE_URL}/api/generate`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "gemma4:9b",
    prompt,
    stream: false,
    // Stay resident for 30 minutes (-1 = forever, 0 = unload immediately)
    keep_alive: "30m",
  }),
});

Residency costs memory, though. If the machine doubles as your workstation, avoid -1 and pick a window that matches your actual working hours. Then warm the model at startup so the first real user isn't the one who waits:

export async function warmUpModel(model: string): Promise<void> {
  try {
    await fetch(`${OLLAMA_BASE_URL}/api/generate`, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ model, prompt: "ok", stream: false, keep_alive: "30m" }),
      signal: AbortSignal.timeout(120_000), // Cold loads are slow — give this one room
    });
  } catch {
    // Never let a failed warm-up kill the process; the next real request loads the model
  }
}

2. Your parallel requests aren't actually parallel

A Promise.all over five agents looks concurrent in your code. How many actually run at once is decided by Ollama's slot count; the rest queue internally. Meanwhile your client-side timeout fires on requests that were only ever waiting, and you get failures that scale with concurrency for no visible reason.

Set the server ceiling:

# Requests processed concurrently
OLLAMA_NUM_PARALLEL=4
# Models resident at once (raise only if you genuinely swap between models)
OLLAMA_MAX_LOADED_MODELS=1

Then stop the client from exceeding it. A plain semaphore is enough:

class Semaphore {
  private queue: Array<() => void> = [];
  private active = 0;
 
  constructor(private readonly limit: number) {}
 
  async run<T>(task: () => Promise<T>): Promise<T> {
    if (this.active >= this.limit) {
      await new Promise<void>((resolve) => this.queue.push(resolve));
    }
    this.active++;
    try {
      return await task();
    } finally {
      this.active--;
      this.queue.shift()?.();
    }
  }
}
 
// Match OLLAMA_NUM_PARALLEL
const llmGate = new Semaphore(4);
 
export function generate(prompt: string): Promise<string> {
  return llmGate.run(() => callOllama(prompt));
}

Throttling feels like it should make the total slower. Usually it makes it faster. Queued work only delays the tail; timed-out work gets retried, which means running the same expensive inference twice.

3. Closed tabs keep the GPU busy

When a user abandons a streaming response, generation doesn't stop. The GPU runs to completion and every queued request behind it waits. The longer the prompt, the more you lose.

Tie an AbortController to both the request and the connection:

export async function streamGenerate(
  prompt: string,
  onToken: (token: string) => void,
  externalSignal: AbortSignal, // Pass your request's disconnect signal
): Promise<void> {
  const controller = new AbortController();
  const abort = () => controller.abort();
  externalSignal.addEventListener("abort", abort, { once: true });
 
  try {
    const res = await fetch(`${OLLAMA_BASE_URL}/api/generate`, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ model: "gemma4:9b", prompt, stream: true, keep_alive: "30m" }),
      signal: controller.signal,
    });
    if (!res.body) throw new Error("no response body");
 
    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let buffer = "";
 
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      buffer += decoder.decode(value, { stream: true });
 
      // Ollama streams NDJSON — only parse lines that are complete
      const lines = buffer.split("\n");
      buffer = lines.pop() ?? "";
      for (const line of lines) {
        if (!line.trim()) continue;
        const chunk = JSON.parse(line) as { response?: string; done?: boolean };
        if (chunk.response) onToken(chunk.response);
      }
    }
  } finally {
    externalSignal.removeEventListener("abort", abort);
    controller.abort(); // Also close the socket on the happy path
  }
}

That abort() in the finally block matters on successful completion too, not just on errors. Skip it and connections accumulate slowly over days of uptime.

Before you add retries

When failures climb, the instinct is to add retry logic. With local models that's backwards. The cause is almost always too much concurrency or a context window that's too large for available memory — and a retry applies the exact same load a second time.

Lower the semaphore limit first. If it still fails, trim the context length. Add retries after that, and only on idempotent reads.

Real-World Use Cases

Code Review Agent

const reviewResult = await agent.run(`
Review this PR code for security, performance, and style:
${pullRequestCode}
`);

Documentation Generator

const docs = await agent.run(`
Generate API docs for:
${apiImplementation}
Include: descriptions, parameters, examples, error cases
`);

Test Generator

const tests = await agent.run(`
Create Jest tests for:
${functionCode}
Aim for 80%+ coverage, include edge cases and mocks
`);

Notes From Solo Development

As an indie developer, the first thing I look at when deciding whether a task belongs on a local model isn't quality — it's frequency. Something that runs hundreds of times a day and can tolerate an occasional bad result is where local inference pays off immediately. Moving a heavy job that runs twice a month costs more in model maintenance than it saves.

The other thing that helped was giving up on replacing the cloud entirely. Classification, summarization, tag extraction — all the preparatory work runs on a local 9B. Whatever a user actually reads still goes to a hosted model. That split cut my API call volume substantially without any drop in output quality.

It's easy to forget that the machine is also your workstation. Running a build while inference is going makes both slower. I ended up batching heavy inference into an overnight queue and allowing only interactive single calls during the day. The wall-clock savings mattered less than not being interrupted.

I reached for 27B more than once. Rebuilding the prompt so 9B could handle it was consistently faster overall. A larger model tends to paper over vague instructions rather than fix them — when output is unstable, rewriting the prompt to be shorter and more specific beats raising the quantization level most of the time.

All of this is an indie developer's conclusion, of course — with effectively one concurrent user, I get to skip most of the queueing design. Add concurrent users and the math changes; at that point a server-side GPU is the honest answer.

One Thing to Try First

You don't need to adopt all of this at once. Start with a single change.

Pick the prompt you send most often in a day, route it to Gemma 4 9B, and run it for a week. Record two things: the eval rate from --verbose, and how often the response came back in the wrong shape. Those two numbers tell you which task to move next and where a bigger model is genuinely required — measured on your hardware rather than someone else's.

When you do move to always-on, add keep_alive and the semaphore before anything else. Both are cheap now and expensive to diagnose later.

The real value of running locally isn't speed — it's that experiments become free. Lower the cost of failure and you try more things, and trying more things is what actually improves prompts and agent design. Getting into that loop takes a bit of setup. Coming back out of it is the hard part.

Thanks for reading this far. Once you have your first eval rate, you have a baseline — and every design decision after that gets easier.

Thank You for Reading

Antigravity Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.