"How do I get the most out of Gemma 4 with Antigravity?" is a question hitting my inbox almost daily. Since Antigravity gained native Gemma 4 support, interest in running serious local LLM setups has spiked among developers.
I've been running Antigravity + Gemma 4 across several personal projects — local-only editor environments, RAG-based codebase search, fine-tuned models in steady operation. This article documents everything I've learned, including the pitfalls, in one comprehensive guide.
The volume goes well beyond what fits in a free article, so this is a premium piece. It's aimed at developers seriously considering Antigravity as a local LLM environment and at engineers who want to extract Gemma 4's full potential.
Why This Combination Is Currently the Hot Choice
Two reasons stand out.
First, Antigravity is the only major VS Code-based editor with deep native Gemma 4 integration. You can call Gemma 4 from Cursor or VS Code via third-party extensions, but the depth of integration in Antigravity is unmatched.
Second, Gemma 4's performance has crossed the practical threshold for locally-runnable models. The 27B variant approaches Claude or GPT-4o quality on coding and dialog tasks while running on consumer-grade GPUs (~24GB VRAM). For developers wanting to reduce cloud AI dependency, a genuinely viable option has finally arrived.
What I personally value most is privacy and offline capability. Client work often comes with NDAs that prohibit sending code to cloud AI. Running Gemma 4 locally lets me keep AI coding assistance without violating confidentiality.
Hardware and OS Requirements
Minimum specs for comfortable Gemma 4 + Antigravity operation:
VRAM depends heavily on quantization. 27B at 8-bit needs ~32GB; at 4-bit (GGUF Q4_K_M) it drops to ~18GB. Apple Silicon via MLX uses unified memory, with M3 Max 64GB+ being the practical line.
System RAM should be at least 32GB beyond VRAM. Antigravity itself uses memory, and model loading temporarily uses system RAM.
Storage: SSD with 100GB+ free. Gemma 4 27B is ~50GB at FP16, ~18GB quantized. Switching between variants needs even more.
OS: macOS (Apple Silicon), Linux (Ubuntu 22.04+), or Windows 11. In my experience, macOS has the fewest issues. Linux occasionally has CUDA driver problems.
Gemma 4 Setup via Ollama
Ollama is the simplest way to run Gemma 4 locally, and Antigravity's integration with it is the most mature.
# Install Ollama (macOS)
brew install ollama
# Start the Ollama service
ollama serve &
# Pull the Gemma 4 model (4-bit quantized)
ollama pull gemma4:27b
# Verify
ollama run gemma4:27b "Name three key features of Antigravity."A frequent issue: error: pull model manifest: file does not exist. This appears when the registry doesn't have the requested version, or the model name is mistyped.
# Check available variants
ollama list
# Search the registry for available models
curl https://ollama.com/api/tags | jq '.[] | select(.name | contains("gemma"))'
# Try different size variants
ollama pull gemma4:4b
ollama pull gemma4:12bOnce Ollama is running, configure Antigravity:
// Add to Antigravity's settings.json
{
"antigravity.localLLM.provider": "ollama",
"antigravity.localLLM.endpoint": "http://localhost:11434",
"antigravity.localLLM.defaultModel": "gemma4:27b",
"antigravity.localLLM.contextSize": 32768,
"antigravity.localLLM.temperature": 0.3
}I'm setting contextSize to 32768 here, but Gemma 4 supports up to 128K. With sufficient VRAM, expanding it lets you process long code files in single passes.
Setup via LM Studio
For GUI-based management, LM Studio is the go-to. On Apple Silicon, MLX backend gives the best performance.
After installing LM Studio, search for gemma-4-27b-it, choose a quantization variant (GGUF Q4_K_M, MLX 4-bit, etc.), and download. Enable Local Server mode and you can hit it via HTTP API.
// Antigravity settings.json
{
"antigravity.localLLM.provider": "lmstudio",
"antigravity.localLLM.endpoint": "http://localhost:1234/v1",
"antigravity.localLLM.defaultModel": "google/gemma-4-27b-it",
"antigravity.localLLM.apiKey": "lm-studio"
}A frequent issue with LM Studio is the gemma 4 support is not ready yet error during model loading. This happens with older LM Studio versions. Update to the latest (0.3.x or newer as of May 2026).
Another issue is memory pressure on Apple Silicon. MLX backend has its own memory management which can affect other apps. Watch memory compression in Activity Monitor and quit other apps as needed.
Using Gemma 4 in Antigravity's Agent Mode
The real value of Antigravity comes from its Agent mode running on local LLMs. Long-running tasks that would be hesitation-inducing on cloud models become easy to throw at local Gemma 4.
// Example using Antigravity's Agent API
const agent = await antigravity.startAgent({
task: 'Investigate the entire src/ directory and remove unused exports',
scope: ['src/'],
model: 'gemma-4-27b-local',
options: {
maxIterations: 50,
maxRuntimeMinutes: 30,
saveCheckpoints: true
}
})
// Monitor progress
agent.on('progress', (event) => {
console.log(`[${event.iteration}] ${event.action}: ${event.target}`)
})
agent.on('checkpoint', (event) => {
console.log(`Checkpoint saved: ${event.path}`)
})
const result = await agent.complete()
console.log(`Done: ${result.filesModified} files modified`)The crucial detail is saveCheckpoints: true. Local LLMs occasionally stall, and checkpoint resumption is what makes long-running operation practical.
Building a RAG Pipeline
Build a pipeline that uses Gemma 4 as the generation side of a Retrieval-Augmented Generation system. This makes your entire codebase searchable, with Gemma 4 leveraging it for answers.
# Minimal RAG pipeline (Python)
from sentence_transformers import SentenceTransformer
from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct
import requests
import os
from pathlib import Path
# Local embedding model
embedder = SentenceTransformer('intfloat/multilingual-e5-large')
# Local Qdrant vector database
client = QdrantClient(host="localhost", port=6333)
client.recreate_collection(
collection_name="codebase",
vectors_config=VectorParams(size=1024, distance=Distance.COSINE)
)
# Chunk and index the codebase
def index_codebase(root_path: str):
points = []
for path in Path(root_path).rglob("*.py"):
content = path.read_text()
chunks = chunk_code(content, max_lines=50)
for i, chunk in enumerate(chunks):
embedding = embedder.encode(chunk).tolist()
points.append(PointStruct(
id=f"{path}:{i}",
vector=embedding,
payload={"path": str(path), "chunk": chunk}
))
client.upsert(collection_name="codebase", points=points)
def chunk_code(content: str, max_lines: int) -> list[str]:
lines = content.split("\n")
return ["\n".join(lines[i:i+max_lines]) for i in range(0, len(lines), max_lines)]
# Generate answers with Gemma 4
def query_codebase(question: str) -> str:
query_vector = embedder.encode(question).tolist()
results = client.search(
collection_name="codebase",
query_vector=query_vector,
limit=5
)
context = "\n\n".join([r.payload["chunk"] for r in results])
response = requests.post(
"http://localhost:11434/api/generate",
json={
"model": "gemma4:27b",
"prompt": f"Use the code snippets below to answer the question.\n\n{context}\n\nQuestion: {question}",
"stream": False
}
)
return response.json()["response"]
# Usage
index_codebase("./src")
answer = query_codebase("Where is the authentication flow implemented?")
print(answer)The practical insight here is chunk size selection. 50 lines is what I've settled on through experience — preserves enough structural context while maximizing Qdrant search precision. Too small loses context; too large pulls in irrelevant noise.
The other point is using multilingual-e5-large as the embedding model. If your codebase has comments in multiple languages, English-only embedders won't match well. Multilingual models give consistent retrieval whether comments are in Japanese or English.
Fine-Tuning — Customizing Gemma 4 with LoRA
To teach Gemma 4 your domain knowledge, LoRA (Low-Rank Adaptation) fine-tuning is the practical path. Far less compute than full fine-tuning, while reflecting your code style or business knowledge.
# Minimal LoRA fine-tuning setup
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model
from datasets import load_dataset
from transformers import TrainingArguments, Trainer
# Load model and tokenizer
model_name = "google/gemma-4-27b-it"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
load_in_4bit=True,
device_map="auto"
)
# LoRA configuration
lora_config = LoraConfig(
r=16,
lora_alpha=32,
target_modules=["q_proj", "v_proj"],
lora_dropout=0.1,
bias="none",
task_type="CAUSAL_LM"
)
model = get_peft_model(model, lora_config)
# Prepare dataset (e.g., your code review comments)
dataset = load_dataset("json", data_files="./training_data.jsonl")
# Train
training_args = TrainingArguments(
output_dir="./gemma4-lora-output",
num_train_epochs=3,
per_device_train_batch_size=2,
gradient_accumulation_steps=8,
learning_rate=2e-4,
fp16=True,
save_steps=100,
logging_steps=10,
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=dataset["train"],
tokenizer=tokenizer,
)
trainer.train()
# Save the fine-tuned model
model.save_pretrained("./gemma4-lora-finetuned")Once trained, load the LoRA adapter into Ollama or LM Studio, and Antigravity can call your custom-tuned Gemma 4.
A common fine-tuning pitfall is overfitting. With small training data (under a few hundred examples), you destroy Gemma 4's base knowledge and lose ability to answer out-of-domain questions. Mitigations: lower learning rate (~2e-5), fewer epochs (1-2), more data (1,000+ examples).
Another pitfall is out-of-memory errors. load_in_4bit=True helps significantly, but 24GB VRAM is the floor. To go further, add gradient_checkpointing=True.
Choosing Quantization — Quality vs. Memory
Quantization choice is the most consequential decision for practical Gemma 4 use.
FP16 (no quantization): highest quality, but ~54GB VRAM for 27B. Effectively datacenter GPU only.
Q8 (8-bit): quality degradation imperceptible in practice, ~30GB VRAM. RTX 4090 (24GB) won't fit; needs A100 or M3 Ultra class.
Q5_K_M (5-bit): minor quality degradation, ~20GB VRAM. Comfortable on RTX 4090 or M2 Pro 32GB. This is what I run daily.
Q4_K_M (4-bit): noticeable quality degradation in some scenarios, ~18GB VRAM. Works on RTX 4080 or M2 Pro 16GB. For coding, still practically usable.
Q3 / Q2: significant quality drop, errors stand out in code generation. Acceptable for chat or experiments, not recommended for production code generation.
My recommendation: Q5_K_M for RTX 4090 or M3 Max class, Q4_K_M for anything below. Q4 vs Q5 differences are subtle in many scenarios — if you have memory headroom, go Q5_K_M.
Operational Pitfalls — Lessons From Real Projects
Real pitfalls from running Gemma 4 + Antigravity in production:
Memory spikes during long-context processing. Running with the full 128K context burns VRAM rapidly. Practically, limit to 32K-64K and expand only when needed.
Sudden Ollama service halts. Ollama occasionally hangs and stops responding, causing Antigravity to get connection refused errors. Run ollama serve under systemd or launchd with health checks and auto-restart for stability.
Corrupted model files. Network interruptions during Ollama updates can corrupt models. ollama list may show them as present, but they don't actually run. Periodic verification scripts catch this early.
Compatibility issues with Antigravity updates. When Antigravity updates, the local LLM settings schema sometimes shifts subtly. Version-control your settings.json and verify functionality after updates.
My Current Setup — For Reference
My Gemma 4 + Antigravity setup as of May 2026, for reference:
Hardware is Mac Studio (M3 Ultra 96GB). MLX backend running Gemma 4 27B Q5_K_M loaded at all times. Unified memory means I can also keep multiple models loaded concurrently (Gemma 4 27B, Gemma 4 12B, embedding model).
Editor is Antigravity (VS Code-based). Agent mode uses Gemma 4 27B Q5_K_M, while simple completions use Gemma 4 12B. The smaller model handles latency-sensitive scenarios (typing-time completions) better.
RAG pipeline runs Qdrant + multilingual-e5-large, indexing my collection of indie projects (~500K lines total). Being able to instantly search "how did I write this in past projects" has dramatically accelerated development.
LoRA fine-tuning happens about monthly, training on my code style (naming conventions, comment patterns, design choices). I switch between base Gemma 4 and my custom-tuned variant depending on the task.
Looking back
Gemma 4 + Antigravity is currently the most viable choice for serious local LLM-based development. Combining Ollama or LM Studio execution, RAG pipelines, LoRA customization, and quantization optimization, you get an offline development experience that rivals cloud AI.
Adoption hurdles are real, but once set up, you're free of monthly AI costs. Privacy-strict client work, offline development, and aggressively cost-conscious indie projects — all are scenarios where local LLM shines.
I haven't moved everything local — I still split between cloud AI and local LLM by task. But Gemma 4 + Antigravity has matured to the point where "local is enough for more situations than before" feels concretely true.
Walking through these steps gets you to a working Gemma 4 + Antigravity setup in your own environment. Stuck on something? Our other articles dig into specific topics in more depth.