Fine-Tuning Gemma 4 with Antigravity: A Practical Guide to Building Custom AI Models

Why Fine-Tune Gemma 4?

Google's Gemma 4, released in April 2026, stands out as an open-source LLM with multimodal support, strong reasoning capabilities, and a commercial-friendly license. Its deep compatibility with Antigravity has made it a go-to model for many developers.

That said, using a general-purpose model off the shelf doesn't always cut it. You might run into situations like:

• The model struggles with domain-specific terminology or internal knowledge
• Output consistency is inconsistent for structured formats (JSON, specific code styles)
• You're handling sensitive data that can't be sent to a cloud API

This is where fine-tuning comes in. In this guide, we'll walk through the full process of fine-tuning Gemma 4 with LoRA/QLoRA, converting the result to a local model, and integrating it with Antigravity.

If you'd like to get familiar with the Gemma 4 API basics first, check out the Antigravity × Gemma 4 API Implementation Guide.

Understanding LoRA and QLoRA

The Challenge with Full Fine-Tuning

Full fine-tuning updates every parameter in the model, which requires tens of gigabytes of VRAM — far beyond what individual developers typically have access to. That's where LoRA (Low-Rank Adaptation) and QLoRA (Quantized LoRA) change the equation.

How LoRA works:

• The original model weights are frozen; only small "adapter matrices" are trained
• Trainable parameters drop to roughly 0.1–1% of the total
• At inference time, adapters are merged back into the model weights

What QLoRA adds:

• Combines LoRA with 4-bit quantization to drastically cut VRAM usage
• Makes it possible to fine-tune 7B-class Gemma 4 models on under 16 GB of VRAM
• Accuracy loss is minimal compared to full fine-tuning (typically a few percent)

A practical starting point is QLoRA. If accuracy still falls short, you can step up to standard LoRA with bf16 precision.

What You Need

• GPU: 16 GB+ VRAM (Google Colab Pro with A100 is ideal) or Apple Silicon Mac with Metal backend
• Python: 3.11 or higher
• Libraries: transformers, peft, trl, bitsandbytes, datasets
• Antigravity: v1.20 or later (supports local custom model connections)

Setting Up Your Environment

Installing Python Dependencies

# Install required packages
pip install transformers==4.48.0 peft==0.13.0 trl==0.13.0 \
            bitsandbytes==0.44.1 datasets==3.2.0 accelerate==1.2.0
 
# Download Gemma 4 from HuggingFace Hub
huggingface-cli login
# Enter your HuggingFace access token
# Make sure you have access to google/gemma-4-7b-it

Configuring Antigravity for Custom Models

Add the following to your Antigravity config file (~/.antigravity/config.json):

{
  "models": {
    "custom": [
      {
        "name": "gemma4-finetuned",
        "provider": "ollama",
        "endpoint": "http://localhost:11434",
        "modelId": "gemma4-custom:latest"
      }
    ]
  }
}

Once your fine-tuned model is served through Ollama, Antigravity's code completion and chat features will use your custom model automatically.

Building Your Training Dataset

Dataset Format

For instruction fine-tuning (SFT: Supervised Fine-Tuning), structure your data like this:

# Sample dataset structure
dataset_sample = [
    {
        "instruction": "Review the following code and suggest improvements.",
        "input": "def divide(a, b):\n    return a / b",
        "output": "def divide(a, b):\n    if b == 0:\n        raise ValueError('Division by zero is not allowed')\n    return a / b\n\n# Reason: Added a guard against ZeroDivisionError before performing the operation."
    },
    # Aim for at least 100 samples; 500-1,000 is ideal
]

Using Antigravity to Speed Up Dataset Creation

Antigravity's multi-file editing and agent features make dataset creation much faster. Open a data generation script in Antigravity and prompt the agent:

# generate_dataset.py - Example script for AI-assisted dataset generation
import json
import random
 
def create_coding_sample(code_snippet: str, language: str) -> dict:
    """Generate a fine-tuning sample from a code snippet."""
    templates = [
        f"Review the following {language} code and suggest improvements.",
        f"Add documentation to the following {language} code.",
        f"Optimize the following {language} code for performance.",
    ]
    return {
        "instruction": random.choice(templates),
        "input": code_snippet,
        "output": "(In real datasets, this is a high-quality human-written response)"
    }
 
# Aim for 500-1,000 samples for consistent improvements

Once this script is open in Antigravity, you can tell the agent: "Generate 200 code review samples from our existing codebase." The AI will scan your project and create a diverse training set automatically.

Running QLoRA Fine-Tuning

# fine_tune_gemma4.py
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model, TaskType
from trl import SFTTrainer, SFTConfig
from datasets import load_dataset
 
# 1. Quantization config (QLoRA: 4-bit)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)
 
# 2. Load model and tokenizer
model_id = "google/gemma-4-7b-it"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto",
)
 
# 3. LoRA adapter configuration
lora_config = LoraConfig(
    r=16,                      # Rank: lower = fewer trainable params
    lora_alpha=32,             # Scaling factor
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM,
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# Expected output: trainable params: 83,886,080 || all params: 7,262,437,376 || trainable%: 1.155
 
# 4. Load dataset
dataset = load_dataset("json", data_files="my_dataset.jsonl", split="train")
 
# 5. Training configuration
training_args = SFTConfig(
    output_dir="./gemma4-finetuned",
    num_train_epochs=3,
    per_device_train_batch_size=2,
    gradient_accumulation_steps=8,
    learning_rate=2e-4,
    warmup_ratio=0.1,
    lr_scheduler_type="cosine",
    save_steps=100,
    logging_steps=10,
    bf16=True,
    max_seq_length=2048,
)
 
trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
)
trainer.train()
 
# 6. Save the adapter
model.save_pretrained("./gemma4-lora-adapter")
tokenizer.save_pretrained("./gemma4-lora-adapter")
print("LoRA adapter saved to: ./gemma4-lora-adapter")

On an A100 GPU (40 GB VRAM), a 500-sample dataset over 3 epochs typically takes 30–60 minutes — entirely within Google Colab Pro's capabilities.

Deploying Your Custom Model in Antigravity

Exporting to Ollama

# 1. Merge LoRA adapter into the base model
python -c "
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained('google/gemma-4-7b-it')
model = PeftModel.from_pretrained(base, './gemma4-lora-adapter')
model.merge_and_unload().save_pretrained('./gemma4-merged')
AutoTokenizer.from_pretrained('google/gemma-4-7b-it').save_pretrained('./gemma4-merged')
"
 
# 2. Convert to GGUF format (using llama.cpp's convert script)
python llama.cpp/convert_hf_to_gguf.py ./gemma4-merged \
    --outfile gemma4-custom.gguf --outtype q4_k_m
 
# 3. Register and test with Ollama
cat > Modelfile << 'EOF'
FROM ./gemma4-custom.gguf
PARAMETER temperature 0.7
SYSTEM You are a skilled software engineer. Answer concisely and clearly.
EOF
ollama create gemma4-custom -f Modelfile
ollama run gemma4-custom "Hello, can you review this code?"

Once Ollama is running, select gemma4-custom in Antigravity's model settings — your fine-tuned model is now powering code completion and chat.

For advanced production deployment patterns using custom local models, the Antigravity × Gemma 4: Building Production AI Agents with Local LLMs guide goes much deeper.

Common Errors and Fixes

CUDA out of memory:

Reduce batch size to 1 and increase gradient_accumulation_steps to 16 to maintain effective batch size while cutting VRAM usage. Also try reducing max_seq_length to 1024.

bitsandbytes not found:

Run pip install bitsandbytes --upgrade. On Apple Silicon Macs, look into the Metal backend alternative for quantized inference.

Ollama model won't start:

If the GGUF file is too large (e.g., q8_0 format), switch to a lighter quantization format like q4_k_m. Verify the model is registered correctly with ollama list.

Looking back

In this guide, we covered the full pipeline for fine-tuning Gemma 4 with QLoRA and integrating it into Antigravity:

• LoRA / QLoRA: Efficient fine-tuning with minimal VRAM requirements
• Dataset creation: Accelerated with Antigravity's agent and multi-file editing features
• Ollama integration: Serving your custom model as a local LLM in Antigravity

Running a domain-specialized AI assistant locally is one of the best ways to balance privacy, performance, and customization. The code in this guide gives you a solid foundation — now it's time to build your own Gemma 4 custom model.