Why Your Local Gemma 4 Feels "Dumber" Than It Should
You connected Gemma 4 through Ollama, launched Antigravity, and asked it to write a TypeScript API client. The response came back in English even though you asked in Japanese. The code had no error handling. Type annotations were missing. It felt like a step backward from Gemini API.
Here's what's happening: when you use the Gemini API through the cloud, Google applies implicit system prompts on the server side. When you run Gemma 4 locally, the model starts with zero context about what you need. Without a system instruction, it behaves as a generic assistant with no specialization.
The fix is straightforward but often overlooked — write a proper system instruction. The difference in output quality is substantial and measurable.
How System Instructions Shape Gemma 4's Output
In Gemma 4's inference pipeline, system instructions occupy the beginning of the token sequence. Through the attention mechanism, these early tokens influence every subsequent token generation. This means what you write in the system instruction effectively sets the trajectory for the entire response.
Here's a concrete example of the difference:
Without instruction:
User: Write an API client in TypeScript
Gemma 4: Sure\! Here's a simple API client...
(no error handling, loose types, generic approach)
With instruction:
System: You are a senior TypeScript engineer.
Write production-ready code with explicit type annotations.
Always include error handling with Result patterns.
User: Write an API client in TypeScript
Gemma 4: I'll implement a type-safe API client with retry logic...
(strict types, Result pattern, retry with exponential backoff)
This gap widens with smaller models. On Gemma 4 E2B (12B parameters), the presence or absence of system instructions creates a night-and-day difference in practical usability.
Three Ways to Set System Instructions in Antigravity
Method 1: AGENTS.md (Recommended)
Create an AGENTS.md file at your project root. This is the most maintainable approach and works well for team projects since it's version-controlled.
# AGENTS.md
## Role
Senior full-stack engineer specializing in TypeScript and Next.js.
Always respond in the same language as the user's message.
## Code Standards
- Use strict TypeScript with explicit return types
- Handle errors with Result pattern (never throw in library code)
- Write JSDoc comments for exported functions
- Prefer composition over inheritance
## Testing
- Write unit tests alongside implementation
- Use Vitest with happy-path + edge-case coverage
- Mock external dependencies, never real APIs in testsA critical detail: even if you write AGENTS.md in English, include the rule Always respond in the same language as the user's message. Gemma 4 tends to follow the language of its instructions, so without this rule, it may respond in English regardless of what language you're using.
Method 2: Bake Into an Ollama Modelfile
When you want system instructions fixed at the model level, use an Ollama Modelfile.
# Modelfile
FROM gemma4:12b
SYSTEM """
You are an experienced software engineer.
Follow these rules:
1. Respond in the same language as the user
2. Always include error handling in code
3. Suggest performance-conscious implementations
4. Ask clarifying questions rather than guessing
"""
PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER num_ctx 8192# Build as a custom model
ollama create gemma4-dev -f Modelfile
# Point Antigravity to this model name
# Settings → AI Models → Local Model → gemma4-devThis approach works well when you want consistent behavior across projects. However, avoid combining it with AGENTS.md — duplicate instructions confuse the model. Pick one or the other.
Method 3: Antigravity Settings UI
Navigate to Settings → AI Models → System Instruction and type directly into the field. Quick for experiments, but not version-controlled. Best for personal prototyping or short-lived tests.
Five Principles for Effective Instructions
Principle 1: Define the Role Specifically
❌ Vague: "You are a programmer"
✅ Specific: "You are a frontend engineer specializing in TypeScript
and React, working on a Next.js app deployed to Cloudflare Workers"
With Gemma 4's 12B model, vague role definitions lead to inconsistent output. Specifying the tech stack narrows the model's "search space" for relevant patterns, and code suggestions become notably more accurate.
Principle 2: Specify Output Format Explicitly
## Response Format
- Start with a one-line summary of the approach
- Show the complete, runnable code (no snippets)
- Explain WHY this approach, not just HOW
- End with potential edge cases to watch forImplicit expectations like "just show the code" or "include an explanation" don't reliably transfer to local models. Spelling out the structure you want makes responses predictable.
Principle 3: Include Negative Constraints
Tell the model what NOT to do, not just what to do.
## Constraints
- Do NOT use `any` type in TypeScript
- Do NOT suggest deprecated APIs
- Do NOT skip error handling even in example code
- Do NOT add unnecessary dependenciesGemma 4 responds more reliably to negative constraints than positive suggestions. In my testing, any type suppression and error handling enforcement had significantly higher compliance rates when framed as "Do NOT" rules.
Principle 4: Keep It Token-Efficient
Gemma 4 E2B defaults to an 8,192-token context window. Every token your system instruction uses is a token unavailable for actual code and conversation.
Through experimentation, I've found 300–500 tokens (roughly 200–400 words in English) is the sweet spot. Beyond that, you're better off putting detailed rules in AGENTS.md and letting the file reference system handle it.
# Check token count of your instruction
echo "Your instruction text here" | ollama run gemma4:12b --verbose 2>&1 | grep "prompt eval"Principle 5: Use Task-Specific Sections
Instead of cramming everything into one block, create sections for different tasks.
# AGENTS.md — Task-Specific Sections
## When Writing New Code
Prioritize readability and type safety. Add JSDoc for all exports.
## When Reviewing Code
Focus on: security vulnerabilities, performance bottlenecks,
missing edge cases. Be specific about line numbers.
## When Debugging
Ask for the error message first. Reproduce before fixing.
Never suggest changes without explaining the root cause.Gemma 4 naturally gravitates toward the section most relevant to the current context. You don't need explicit switching logic — the model picks up on contextual cues.
Measured Results: Before and After Optimization
I compared response quality across 10 tasks on the same project (Next.js + TypeScript + Prisma), before and after instruction optimization:
- Type safety (
anyusage): Before — 6 out of 10 tasks usedany→ After — 0 out of 10 - Error handling included: Before — 3/10 tasks → After — 9/10 tasks
- Language consistency (Japanese input → Japanese response): Before — 40% → After — 95%
- First-response acceptance rate (usable without edits): Before — 20% → After — 60%
The language consistency improvement is the most striking. A single line — Always respond in the same language as the user's message — drove that metric from 40% to 95%.
Common Pitfalls and How to Avoid Them
Pitfall 1: Context overflow from long instructions
When you have a large file open and request inline edits, the system instruction + file content + conversation history can exceed the context window. The response cuts off mid-sentence. Either expand num_ctx to 16384, or trim your instruction below 200 tokens.
Pitfall 2: Contradictory instructions
"Be concise" and "Include detailed explanations" in the same instruction set causes Gemma 4 to flip randomly between modes. Review your instructions for internal consistency.
Pitfall 3: Instructions entirely in a non-English language
Since Gemma 4's training data is predominantly English, writing instructions in English with a "respond in the user's language" rule produces more stable results. Full Japanese instructions can cause side effects like Japanese variable names or Japanese code comments where you'd expect English.
Ready-to-Use Templates
Web Application Development
# AGENTS.md
## Role
Full-stack TypeScript engineer. Tech stack: Next.js 16,
Prisma ORM, Cloudflare Workers, Tailwind CSS.
## Rules
- Respond in the user's language
- Use server components by default, client components only when needed
- All database queries through Prisma with explicit select
- CSS via Tailwind utility classes, no custom CSS files
## Do NOT
- Use `any` or `as` type assertions without justification
- Import from barrel files (use direct path imports)
- Create API routes for data that can be fetched in server componentsMobile App Development
# AGENTS.md
## Role
iOS/Android engineer using Swift and Kotlin.
Focus on performance and battery efficiency.
## Rules
- Respond in the user's language
- SwiftUI for iOS, Jetpack Compose for Android
- Follow platform HIG/Material Design guidelines
- Handle offline scenarios in every network call
## Do NOT
- Block the main thread with synchronous operations
- Store sensitive data in UserDefaults/SharedPreferences
- Use force unwrap (\!) in Swift production codeData Engineering and Scripts
# AGENTS.md
## Role
Data engineer proficient in Python and SQL.
## Rules
- Respond in the user's language
- Use pandas for tabular data, polars for large datasets
- Type hints on all function signatures
- Include data validation before processing
## Do NOT
- Use mutable global state
- Ignore encoding issues (always specify utf-8)
- Skip null/NaN handlingThe Takeaway: A Small Investment with Outsized Returns
When running Gemma 4 locally, a system instruction isn't optional — it's the single highest-leverage configuration you can make. A 300–500 token investment transforms the model from a generic assistant into a project-aware specialist.
Start by creating an AGENTS.md at your project root with three things: role definition, tech stack, and a short list of "Do NOT" rules. That alone will noticeably change how your local Gemma 4 behaves.
If you haven't set up Gemma 4 with Antigravity yet, the Antigravity × Gemma 4 Integration Guide walks through the Ollama setup step by step. Once that's running, come back here and apply these templates.