When I first started using Antigravity, I only connected it to cloud LLMs like Gemini and Claude. That seemed sufficient at the time. Then a project came in involving sensitive internal documents I couldn't ship to a cloud API, and the situation flipped. I needed to keep Antigravity but wanted inference to run locally.
Antigravity turned out to be more flexible than I expected. You can wire it up to Ollama or LM Studio for local inference. Not "one click and done" like cloud LLMs, but with the right configuration, the combination is genuinely production-usable.
Here are the configurations I converged on, the cloud-vs-local task splits I learned through use, and the workarounds for the constraints unique to local LLMs. The goal is to nudge the "interested but it sounds like a hassle" reader into actually trying it.
Why Use Local LLMs Through Antigravity
My motivations come down to three:
Sensitive data handling. I sometimes work with NDA-protected client data marked "must not be sent to external APIs." Cloud LLMs are off the table; local-only completion is allowed. This is the most pressing motivator.
Cost. Automation that calls an API hundreds of times a day adds up. If your local hardware has spare capacity, sending routine work to local LLMs and reserving the cloud for hard judgments is economically sound.
Offline resilience. Sounds minor, but if you ever work in flaky network environments, having local inference available is a quiet blessing. I sometimes work overseas, and local LLMs save me when local internet is slow or APIs lag.
When not to use local LLMs: tasks that need state-of-the-art reasoning (complex design judgment, long-form logical consistency checks). Models you can run on your own machine don't reach those heights.
Ollama or LM Studio?
The two main local inference servers Antigravity connects to are Ollama and LM Studio. I've used both and split them by use case.
Ollama's strength: CLI-driven and stable. ollama pull gemma3:27b to fetch a model, ollama serve to expose an OpenAI-compatible API. Simple structure, well-suited to "running in the background, always on" scenarios. I have it parked on a Mac mini and hit it from every device on my home LAN.
LM Studio's strength: All-in-GUI. Model downloads, quantization choice, system-instruction tuning — all on screen. For exploratory prompt iteration, LM Studio is far easier. For ongoing operations, Ollama wins on stability.
My split: "LM Studio for hands-on exploration, Ollama for production." Same GGUF models work in both, so migration cost is low.
Concrete Setup — Ollama
To call Ollama from Antigravity, add it as a custom provider in Antigravity's settings, with an OpenAI-compatible endpoint.
Provider Name: Ollama (local)
Base URL: http://localhost:11434/v1
API Key: ollama (any string is fine)
Model: gemma3:27b (the model you previously ollama-pulled)
The trailing /v1 on the Base URL matters. Without it, Antigravity's OpenAI Chat Completions request hits a 404.
The API Key isn't actually used by Ollama, but Antigravity's validation rejects empty values. Any string works — ollama, dummy, anything.
To reach Ollama on a different machine, allow external connections on the Ollama side. Default is localhost-only, so launch with OLLAMA_HOST=0.0.0.0:11434 ollama serve. Within the same LAN, you can then point Antigravity at http://192.168.1.10:11434/v1.
For security: do not expose this to external LAN or WAN. At minimum, route via VPN or SSH tunnel. Anything less means anyone can borrow your machine for free inference.
Concrete Setup — LM Studio
In LM Studio, the "Local Server" tab launches the API server. The default port is 1234 and it also speaks OpenAI-compatible API.
Provider Name: LM Studio (local)
Base URL: http://localhost:1234/v1
API Key: lm-studio
Model: name of the currently-loaded model
A handy LM Studio feature: server-launch system instructions are configurable in the GUI. Forcing Japanese responses, tweaking quantization behavior — all without changing prompts on the Antigravity side.
The catch: the LM Studio API server only runs while the app is open, so it's not for long-running production. A common pattern is "LM Studio for daytime experimentation, Ollama for overnight batches."
Model Selection for Antigravity Use
The models I'm currently using from Antigravity. Optimal choice varies by your hardware, but as a reference:
Main driver: Gemma 3 27B. On an M2 Pro Mac mini with 32GB RAM, the Q4_K_M quantization runs comfortably. Code completion, prose editing, summarization — quality approaches cloud LLM levels for general tasks.
For long contexts: quantized Llama 3.3 70B. Noticeably slower, so not for interactive use. Reserved for batch jobs that can take their time.
For lightweight tasks: Phi-3.5 mini (3.8B). Filename normalization, simple classification, JSON formatting — "mechanical transformation" tasks. Fast enough to run as a constantly-available helper agent.
The most common selection mistake: "go as big as possible." A model that just fits your RAM swaps mid-inference and crawls. Pick a model size around 70% of effective RAM for safety.
Routing Between Cloud and Local
One of Antigravity's strengths is letting providers coexist. I rule-encode which provider to use per project.
Projects with sensitive data: all inference pinned to local LLMs via Ollama. With the default provider set to Ollama at the Antigravity project level, accidental cloud sends are prevented.
Low-sensitivity OSS projects: default to cloud (Gemini or Claude), with local taking lighter tasks like code completion. Cost optimization meets quality.
A useful technique across projects: write the defaultProvider into Antigravity's project config (e.g., .antigravity/config.json — paths vary by version). Opening the project auto-selects the right provider.
Local LLM Constraint 1: Context Length
The classic local-LLM limit. Cloud LLMs support 1M-token contexts; local LLMs typically max out at 8k–32k. Some Gemma 3 variants extend to 128k, but RAM consumption explodes — practical range is limited.
Three workarounds:
Use Antigravity's context-splitting features. When operating on a long file, Antigravity has a "extract relevant portions and pass only those to the model" mode. Enable it and you stay inside local-LLM context limits more often.
Pre-summarize before sending to local. For long meeting-notes summarization: summarize chapter by chapter on cloud LLM, then merge on local LLM. Antigravity lets you mix providers, so this two-stage approach is easy to wire.
Concentrate context-light tasks on local. "Refactor this single function" needs less context than "review this whole codebase." Mind your task granularity and the context limit becomes much less of a wall.
Local LLM Constraint 2: Function Calling Quality
Cloud function calling has gotten dramatically smarter; local LLMs lag here. Even Gemma 3 27B struggles with complex tool chains.
Two responses:
Don't ask local LLMs to do complex function calling. Keep agentic patterns in the cloud; local handles "single-shot response generation."
Use a function-calling alternative. Instead of OpenAI-style tool calls, use "force JSON output" prompts. Write "respond with only JSON in this exact shape: { ... }" and parse the response yourself. Most local models do this stably.
If you write Antigravity plugins, building a thin local-LLM wrapper is worthwhile. One plugin handles "prompt → JSON output → parse → invoke function" end to end.
Local LLM Constraint 3: Speed
The last constraint is raw inference speed. Gemma 3 27B on Mac mini M2 Pro generates roughly 12–18 tokens/sec. Several times slower than cloud LLMs, which makes interactive use feel sluggish.
The answer is task allocation. Real-time tasks (code completion, dialog) → cloud. Batch tasks (overnight automation, log summarization, content generation) → local.
My site automation runs in the small hours, so a slower model doesn't matter. Local LLMs do their work in the quiet hours and keep API costs down. The division has settled into a stable rhythm.
Realistic Hardware Choices
If you're going to use local LLMs through Antigravity seriously, hardware matters. Three price tiers based on my experience:
Cheapest: leverage an existing M-series Mac. MacBook Pro M3 Pro or better with 32GB+ RAM runs Gemma 3 27B–class models usefully. No new investment to start.
Mid-range dedicated: Mac mini M2 Pro / M4 Pro with 32–64GB RAM. Around $2,000–$3,000 buys a 24/7 inference server. This is what I use.
High-end: Apple Silicon Mac Studio (96GB+) or a Linux workstation with NVIDIA RTX 4090 / 5090. For comfortable 70B-class operation. ~$7,000–$10,000 investment, but if you're spending $300+/month on cloud LLMs, ROI lands within a year.
Production Monitoring
If you're running local LLMs in production, basic monitoring is non-negotiable. The metrics I watch:
GPU / Neural Engine temperature and utilization. Long sustained loads can trigger thermal throttling and inference suddenly slows. Tools like asitop for periodic checks.
Memory usage and swap activity. Once swap kicks in, inference slows by orders of magnitude. Adjust model size to stay below the threshold.
Median and p95 response time. Looking only at median hides occasional latency spikes. p95 reveals the user-perceived bad experiences.
A simple Grafana dashboard or homemade visualization makes problem isolation much faster when something breaks.
When Switching to a Local Model Makes Every Response Turn English
The first thing that trips people up after wiring in a local LLM: task summaries that came back in your chosen language on the cloud model suddenly arrive in English the moment you point Antigravity at Ollama.
Your Antigravity settings are not being ignored. What happens is that smaller models tend to drop the language instruction from the system prompt. At the Gemma 3 4B scale, a single line saying "respond in Japanese" is not enough — the model is pulled around by where that instruction sits and by the language mix of the prompt itself.
Working through this on my own machine as an indie developer, two changes finally made it stable.
- Put the output-language instruction at the end of each request, not only in the system prompt. The model reacts far more strongly to recent tokens
- Include exactly one few-shot example in the target language. A single example is usually enough to lock the response language for the rest of the session
The opposite failure also exists: sometimes you do not want code comments translated. In that case, state both rules explicitly — prose in one language, code comments in another. Specify only one, and the model will drag everything toward it.
| Symptom | What fixed it |
|---|---|
| Only summaries turn English | Restate the output language at the end of the request |
| Response language drifts each run | Add one few-shot example in the target language |
| Code comments get translated too | Declare prose and code languages separately |
For where those language settings actually live, see "Setting Up Antigravity 2.0 with a Japanese UI", and if you want the whole model runtime pinned inside a container for reproducibility, "Antigravity DevContainer: A Complete Setup Guide" covers that ground.
Minimum Steps for Tomorrow
The smallest path to start tomorrow:
Install Ollama (brew install ollama), pull a small model with ollama pull gemma3:4b, launch with ollama serve, then add the custom provider in Antigravity using the settings above. Run a single text-generation task in Antigravity and confirm a response.
That's about an hour of work. Once it's working, climb up to bigger models and harder tasks.
The realistic framing: local LLMs are a complement to cloud LLMs, not a replacement. An Antigravity setup that handles both confidently widens the kind of work you can take on. I started accepting confidential projects only after building this combination. Hopefully this writeup helps anyone facing a similar wall.