Replacing Prompt Engineering with DSPy in Antigravity: A Production Playbook for Self-Optimizing LLM Programs

Setting Up DSPy in an Antigravity Project

Open a terminal in Antigravity, create a virtualenv, and install the dependencies. I keep mine at the project root so the workspace's TypeScript and Python sides share the same .env.

# Run from Antigravity's integrated terminal
python -m venv .venv
source .venv/bin/activate
 
# Core DSPy and the lightweight provider router
pip install "dspy-ai>=2.6.0" "litellm>=1.50.0" python-dotenv
 
# Pydantic for typed outputs
pip install pydantic

Drop your provider keys in .env, and add a Custom Rule that tells Antigravity's AI to never inline secrets — only reference them through os.getenv. If you do not have a Custom Rules file yet, Mastering Custom Rules and Project Config in Antigravity walks through a setup that prevents the AI from leaking credentials into your repo.

Run a smoke test to confirm the wiring:

# scripts/dspy_smoke.py
import os
from dotenv import load_dotenv
import dspy
 
load_dotenv()
 
lm = dspy.LM(
    "gemini/gemini-2.5-flash",
    api_key=os.environ["GEMINI_API_KEY"],
    temperature=0.0,
    max_tokens=1024,
)
dspy.settings.configure(lm=lm)
 
class SimpleQA(dspy.Signature):
    """Answer the question concisely in one or two sentences."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField(desc="One or two sentences in English")
 
predict = dspy.Predict(SimpleQA)
print(predict(question="What problem does DSPy solve?").answer)

Expected output (your wording will vary but the shape stays stable):

DSPy lets you declare LLM programs as typed functions and have an optimizer learn the prompts and few-shot examples from data, replacing brittle hand-written prompts.

Once this works, every later example in this article slots into the same project structure with no additional ceremony. If you ask Antigravity's Manager Surface to "rewrite this signature with ChainOfThought," it will produce the exact diff you need without any prompt-string surgery on your part.

A Real Workload: A Structured-Extraction Module

Let's leave the toy examples behind. The case I have shipped most often is structured extraction from invoice PDFs. To keep the article portable, assume an upstream OCR step has already produced raw text — your job is to extract a typed Invoice object from it.

# pipelines/invoice_extraction.py
from typing import List
from pydantic import BaseModel, Field
import dspy
 
class LineItem(BaseModel):
    description: str = Field(description="Product or service name")
    quantity: float = Field(description="Quantity. Default to 1.0 if unknown.")
    unit_price: float = Field(description="Unit price (excluding tax, JPY)")
 
class Invoice(BaseModel):
    vendor: str = Field(description="Vendor name")
    invoice_date: str = Field(description="Issue date in YYYY-MM-DD")
    total_amount: float = Field(description="Total including tax (JPY)")
    line_items: List[LineItem] = Field(default_factory=list)
 
class ExtractInvoice(dspy.Signature):
    """Extract structured invoice fields from raw OCR text.
    Numbers must be half-width digits. Dates must be normalized to YYYY-MM-DD.
    Never return null for required fields — fill in the most likely value."""
    ocr_text: str = dspy.InputField(desc="Plain text from OCRing the invoice PDF")
    invoice: Invoice = dspy.OutputField(desc="The extracted, typed result")
 
class InvoiceExtractor(dspy.Module):
    def __init__(self):
        super().__init__()
        # ChainOfThought lifts accuracy on extraction tasks where line totals
        # have to reconcile against the invoice total.
        self.extract = dspy.ChainOfThought(ExtractInvoice)
 
    def forward(self, ocr_text: str) -> Invoice:
        return self.extract(ocr_text=ocr_text).invoice

Notice that this code contains zero string prompts. The signature's docstring and the Pydantic field descriptions are what end up being assembled into the actual instruction. After you compile this module with an optimizer, the wording is something the optimizer chose — and almost always something you would not have written by hand.

The reason I wrap the signature in ChainOfThought here is that invoice extraction has implicit consistency constraints (the line totals should sum to the invoice total). Internal reasoning steps stabilize the output. For lighter tasks where every token is a cost line item, plain dspy.Predict is the better default. The right call comes from running both against your evaluation set and reading the metric.

A Real Workload: A ReAct Tool-Using Agent

Some tasks need to reach out to an external API before they can answer. DSPy's dspy.ReAct packages the tool-calling loop into a single module. I use this pattern when an Antigravity-orchestrated multi-agent system delegates a focused subtask to a DSPy worker.

# agents/exchange_rate_agent.py
import requests
import dspy
 
def get_exchange_rate(from_ccy: str, to_ccy: str) -> str:
    """Look up an FX rate. Returns a human-readable string or an error message."""
    try:
        r = requests.get(
            "https://api.exchangerate.host/convert",
            params={"from": from_ccy, "to": to_ccy},
            timeout=5.0,
        )
        r.raise_for_status()
        rate = r.json().get("result")
        return f"1 {from_ccy} = {rate} {to_ccy}" if rate else "rate not available"
    except requests.RequestException as e:
        return f"error: {e}"
 
class FxAnswer(dspy.Signature):
    """Answer the user's FX question, using the exchange-rate tool when needed."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField(desc="Answer in natural English")
 
agent = dspy.ReAct(FxAnswer, tools=[get_exchange_rate], max_iters=4)
 
if __name__ == "__main__":
    print(agent(question="How much is 100 USD in JPY today?").answer)

The single most important argument here is max_iters=4. Without an upper bound, a ReAct agent can keep retrying a flaky tool until your token bill makes someone unhappy. I treat this as a runtime invariant; Antigravity's Custom Rules file in my repo carries a literal "no dspy.ReAct without max_iters" rule. For the broader cost story, Halving Token Costs for AI Agents in Antigravity is required reading before you go to production.

The Metric Defines the Quality of the Optimization

Here is the part where most teams stumble. Optimizers are blind: they will move whatever direction the metric tells them to. If your metric does not reflect the business priorities, the optimizer will happily ship a "great" model that fails on every case that matters.

# eval/invoice_metric.py
from pipelines.invoice_extraction import Invoice
 
def invoice_match(example, pred, trace=None) -> float:
    """Compare ground truth against prediction; returns a score in [0.0, 1.0]
    weighted by business importance."""
    gt: Invoice = example.invoice
    pr: Invoice | None = getattr(pred, "invoice", None)
    if pr is None:
        return 0.0
 
    score = 0.0
    if gt.vendor.strip() == pr.vendor.strip():
        score += 0.3
    if gt.invoice_date == pr.invoice_date:
        score += 0.2
    # Total amount: tolerate ±0.5% drift caused by OCR digit noise.
    if pr.total_amount and abs(gt.total_amount - pr.total_amount) / max(gt.total_amount, 1) < 0.005:
        score += 0.3
    if len(gt.line_items) == len(pr.line_items):
        score += 0.1
    gt_sum = sum(li.quantity * li.unit_price for li in gt.line_items)
    pr_sum = sum(li.quantity * li.unit_price for li in pr.line_items)
    if abs(gt_sum - pr_sum) / max(gt_sum, 1) < 0.01:
        score += 0.1
    return score

In invoice extraction, a wrong vendor name annoys an accountant. A wrong total amount produces an audit finding. So total_amount should weigh more, not less. Most metric bugs I have shipped came from giving every field equal weight, then watching the optimizer happily compromise on the most important number to lift the score on lower-stakes ones.

If you are setting up an evaluation harness for the first time, Shipping an AI Agent Evaluation Framework with Antigravity covers dataset design pitfalls — temporal splits, label noise, evaluation set drift — that I won't repeat here.

Compiling: From BootstrapFewShot to MIPROv2

With a metric and labeled data in place, you can finally compile. The conventional path is BootstrapFewShot first to verify the plumbing, then MIPROv2 for serious lift.

# train/optimize_invoice.py
import json
import dspy
from dspy.teleprompt import BootstrapFewShotWithRandomSearch, MIPROv2
from pipelines.invoice_extraction import InvoiceExtractor, Invoice
from eval.invoice_metric import invoice_match
 
def load_examples(path: str):
    out = []
    with open(path) as f:
        for line in f:
            row = json.loads(line)
            out.append(
                dspy.Example(
                    ocr_text=row["ocr_text"],
                    invoice=Invoice(**row["invoice"]),
                ).with_inputs("ocr_text")
            )
    return out
 
trainset = load_examples("data/invoice_train.jsonl")
valset = load_examples("data/invoice_val.jsonl")
 
# Step 1: BootstrapFewShot for sanity
bs_optimizer = BootstrapFewShotWithRandomSearch(
    metric=invoice_match,
    max_bootstrapped_demos=4,
    max_labeled_demos=2,
    num_candidate_programs=8,
    num_threads=4,
)
bs_program = bs_optimizer.compile(InvoiceExtractor(), trainset=trainset, valset=valset)
 
# Step 2: MIPROv2 for joint instruction + few-shot optimization
mipro = MIPROv2(
    metric=invoice_match,
    auto="medium",   # light / medium / heavy
    num_threads=4,
)
optimized = mipro.compile(
    bs_program,
    trainset=trainset,
    valset=valset,
    requires_permission_to_run=False,
)
 
optimized.save("artifacts/invoice_v3.json")
print("compiled:", optimized.score)

Start with auto="light" and only escalate when the metric plateaus. The heavy setting can run an order of magnitude more LLM calls, so I gate the escalation on a CI sanity run that proves the lighter setting actually maxed out.

Debugging from Antigravity: Trace, Diff, Approve

A workflow I now refuse to live without: before saving any compiled artifact, run a side-by-side diff between the baseline module and the optimized one over a held-out sample. Commit the diff as Markdown. Future-you will thank present-you.

# scripts/diff_optimized.py
import dspy
from pipelines.invoice_extraction import InvoiceExtractor
 
baseline = InvoiceExtractor()
optimized = dspy.load("artifacts/invoice_v3.json")
 
samples = [...]  # five to ten representative OCR strings
for s in samples:
    print("==== sample ====")
    print("baseline:", baseline(ocr_text=s).model_dump())
    print("optimized:", optimized(ocr_text=s).model_dump())
 
dspy.inspect_history(n=5)

I wire this script up as a Custom Command in Antigravity called Compare DSPy programs, so reviewers can run it from the command palette. For the larger story of how to track prompt and program versions over time, the workflow described in Productionizing Prompt Versioning and A/B Testing in Antigravity extends naturally to compiled DSPy artifacts — the JSON file is just another versioned blob.

Production Deployment: Treat Compiled Programs as Artifacts

DSPy's strength in production is that the compiled program serializes to JSON. Your serving layer does not need the optimizer, the training data, or any heavyweight dependency — just dspy-ai, the JSON file, and an LLM provider.

The shape I have settled on after a few iterations:

• A nightly job runs the optimizer on a self-hosted GitHub Actions runner.
• The compiled artifacts/<module>_<git_sha>.json is uploaded to S3-compatible storage (Cloudflare R2 in my case) with a version tag.
• The serving service loads the latest version on boot and exposes the version via /healthz.
• Promotion to production is automatic only when the new version's score on a holdout set exceeds the current production score by a configurable margin. Otherwise, a Slack message gets posted and a human approves.

That last bullet is the load-bearing piece. Optimizers will always move toward the metric. The bits of "good" behavior that did not make it into the metric — tone, refusal politeness, latency on edge cases — will quietly drift unless you keep a human gate. Pair this with the failover strategy in Multi-Provider LLM Failover for Antigravity Workloads, and you can keep separate compiled artifacts per provider so a price hike or an outage doesn't take you down.

When DSPy Is the Wrong Tool

I have been recommending DSPy a lot in this article, but I want to be honest about the cases where I would not reach for it.

If you are still in the discovery phase and changing the shape of your output every week — fields appearing and disappearing, the meaning of a field shifting — you do not yet have a stable signature, and DSPy's compilation step will be churn. Stay with hand-written prompts until your output schema settles. Once it stops changing for two weeks, that is your cue to migrate.

If your task is genuinely a single-shot, no-data-available problem (a one-off email summary, a scratch translation), the optimizer has no signal to work with. There is no shame in writing the prompt yourself for those cases. DSPy shines specifically when you have repeated, evaluatable calls.

If your latency budget is below 200 ms and your task is actually trivial, the overhead of an extra reasoning step or a few-shot block can be more expensive than the accuracy gain. Profile both dspy.Predict and a hand-written prompt before you assume one will be faster.

Finally, if your team currently has zero evaluation infrastructure, do not start with DSPy. Start with the eval. The framework is downstream of the discipline. I have watched teams adopt DSPy as a way to avoid doing the eval work, and the result is the same broken loop with more YAML.

Cost Engineering: The Numbers I Actually See

I want to share rough numbers from a real workload so you can calibrate expectations. The pipeline is the invoice extractor described above, running on Gemini 2.5 Flash, with about 4,000 invoices per day in production.

Before DSPy, with a hand-tuned ChainOfThought prompt:

• Token cost per invoice: ~6.2k input + ~1.8k output = roughly $0.0014
• F1 on the holdout set: 0.81
• Time spent rewriting the prompt that month: about three engineer-days

After compiling with MIPROv2 on a 200-example training set:

• Token cost per invoice: ~3.4k input + ~0.9k output = roughly $0.0007 (the optimizer found a tighter instruction set and shorter few-shot examples)
• F1 on the holdout set: 0.89
• Time spent rewriting the prompt that month: about half an engineer-day, all of it data labeling rather than prompt wording

The cost halved, the accuracy improved, and the human time stopped being spent on prose. That is the trade I want every team I advise to be able to make.

The cost halving is the part that surprised most engineers I showed this to. The intuition is that human prompt-writers tend to over-explain — long instructions, many fluff sentences, redundant clarifications "just in case." Optimizers do not have that anxiety; they tighten until the metric drops. The result is consistently shorter prompts than what I would have written by hand.

Living With DSPy in a Multi-Agent World

Most production systems in 2026 are not single DSPy modules — they are graphs. LangGraph or Antigravity's own multi-agent runtime provides the orchestration, and DSPy modules become specialized workers inside that graph. The pattern I use most:

• The orchestrator (LangGraph state machine or Antigravity Manager Surface) decides which worker to call next based on the conversation state.
• Each worker is a DSPy Module compiled to JSON.
• Workers are stateless — they take typed input, return typed output, and let the orchestrator handle memory.
• The orchestrator's tests are integration tests; the workers' tests are evaluation runs against held-out sets.

This separation pays off when you need to change the orchestration logic without retraining workers, or vice versa. I keep the orchestrator code in agents/orchestrator/ and the DSPy workers in agents/workers/<name>/, with a single manifest.json per worker pointing at its compiled artifact. The artifact path is what you bump on promotion; the orchestrator code is unchanged.

For workers that need persistent state across calls (a long-running research agent, for example), I do not put state inside DSPy. The state lives in the orchestrator, the worker reads it as input, returns its output, and the orchestrator writes the next state. This keeps each worker's evaluation tractable — you can replay any single worker call against your eval set without dragging the whole graph along.

Common Mistakes I Have Made So You Don't Have To

1. Evaluation leakage. Splitting train and val randomly from the same source means look-alike rows live in both. The optimizer overfits to the val set; production craters. Split temporally (later invoices for val) or by entity (held-out vendors).

2. Coarse metrics. Binary all-or-nothing scoring gives the optimizer no gradient to follow. Use the partial-credit weighting shown earlier so each correct field nudges the score.

3. Forgetting max_iters on dspy.ReAct. A failing tool plus an ambitious agent equals a token bill nobody wants to explain. I cap at four in production, eight in development, and treat any code review without that cap as a blocker.

4. Treating temperature=0.0 as deterministic. It is not, especially for long outputs. Score over three to five samples per case, or enable DSPy's caching with dspy.settings.configure(rm=...) to make CI runs reproducible.

5. Loading a stale artifact after a signature change. Adding a field to your Invoice model and then loading last week's compiled JSON does not error at load time — it errors at the first request. Bump the version tag on every signature change and refuse to serve mismatched pairs at startup.

6. Deploying without observability. dspy.inspect_history produces too much in production, but disabling tracing entirely makes regressions undebuggable. I sample one in a hundred calls into OpenTelemetry. The full picture is in Designing Distributed Tracing for AI Agents in Antigravity.

A Concrete CI Setup You Can Copy

Talk is cheap, so here is the actual GitHub Actions skeleton I run for the invoice extractor. It evaluates the candidate program, posts the diff back to the pull request, and only blocks the merge when the metric drops by more than a tolerance.

# .github/workflows/dspy-eval.yml
name: dspy-eval
 
on:
  pull_request:
    paths:
      - "pipelines/**"
      - "agents/**"
      - "eval/**"
      - "data/eval/**"
 
jobs:
  evaluate:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install -r requirements.txt
      - name: Run evaluation
        env:
          GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
        run: |
          python eval/run_holdout.py --module pipelines.invoice_extraction:InvoiceExtractor \
            --artifact artifacts/invoice_latest.json \
            --data data/eval/invoice_holdout.jsonl \
            --report eval-report.md
      - name: Comment evaluation report on PR
        uses: marocchino/sticky-pull-request-comment@v2
        with:
          path: eval-report.md
      - name: Enforce non-regression
        run: python eval/check_regression.py eval-report.md --tolerance 0.01

The check_regression.py script reads the prior production score from a small JSON file in the repo (eval/baseline.json), compares it against the latest report, and exits non-zero if the new score is lower than baseline minus the tolerance. The baseline file is updated only when a maintainer manually runs the promote command after a successful production deployment.

This setup gives you three things at once: visible diffs in code review, an automatic guardrail against regressions, and a clean record of which artifact corresponds to which PR. Combined with the deployment pattern in the previous section, you end up with an evaluation-driven workflow where every prompt change is treated like a database migration — small, reviewable, reversible.

What to Do Tomorrow Morning

Pick one place in your codebase where you keep editing the same prompt, and rewrite it as a dspy.Signature. Just one. The act of declaring the input and output types — without writing any wording — exposes assumptions you didn't realize you were making, and gives you a starting point for collecting evaluation data.

DSPy is not a silver bullet; it is the scaffolding that lets you turn the eval-data flywheel into something you actually run. The first signature you write will feel awkward — you will not know what to put in the docstring, and the field descriptions will look uncomfortably terse. That feeling is normal. Treat the first version as a placeholder and let the optimizer disagree with you. The wording you would have spent two hours debating in code review becomes the optimizer's problem, and you get to spend that time on the data instead.

What I have come to believe, after a year of running this loop, is that the real product is the evaluation set. The prompts are downstream artifacts. The model is a commodity. The signature is a contract. The only piece that compounds with your time is the labeled data, because every label you collect makes future optimizations cheaper and every regression you catch makes the system safer. DSPy is the framework that finally lets that compounding happen, because the rest of the pipeline stops being hand-tuned text.

If you want to deepen your foundation while you are at it, Prompt Engineering for LLMs by John Berryman and Albert Ziegler covers the conceptual side, and Machine Learning Design Patterns by Lakshmanan, Robinson, and Munn gives you the evaluation-design language you'll need to make the optimizer's metric trustworthy. For me, using Antigravity as the greenhouse where DSPy programs grow and get tended has measurably reduced the number of Friday nights I spend wrestling prompts. I hope your next Friday turns out better than the one I started this article describing.