Antigravity × Gemini API Multimodal Complete Implementation Guide: Building Production-Ready AI Apps with Text, Images, and Audio

Setup and context: Why Multimodal AI Changes Everything for Indie Developers

In 2026, one of the biggest shifts in AI app development is the move toward multimodal systems. We've gone from processing text alone to building systems that understand images, audio, video, and PDF documents in an integrated way — and this shift creates enormous business opportunities for solo developers.

By combining Antigravity IDE with the Gemini API, you can now build multimodal AI apps that previously required entire engineering teams, and ship them to production as a solo developer in a fraction of the time.

In this guide, you'll learn through working code and production-tested design patterns:

How to integrate Gemini's multimodal capabilities (Vision, Audio, Document)
An efficient implementation workflow using Antigravity's AI agent features
Error handling and cost optimization strategies for production environments
Full implementations of practical use cases: receipt scanning, audio transcription, and document summarization

This guide assumes you're familiar with Antigravity's core features and have a working knowledge of JavaScript/TypeScript. If you want to ship AI features professionally in your next product, you're in the right place.

Understanding the Gemini Multimodal Feature Landscape

Before writing any code, it helps to understand the available models and when to use each one.

Gemini 2.5 Pro Delivers the highest accuracy for multimodal understanding. Best for complex image analysis, long-form document summarization, and nuanced audio transcription. Higher cost and latency mean it should be reserved for tasks where quality is the top priority.

Gemini 2.5 Flash Excellent balance of speed and cost. Ideal for real-time or near-real-time scenarios and high-volume batch processing. In most production environments, this is your default workhorse.

Gemini 2.0 Flash Lite The most cost-efficient option. Great for lightweight tasks like simple image classification or short text transformations.

Key multimodal use cases to keep in mind:

Image → Text: OCR, receipt scanning, product description generation, accessibility alt text
Audio → Text + Analysis: Meeting transcription, sentiment analysis, summarization
PDF/Document → Structured Data: Invoice parsing, contract review, research paper summaries
Video → Insights: Highlight extraction from product demos, educational content summarization
Combined inputs: "Analyze this image and this audio note, then generate a combined report"

Antigravity Environment Setup and API Configuration

Project Structure

Start by creating a new project in Antigravity and installing the required dependencies.

# Run in Antigravity's terminal
npm create next-app@latest multimodal-ai-app -- --typescript --tailwind --app
cd multimodal-ai-app
npm install @google/generative-ai @google/genai
npm install zod react-dropzone

Environment Variables

Create .env.local in the project root. To prevent Antigravity's AI agent from accidentally hardcoding keys, explicitly instruct it in your project rules.

# .env.local
GEMINI_API_KEY=YOUR_GEMINI_API_KEY
NEXT_PUBLIC_MAX_FILE_SIZE=10485760  # 10MB

Add the following to your .antigravity/rules.md project rules file:

# Security Rules
- Never write API keys or secrets directly in code. Always use process.env
- Never commit files containing real credentials
- Never reference GEMINI_API_KEY on the client side — server-side only

Initializing the Gemini Client

Create src/lib/gemini.ts:

import { GoogleGenerativeAI, HarmCategory, HarmBlockThreshold } from "@google/generative-ai";
 
if (!process.env.GEMINI_API_KEY) {
  throw new Error("GEMINI_API_KEY is not set in environment variables");
}
 
// Lazily initialize and cache model instances
const clients = new Map<string, ReturnType<GoogleGenerativeAI["getGenerativeModel"]>>();
const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);
 
// Production-grade safety settings
const safetySettings = [
  {
    category: HarmCategory.HARM_CATEGORY_HARASSMENT,
    threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
  },
  {
    category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,
    threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
  },
];
 
export function getGeminiModel(modelName: "gemini-2.5-pro" | "gemini-2.5-flash" | "gemini-2.0-flash-lite") {
  if (!clients.has(modelName)) {
    clients.set(
      modelName,
      genAI.getGenerativeModel({ model: modelName, safetySettings })
    );
  }
  return clients.get(modelName)!;
}
 
// Convert a file buffer into the format Gemini expects
export function fileToGenerativePart(data: ArrayBuffer, mimeType: string) {
  return {
    inlineData: {
      data: Buffer.from(data).toString("base64"),
      mimeType,
    },
  };
}

Building a Receipt Scanning API (Image to Text Pattern)

Let's implement a concrete, practical example: an API that extracts structured data from a receipt image.

Receipt Analysis API Route

// src/app/api/analyze-receipt/route.ts
import { NextRequest, NextResponse } from "next/server";
import { z } from "zod";
import { getGeminiModel, fileToGenerativePart } from "@/lib/gemini";
 
const ReceiptSchema = z.object({
  storeName: z.string(),
  date: z.string(),
  totalAmount: z.number(),
  currency: z.string().default("USD"),
  items: z.array(
    z.object({
      name: z.string(),
      quantity: z.number().optional(),
      unitPrice: z.number().optional(),
      totalPrice: z.number(),
    })
  ),
  taxAmount: z.number().optional(),
  paymentMethod: z.string().optional(),
});
 
export type ReceiptData = z.infer<typeof ReceiptSchema>;
 
const EXTRACTION_PROMPT = `
This image contains a receipt. Extract the following information and return it as valid JSON.
 
{
  "storeName": "store or business name",
  "date": "date in YYYY-MM-DD format",
  "totalAmount": total amount as a number only,
  "currency": "currency code (e.g. USD, JPY, EUR)",
  "items": [
    {
      "name": "item name",
      "quantity": quantity as a number (omit if unclear),
      "unitPrice": unit price as a number (omit if unclear),
      "totalPrice": line total as a number
    }
  ],
  "taxAmount": tax amount as a number (omit if not shown),
  "paymentMethod": "payment method (omit if not shown)"
}
 
Return numbers as digits only — no currency symbols or commas.
If the date is illegible, use "unknown".
`;
 
export async function POST(req: NextRequest) {
  try {
    const MAX_SIZE = 10 * 1024 * 1024;
 
    const formData = await req.formData();
    const file = formData.get("file") as File | null;
 
    if (!file) {
      return NextResponse.json({ error: "No file provided" }, { status: 400 });
    }
 
    if (file.size > MAX_SIZE) {
      return NextResponse.json(
        { error: "File size exceeds 10MB limit" },
        { status: 413 }
      );
    }
 
    const allowedTypes = ["image/jpeg", "image/png", "image/webp", "image/heic"];
    if (!allowedTypes.includes(file.type)) {
      return NextResponse.json(
        { error: "Only JPEG, PNG, WebP, and HEIC are supported" },
        { status: 415 }
      );
    }
 
    const arrayBuffer = await file.arrayBuffer();
    const imagePart = fileToGenerativePart(arrayBuffer, file.type);
 
    const model = getGeminiModel("gemini-2.5-flash");
    const result = await model.generateContent([EXTRACTION_PROMPT, imagePart]);
    const responseText = result.response.text();
 
    const jsonMatch = responseText.match(/\{[\s\S]*\}/);
    if (!jsonMatch) {
      throw new Error("Failed to extract JSON from response");
    }
 
    const parsed = JSON.parse(jsonMatch[0]);
    const validated = ReceiptSchema.parse(parsed);
 
    return NextResponse.json({ success: true, data: validated });
  } catch (error) {
    console.error("Receipt analysis error:", error);
 
    if (error instanceof z.ZodError) {
      return NextResponse.json(
        { error: "Invalid data format", details: error.errors },
        { status: 422 }
      );
    }
 
    return NextResponse.json(
      { error: "Analysis failed. Please try again." },
      { status: 500 }
    );
  }
}

Debugging Efficiently with Antigravity

Use Antigravity's inline chat (Cmd+I) with prompts like this to improve your code:

Review this API route's error handling and add proper retry logic
for Gemini API rate limit errors (429) and network failures,
using exponential backoff with jitter.

Document and PDF Analysis Pipeline

PDF and document processing is one of the highest-value capabilities for business-facing applications.

Document Analyzer Service

// src/lib/document-analyzer.ts
import { getGeminiModel, fileToGenerativePart } from "./gemini";
 
export interface DocumentAnalysisOptions {
  extractType: "summary" | "key-points" | "qa" | "structured";
  language?: "ja" | "en";
}
 
export interface DocumentAnalysisResult {
  title?: string;
  summary?: string;
  keyPoints?: string[];
  structuredData?: Record<string, unknown>;
  processingTimeMs: number;
}
 
const ANALYSIS_PROMPTS: Record<DocumentAnalysisOptions["extractType"], string> = {
  summary: `
Read this document and return the following in JSON:
1. title or main subject (one line)
2. summary (under 150 words)
3. key points (3 to 5 bullet points)
 
JSON format:
{"title": "...", "summary": "...", "keyPoints": ["...", "..."]}
`,
  "key-points": `
Extract 5 to 10 of the most important pieces of information from this document.
Return as JSON: {"keyPoints": ["...", "..."]}
`,
  qa: `
Generate 5 Q&A pairs based on the content of this document.
Return as JSON: {"qa": [{"q": "...", "a": "..."}, ...]}
`,
  structured: `
Extract all structured data from this document (tables, lists, numbers, dates, etc.).
Return as clean JSON.
`,
};
 
export async function analyzeDocument(
  fileData: ArrayBuffer,
  mimeType: string,
  options: DocumentAnalysisOptions
): Promise<DocumentAnalysisResult> {
  const startTime = Date.now();
 
  const modelName =
    mimeType === "application/pdf" ? "gemini-2.5-pro" : "gemini-2.5-flash";
 
  const model = getGeminiModel(modelName);
  const filePart = fileToGenerativePart(fileData, mimeType);
  const prompt = ANALYSIS_PROMPTS[options.extractType];
 
  try {
    const result = await model.generateContent([prompt, filePart]);
    const text = result.response.text();
 
    const jsonMatch = text.match(/\{[\s\S]*\}/);
    const data = jsonMatch ? JSON.parse(jsonMatch[0]) : {};
 
    return {
      title: data.title,
      summary: data.summary,
      keyPoints: data.keyPoints,
      structuredData: options.extractType === "structured" ? data : undefined,
      processingTimeMs: Date.now() - startTime,
    };
  } catch (error) {
    throw new Error(
      `Document analysis failed: ${error instanceof Error ? error.message : "Unknown error"}`
    );
  }
}

Supported Formats and Limits

Formats Gemini handles natively:

Images: JPEG, PNG, WebP, HEIC/HEIF (up to 20MB)
Video: MP4, MOV, AVI, MKV (up to 2GB via File API)
Audio: WAV, MP3, AAC, OGG, FLAC (up to 20MB)
Documents: PDF (up to 1,000 pages, 100MB)

Audio Transcription API

Audio-to-Text API Route

// src/app/api/transcribe/route.ts
import { NextRequest, NextResponse } from "next/server";
import { getGeminiModel, fileToGenerativePart } from "@/lib/gemini";
 
interface TranscriptionResult {
  text: string;
  language?: string;
  confidence?: "high" | "medium" | "low";
}
 
export async function POST(req: NextRequest) {
  try {
    const formData = await req.formData();
    const audioFile = formData.get("audio") as File | null;
    const lang = (formData.get("lang") as string) || "auto";
 
    if (!audioFile) {
      return NextResponse.json({ error: "Audio file is required" }, { status: 400 });
    }
 
    const allowedTypes = [
      "audio/wav", "audio/mpeg", "audio/mp4",
      "audio/aac", "audio/ogg", "audio/flac",
    ];
 
    if (!allowedTypes.includes(audioFile.type)) {
      return NextResponse.json({ error: "Unsupported audio format" }, { status: 415 });
    }
 
    const arrayBuffer = await audioFile.arrayBuffer();
    const audioPart = fileToGenerativePart(arrayBuffer, audioFile.type);
 
    const langInstruction =
      lang === "auto"
        ? "Automatically detect the language and transcribe accordingly."
        : `The language is ${lang}.`;
 
    const prompt = `
${langInstruction}
Transcribe this audio file and return the result as JSON:
 
{
  "text": "full transcription text",
  "language": "detected language code (e.g. en, ja)",
  "confidence": "high/medium/low"
}
 
Add appropriate punctuation. If speakers change, separate with a newline.
Mark inaudible sections as [inaudible].
`;
 
    const model = getGeminiModel("gemini-2.5-flash");
    const result = await model.generateContent([prompt, audioPart]);
    const text = result.response.text();
 
    const jsonMatch = text.match(/\{[\s\S]*\}/);
    const data: TranscriptionResult = jsonMatch
      ? JSON.parse(jsonMatch[0])
      : { text, confidence: "low" };
 
    return NextResponse.json({ success: true, data });
  } catch (error) {
    console.error("Transcription error:", error);
    return NextResponse.json({ error: "Transcription failed" }, { status: 500 });
  }
}

Building a Multimodal Integration Pipeline

The real power of multimodal AI emerges when you combine multiple modalities together. Here's a production-ready meeting minutes generator that processes audio recordings alongside slide images to automatically produce structured meeting notes.

Meeting Minutes Pipeline

// src/lib/meeting-pipeline.ts
import { GoogleGenerativeAI } from "@google/generative-ai";
import { fileToGenerativePart } from "./gemini";
 
interface MeetingInput {
  audioData: ArrayBuffer;
  audioMimeType: string;
  slideImages?: Array<{ data: ArrayBuffer; mimeType: string }>;
  meetingTitle: string;
  attendees?: string[];
}
 
interface MeetingMinutes {
  title: string;
  date: string;
  attendees: string[];
  agenda: string[];
  discussions: Array<{
    topic: string;
    content: string;
    decisions: string[];
    actionItems: string[];
  }>;
  nextSteps: string[];
  generatedAt: string;
}
 
export async function generateMeetingMinutes(input: MeetingInput): Promise<MeetingMinutes> {
  const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY!);
  const model = genAI.getGenerativeModel({ model: "gemini-2.5-pro" });
 
  const parts: any[] = [];
 
  parts.push(fileToGenerativePart(input.audioData, input.audioMimeType));
 
  if (input.slideImages?.length) {
    for (const slide of input.slideImages) {
      parts.push(fileToGenerativePart(slide.data, slide.mimeType));
    }
  }
 
  const attendeesStr = input.attendees?.join(", ") || "Unknown";
 
  parts.push(`
Meeting title: ${input.meetingTitle}
Attendees: ${attendeesStr}
Today's date: ${new Date().toISOString().split("T")[0]}
 
Analyze the audio recording${input.slideImages?.length ? " and slide images" : ""} above
and produce detailed meeting minutes in the following JSON format:
 
{
  "title": "meeting title",
  "date": "YYYY-MM-DD",
  "attendees": ["name1", "name2"],
  "agenda": ["item1", "item2"],
  "discussions": [
    {
      "topic": "discussion topic",
      "content": "summary under 100 words",
      "decisions": ["decision1", "decision2"],
      "actionItems": ["Action: Owner to Due date"]
    }
  ],
  "nextSteps": ["step1", "step2"],
  "generatedAt": "ISO 8601 timestamp"
}
`);
 
  const result = await model.generateContent(parts);
  const text = result.response.text();
 
  const jsonMatch = text.match(/\{[\s\S]*\}/);
  if (!jsonMatch) throw new Error("Failed to generate meeting minutes");
 
  return JSON.parse(jsonMatch[0]) as MeetingMinutes;
}

Streaming Responses for Large Files

For large files or long documents, streaming significantly improves user experience by eliminating the blank loading screen.

// src/app/api/analyze-stream/route.ts
import { NextRequest } from "next/server";
import { GoogleGenerativeAI } from "@google/generative-ai";
import { fileToGenerativePart } from "@/lib/gemini";
 
export async function POST(req: NextRequest) {
  const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY!);
  const model = genAI.getGenerativeModel({ model: "gemini-2.5-flash" });
 
  const formData = await req.formData();
  const file = formData.get("file") as File;
  const prompt = (formData.get("prompt") as string) || "Analyze this file.";
 
  const arrayBuffer = await file.arrayBuffer();
  const filePart = fileToGenerativePart(arrayBuffer, file.type);
 
  const stream = new ReadableStream({
    async start(controller) {
      try {
        const result = await model.generateContentStream([prompt, filePart]);
 
        for await (const chunk of result.stream) {
          const text = chunk.text();
          if (text) {
            controller.enqueue(
              new TextEncoder().encode(`data: ${JSON.stringify({ text })}\n\n`)
            );
          }
        }
 
        controller.enqueue(
          new TextEncoder().encode(`data: ${JSON.stringify({ done: true })}\n\n`)
        );
        controller.close();
      } catch (error) {
        controller.error(error);
      }
    },
  });
 
  return new Response(stream, {
    headers: {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
      Connection: "keep-alive",
    },
  });
}

Cost Optimization and Performance Tuning for Production

Managing API costs is critical when running multimodal AI in production.

Model Routing Strategy

Routing requests to the right model based on task complexity can cut your API bill by 50 to 70%.

// src/lib/model-router.ts
interface RoutingDecision {
  model: "gemini-2.0-flash-lite" | "gemini-2.5-flash" | "gemini-2.5-pro";
  reasoning: string;
}
 
export function routeToModel(
  task: string,
  fileType: string,
  fileSizeMB: number
): RoutingDecision {
  if (fileType === "application/pdf" && fileSizeMB > 5) {
    return { model: "gemini-2.5-pro", reasoning: "Large PDF requires high-accuracy model" };
  }
 
  if (
    fileType.startsWith("image/") &&
    fileSizeMB < 1 &&
    (task.includes("classify") || task.includes("detect") || task.includes("yes/no"))
  ) {
    return { model: "gemini-2.0-flash-lite", reasoning: "Simple classification — minimize cost" };
  }
 
  return { model: "gemini-2.5-flash", reasoning: "Standard multimodal task" };
}

Response Caching

Cache identical file analyses to avoid redundant API calls.

// src/lib/analysis-cache.ts
import crypto from "crypto";
 
const cache = new Map<string, { result: unknown; expiresAt: number }>();
const CACHE_TTL_MS = 60 * 60 * 1000; // 1 hour
 
export function getCacheKey(fileData: ArrayBuffer, prompt: string): string {
  return crypto
    .createHash("sha256")
    .update(Buffer.from(fileData))
    .update(prompt)
    .digest("hex")
    .substring(0, 32);
}
 
export function getCached<T>(key: string): T | null {
  const entry = cache.get(key);
  if (!entry) return null;
  if (Date.now() > entry.expiresAt) { cache.delete(key); return null; }
  return entry.result as T;
}
 
export function setCached<T>(key: string, result: T): void {
  cache.set(key, { result, expiresAt: Date.now() + CACHE_TTL_MS });
}

Retry Logic with Exponential Backoff

// src/lib/retry.ts
interface RetryOptions {
  maxRetries?: number;
  initialDelayMs?: number;
  maxDelayMs?: number;
}
 
export async function withRetry<T>(fn: () => Promise<T>, options: RetryOptions = {}): Promise<T> {
  const { maxRetries = 3, initialDelayMs = 1000, maxDelayMs = 10000 } = options;
  let lastError: Error;
 
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      lastError = error as Error;
 
      if (error instanceof Error) {
        if (error.message.includes("INVALID_ARGUMENT") || error.message.includes("PERMISSION_DENIED")) {
          throw error;
        }
      }
 
      if (attempt < maxRetries - 1) {
        const delay = Math.min(
          initialDelayMs * Math.pow(2, attempt) + Math.random() * 1000,
          maxDelayMs
        );
        await new Promise((resolve) => setTimeout(resolve, delay));
      }
    }
  }
 
  throw lastError!;
}

Building a React Frontend for File Upload

Now that the API routes are in place, let's build the client-side component that ties everything together. This React component handles drag-and-drop file uploads, shows real-time progress, and displays the AI analysis results.

Multimodal Upload Component

// src/components/MultimodalUploader.tsx
"use client";
 
import { useState, useCallback } from "react";
import { useDropzone } from "react-dropzone";
 
type AnalysisMode = "receipt" | "document" | "audio";
 
interface AnalysisResult {
  success: boolean;
  data?: unknown;
  error?: string;
}
 
interface Props {
  mode: AnalysisMode;
}
 
const MODE_CONFIG: Record<AnalysisMode, {
  accept: Record<string, string[]>;
  label: string;
  endpoint: string;
  fileKey: string;
}> = {
  receipt: {
    accept: { "image/*": [".jpg", ".jpeg", ".png", ".webp", ".heic"] },
    label: "Drop a receipt image here",
    endpoint: "/api/analyze-receipt",
    fileKey: "file",
  },
  document: {
    accept: {
      "application/pdf": [".pdf"],
      "image/*": [".jpg", ".jpeg", ".png"],
    },
    label: "Drop a PDF or document image here",
    endpoint: "/api/analyze-document",
    fileKey: "file",
  },
  audio: {
    accept: {
      "audio/*": [".mp3", ".wav", ".aac", ".ogg", ".flac"],
    },
    label: "Drop an audio file here",
    endpoint: "/api/transcribe",
    fileKey: "audio",
  },
};
 
export function MultimodalUploader({ mode }: Props) {
  const [isProcessing, setIsProcessing] = useState(false);
  const [result, setResult] = useState<AnalysisResult | null>(null);
  const [fileName, setFileName] = useState<string | null>(null);
 
  const config = MODE_CONFIG[mode];
 
  const onDrop = useCallback(
    async (acceptedFiles: File[]) => {
      const file = acceptedFiles[0];
      if (\!file) return;
 
      setFileName(file.name);
      setIsProcessing(true);
      setResult(null);
 
      try {
        const formData = new FormData();
        formData.append(config.fileKey, file);
 
        const response = await fetch(config.endpoint, {
          method: "POST",
          body: formData,
        });
 
        const data = await response.json();
 
        if (\!response.ok) {
          setResult({ success: false, error: data.error || "Request failed" });
        } else {
          setResult({ success: true, data: data.data });
        }
      } catch (err) {
        setResult({
          success: false,
          error: err instanceof Error ? err.message : "Network error",
        });
      } finally {
        setIsProcessing(false);
      }
    },
    [config]
  );
 
  const { getRootProps, getInputProps, isDragActive } = useDropzone({
    onDrop,
    accept: config.accept,
    maxFiles: 1,
    maxSize: 10 * 1024 * 1024, // 10MB
    disabled: isProcessing,
  });
 
  return (
    <div className="w-full max-w-2xl mx-auto space-y-4">
      {/* Drop zone */}
      <div
        {...getRootProps()}
        className={[
          "border-2 border-dashed rounded-xl p-10 text-center cursor-pointer transition-colors",
          isDragActive ? "border-blue-500 bg-blue-50" : "border-gray-300 hover:border-gray-400",
          isProcessing ? "opacity-50 cursor-not-allowed" : "",
        ].join(" ")}
      >
        <input {...getInputProps()} />
        {isProcessing ? (
          <p className="text-gray-600 animate-pulse">Analyzing with Gemini AI...</p>
        ) : isDragActive ? (
          <p className="text-blue-600 font-medium">Drop to analyze</p>
        ) : (
          <div>
            <p className="text-gray-700 font-medium">{config.label}</p>
            <p className="text-sm text-gray-400 mt-1">or click to browse — max 10MB</p>
          </div>
        )}
      </div>
 
      {/* Results display */}
      {result && (
        <div
          className={[
            "rounded-xl p-5 border",
            result.success
              ? "bg-green-50 border-green-200"
              : "bg-red-50 border-red-200",
          ].join(" ")}
        >
          {result.success ? (
            <div>
              <p className="text-sm font-medium text-green-800 mb-3">
                Analysis complete — {fileName}
              </p>
              <pre className="text-xs text-gray-700 overflow-auto max-h-64 bg-white rounded p-3">
                {JSON.stringify(result.data, null, 2)}
              </pre>
            </div>
          ) : (
            <p className="text-sm text-red-700">{result.error}</p>
          )}
        </div>
      )}
    </div>
  );
}

Integrating the Component into a Page

// src/app/tools/receipt-scanner/page.tsx
import { MultimodalUploader } from "@/components/MultimodalUploader";
 
export const metadata = {
  title: "AI Receipt Scanner",
  description: "Extract structured data from receipt images using Gemini AI",
};
 
export default function ReceiptScannerPage() {
  return (
    <main className="min-h-screen bg-gray-50 py-12 px-4">
      <div className="max-w-3xl mx-auto">
        <h1 className="text-2xl font-bold text-gray-900 mb-2">
          AI Receipt Scanner
        </h1>
        <p className="text-gray-500 mb-8">
          Upload a receipt image and get structured data extracted instantly using Gemini AI.
        </p>
        <MultimodalUploader mode="receipt" />
      </div>
    </main>
  );
}

Displaying Results in a Structured Format

For a polished user experience, you'll want to render the extracted receipt data in a readable layout rather than raw JSON.

// src/components/ReceiptDisplay.tsx
"use client";
 
import type { ReceiptData } from "@/app/api/analyze-receipt/route";
 
export function ReceiptDisplay({ data }: { data: ReceiptData }) {
  return (
    <div className="bg-white rounded-xl border border-gray-200 overflow-hidden">
      {/* Header */}
      <div className="px-5 py-4 border-b border-gray-100">
        <h3 className="font-semibold text-gray-900">{data.storeName}</h3>
        <p className="text-sm text-gray-500">{data.date}</p>
      </div>
 
      {/* Items */}
      <ul className="divide-y divide-gray-50">
        {data.items.map((item, i) => (
          <li key={i} className="px-5 py-3 flex justify-between text-sm">
            <span className="text-gray-700">{item.name}</span>
            <span className="text-gray-900 font-medium tabular-nums">
              {data.currency} {item.totalPrice.toLocaleString()}
            </span>
          </li>
        ))}
      </ul>
 
      {/* Totals */}
      <div className="px-5 py-4 bg-gray-50 space-y-1">
        {data.taxAmount \!== undefined && (
          <div className="flex justify-between text-sm text-gray-500">
            <span>Tax</span>
            <span>{data.currency} {data.taxAmount.toLocaleString()}</span>
          </div>
        )}
        <div className="flex justify-between font-semibold text-gray-900">
          <span>Total</span>
          <span>{data.currency} {data.totalAmount.toLocaleString()}</span>
        </div>
        {data.paymentMethod && (
          <p className="text-xs text-gray-400 pt-1">{data.paymentMethod}</p>
        )}
      </div>
    </div>
  );
}

With Antigravity's AI agent, you can generate this entire component setup by describing what you need in the chat: "Create a React component with drag-and-drop file upload that calls /api/analyze-receipt and displays the result as a formatted receipt." The agent will scaffold the full implementation, which you can then refine with targeted follow-up prompts.

Summary

This guide covered a complete implementation approach for multimodal AI apps using Antigravity and the Gemini API. The key takeaways are these: model selection is the single biggest lever for cost control, routing tasks to Lite or Flash instead of Pro can cut your bill by up to 70%; Zod-based schema validation gives you type-safe, predictable AI responses that are safe to ship; streaming responses remove the anxiety of long loading screens when processing large files; and every production multimodal app needs both caching and retry logic — caching reduces unnecessary spend, while retry logic provides the reliability users expect.

Multimodal AI is one of the fastest-evolving areas in 2026 and beyond. Combined with Antigravity's AI-assisted development environment, solo developers can now ship production-quality multimodal apps in days rather than months. Start with one use case — receipt scanning, audio transcription, or document summarization — and build your intuition from there.

For deeper Gemini ecosystem coverage, see our guides on Gemini API Custom AI Tools and the Gemini API Advanced Integration Masterclass.