Antigravity × Cloudflare Vectorize: Build a Production RAG Pipeline That Runs at the Edge

Ingestion Pipeline: Chunking, Embeddings, and Metadata

The minimal version works, but it's not production. The single biggest factor in retrieval quality is chunking strategy.

I started with fixed 1,000-character chunks. The result: chunks that broke mid-heading, with "title-only fragments" frequently ranking near the top. After much iteration, I settled on a Markdown-aware chunker that respects heading hierarchy and aims for 600–900 chars per chunk on paragraph boundaries.

A Markdown-Aware Chunker

// src/lib/chunker.ts
type Section = { heading: string; level: number; body: string };
 
export function parseMarkdown(md: string): Section[] {
  const lines = md.split("\n");
  const sections: Section[] = [];
  let current: Section = { heading: "", level: 1, body: "" };
 
  for (const line of lines) {
    const headingMatch = line.match(/^(#{1,6})\s+(.+)$/);
    if (headingMatch) {
      if (current.body.trim() || current.heading) sections.push(current);
      current = {
        heading: headingMatch[2].trim(),
        level: headingMatch[1].length,
        body: "",
      };
    } else {
      current.body += line + "\n";
    }
  }
  if (current.body.trim() || current.heading) sections.push(current);
  return sections;
}
 
export function chunkSections(
  sections: Section[],
  target = 800,
  overlap = 100
): { text: string; heading: string }[] {
  const out: { text: string; heading: string }[] = [];
 
  for (const section of sections) {
    const fullText = section.heading
      ? `## ${section.heading}\n${section.body.trim()}`
      : section.body.trim();
 
    if (fullText.length <= target) {
      out.push({ text: fullText, heading: section.heading });
      continue;
    }
 
    const paragraphs = fullText.split(/\n{2,}/);
    let buf = "";
    for (const p of paragraphs) {
      if ((buf + "\n\n" + p).length > target && buf) {
        out.push({ text: buf.trim(), heading: section.heading });
        const tail = buf.slice(Math.max(0, buf.length - overlap));
        buf = tail + "\n\n" + p;
      } else {
        buf = buf ? `${buf}\n\n${p}` : p;
      }
    }
    if (buf.trim()) out.push({ text: buf.trim(), heading: section.heading });
  }
  return out;
}

Switching to this chunker boosted my site search click-through (the rate at which a search result actually got clicked) by about 1.6×. The reason is unglamorous: keeping the heading at the top of each chunk preserves enough surrounding context for the embedding to reflect what the section is actually about.

Choosing an Embedding Model

The main embedding models available on Workers AI:

• @cf/baai/bge-small-en-v1.5 (384 dim, English)
• @cf/baai/bge-base-en-v1.5 (768 dim, English)
• @cf/baai/bge-large-en-v1.5 (1024 dim, English)
• @cf/baai/bge-m3 (1024 dim, multilingual)

If you're indexing Japanese content, bge-m3 is the only sensible choice. I lost a full day in my early days trying to debug poor retrieval on Japanese queries before realizing I was running an English-only model. Lesson: check the language coverage of your embedding model first.

That said, bge-m3 is 1024-dim, which directly increases Vectorize storage costs and query latency. For predominantly English content, bge-base-en-v1.5 (768-dim) is more economical.

Metadata Design You'll Be Glad You Made

The metadata you attach at upsert time becomes the lever you pull later for filtering. The fields I wish I'd added from day one:

• docId: stable ID of the source document (essential for updates and deletes)
• title: for display
• category / tag: for filtered search
• lang: to scope queries by language
• updatedAt (unix timestamp): for re-indexing decisions
• text: the chunk body itself (Vectorize doesn't return body text by default, so you must store it)

You must create metadata indexes ahead of time with wrangler vectorize create-metadata-index. Adding them later doesn't backfill existing vectors, so be exhaustive before going to production.

Lifting Search Quality to Production: Hybrid Search and Reranking

Pure vector search struggles with proper nouns and version numbers. A query like "Antigravity 1.20" embeds near a lot of vaguely-related content, and unrelated articles bubble up.

I solved this with hybrid search (vector + full-text) + reranking in two stages.

Hybrid Search with D1 as a Full-Text Index

// src/lib/hybrid-search.ts
type HybridResult = {
  id: string;
  score: number;
  source: "vector" | "fts" | "both";
  text: string;
  title: string;
};
 
export async function hybridSearch(
  env: { VECTORIZE_INDEX: VectorizeIndex; DB: D1Database; AI: Ai },
  query: string,
  topK = 10
): Promise<HybridResult[]> {
  // Run vector and FTS in parallel to minimize wall-clock
  const [vectorRes, ftsRes] = await Promise.all([
    (async () => {
      const { data } = await env.AI.run("@cf/baai/bge-m3", { text: [query] });
      return env.VECTORIZE_INDEX.query(data[0], {
        topK,
        returnMetadata: "all",
      });
    })(),
    env.DB.prepare(
      `SELECT id, title, text, rank
       FROM chunks_fts
       WHERE chunks_fts MATCH ?1
       ORDER BY rank LIMIT ?2`
    )
      .bind(query, topK)
      .all<{ id: string; title: string; text: string; rank: number }>(),
  ]);
 
  // Reciprocal Rank Fusion (RRF) blends the two
  const map = new Map<string, HybridResult>();
  vectorRes.matches.forEach((m, i) => {
    map.set(m.id, {
      id: m.id,
      score: 1 / (60 + i),
      source: "vector",
      text: String(m.metadata?.text ?? ""),
      title: String(m.metadata?.title ?? ""),
    });
  });
  (ftsRes.results ?? []).forEach((r, i) => {
    const existing = map.get(r.id);
    const ftsScore = 1 / (60 + i);
    if (existing) {
      existing.score += ftsScore;
      existing.source = "both";
    } else {
      map.set(r.id, {
        id: r.id,
        score: ftsScore,
        source: "fts",
        text: r.text,
        title: r.title,
      });
    }
  });
 
  return Array.from(map.values()).sort((a, b) => b.score - a.score);
}

On the D1 side, set up an FTS5 virtual table:

-- migrations/0001_chunks_fts.sql
CREATE VIRTUAL TABLE chunks_fts USING fts5(
  id UNINDEXED,
  title,
  text,
  tokenize='unicode61 remove_diacritics 2'
);

Reciprocal Rank Fusion is simple but powerful. It merges results from vector search and FTS using only their ranks, so you don't have to worry about score normalization between two scoring systems with very different magnitudes.

Re-Rank the Top N with Workers AI Reranker

Pull a wider net (top 20) with hybrid search, then have a reranker decide which are actually most relevant.

// src/lib/rerank.ts
export async function rerank(
  env: { AI: Ai },
  query: string,
  candidates: { id: string; text: string }[]
): Promise<{ id: string; score: number }[]> {
  const result = await env.AI.run("@cf/baai/bge-reranker-base", {
    query,
    contexts: candidates.map((c) => ({ text: c.text })),
  });
 
  return result.response
    .map((r: { index: number; score: number }) => ({
      id: candidates[r.index].id,
      score: r.score,
    }))
    .sort((a, b) => b.score - a.score);
}

Adding a reranker visibly improved the factual quality of LLM answers in my system. Before reranking, the LLM occasionally "filled in" facts that weren't in the documents at all. After reranking, those hallucinations almost vanished.

Cost Design: Make Tokens, Calls, and Storage Visible

Vectorize cost has three axes:

• Queries (vector × topK)
• Stored vectors (GB/month)
• Workers AI calls (embeddings, LLM, and reranker each metered separately)

For a small SaaS scale (10K queries/month, 50K stored chunks), my bill stays under about $7/month. But if you're not careful, Workers AI tokens are where bloat creeps in.

Logging Usage to Make Cost Visible

// src/lib/cost-logger.ts
type UsageEvent = {
  type: "embedding" | "llm" | "rerank" | "vectorize_query";
  inputTokens?: number;
  outputTokens?: number;
  vectorCount?: number;
  durationMs: number;
  ts: number;
};
 
export class UsageLogger {
  private events: UsageEvent[] = [];
  constructor(private analytics: AnalyticsEngineDataset) {}
 
  log(event: UsageEvent) {
    this.events.push(event);
    this.analytics.writeDataPoint({
      indexes: [event.type],
      blobs: [JSON.stringify(event)],
      doubles: [event.durationMs, event.inputTokens ?? 0],
    });
  }
 
  summary(): Record<string, { calls: number; tokens: number; ms: number }> {
    return this.events.reduce((acc, e) => {
      const k = e.type;
      acc[k] ??= { calls: 0, tokens: 0, ms: 0 };
      acc[k].calls += 1;
      acc[k].tokens += (e.inputTokens ?? 0) + (e.outputTokens ?? 0);
      acc[k].ms += e.durationMs;
      return acc;
    }, {} as Record<string, { calls: number; tokens: number; ms: number }>);
  }
}

Analytics Engine is Cloudflare's pay-as-you-go log fabric, free for hundreds of thousands of events per month in practice. I thread UsageLogger through every endpoint and surface a usage summary in the response headers during development.

Three Cost Wins That Actually Mattered

The optimizations that moved the needle in my deployment:

1. Cache layer: Hash the query, look it up in KV before doing anything else. FAQ-shaped questions hit cache 70%+ of the time
2. Early return on weak matches: If the top vector score is below a threshold (I use 0.55), don't even call the LLM
3. Model downgrade: Route easy questions to llama-3.2-3b-instruct instead of llama-3.1-8b-instruct

The cache alone cut Workers AI costs by about 40%. The implementation is trivial:

// src/lib/query-cache.ts
import { createHash } from "node:crypto";
 
export async function getCachedAnswer(
  kv: KVNamespace,
  query: string
): Promise<string | null> {
  const key = `q:${createHash("sha256").update(query).digest("hex").slice(0, 32)}`;
  return await kv.get(key);
}
 
export async function setCachedAnswer(
  kv: KVNamespace,
  query: string,
  answer: string,
  ttlSec = 86400
) {
  const key = `q:${createHash("sha256").update(query).digest("hex").slice(0, 32)}`;
  await kv.put(key, answer, { expirationTtl: ttlSec });
}

Observability: Logs, Metrics, and Surfacing Failure Modes

What matters most in production is being able to identify the cause when accuracy drops, costs spike, or latency creeps up. If you want to keep everything inside Cloudflare, this is the stack I recommend:

• Workers Logs (built-in): error stack traces, cold-start timing
• Analytics Engine: custom metrics (query rate, average latency, reranker score distribution)
• Tail Workers: forward failed requests to a separate Worker for long-term storage and Slack alerts

Monitoring the distribution of reranker scores is what lets you catch quality regressions early. I once bulk-updated some articles and the chunk boundaries shifted; the reranker's top score average fell from ~0.6 to ~0.3 overnight. Without dashboards I'd have only learned about it from a user complaint days later.

// src/lib/health-check.ts
export async function searchHealthCheck(env: Bindings) {
  const goldenQueries = [
    { q: "How do I get started with Antigravity?", expectedDocId: "antigravity-getting-started" },
    { q: "What does Cloudflare Vectorize cost?", expectedDocId: "antigravity-cloudflare-vectorize-rag-production-guide" },
  ];
 
  const failures: { q: string; got: string[] }[] = [];
  for (const { q, expectedDocId } of goldenQueries) {
    const results = await hybridSearch(env, q, 5);
    const ids = results.map((r) => r.id.split("#")[0]);
    if (!ids.includes(expectedDocId)) failures.push({ q, got: ids });
  }
  return { passed: failures.length === 0, failures };
}

Run this from a Cron Trigger every hour and post failures to Slack. You'll catch regressions long before users do.

Common Pitfalls

You can read this entire article, feel prepared, and still trip on something. Here are the traps I (and other indie developers I know) have actually fallen into, with fixes.

1. Forgetting to Create Metadata Indexes

returnMetadata: "all" lets you read metadata from results, but using metadata in a filter clause requires that you create a metadata index up front. I missed this initially: my "filtered" queries appeared to ignore the filter entirely. Half a day was lost to it. The Vectorize docs are explicit, but it's an easy thing to glide past.

# Without this, filter: { docId: "xxx" } won't work
wrangler vectorize create-metadata-index blog-knowledge \
  --property-name=docId --type=string

2. Switching Embedding Models Mid-Index

If you decide to swap embedding models because accuracy isn't great, you can't keep using the old index. Different dimensions can't even upsert; even with the same dimension, the vector spaces are different and similarity becomes meaningless. Always rebuild the index when you change the embedding model. I run a parallel index with a _v2 suffix, then flip the binding on cutover day, like a DNS change.

3. Regretting Your Chunk ID Scheme

I started with simple IDs like ${docId}_${i}. The problem: every time a document was updated, I needed to delete the old chunks before upserting new ones—but Vectorize's deleteByIds requires exact ID matches, so I needed to know how many old chunks existed. I now store the current chunk count in KV under doc:${docId}:chunkCount, and on update I bulk-delete the old range first.

async function reindexDoc(env: Bindings, docId: string, body: string) {
  const oldCount = parseInt((await env.KV.get(`doc:${docId}:chunkCount`)) ?? "0");
  if (oldCount > 0) {
    const ids = Array.from({ length: oldCount }, (_, i) => `${docId}#${i}`);
    await env.VECTORIZE_INDEX.deleteByIds(ids);
  }
  const chunks = chunk(body);
  const vectors = await embedAll(env, chunks, docId);
  await env.VECTORIZE_INDEX.upsert(vectors);
  await env.KV.put(`doc:${docId}:chunkCount`, String(chunks.length));
}

4. Embedding Cold Starts Are Real

The first request to a Workers AI model after an idle period can take a few seconds. If that happens on a user-driven query, the experience suffers. I run a Cron Trigger every 5 minutes that hits a /health endpoint to keep the embedding model warm. Costs are pennies a month, and p99 latency drops by seconds.

5. Japanese Queries Don't Work with Default FTS

D1's FTS5 with the unicode61 tokenizer doesn't recognize word boundaries in Japanese, so it returns much less than you'd expect. For Chinese/Japanese/Korean documents, switch to the trigram tokenizer (tokenize='trigram', available in SQLite 3.34+). Doing so roughly doubled recall on Japanese queries in my deployment.

6. Forgetting Idempotent Ingestion

Early on, I let /ingest be called multiple times on the same document without any guard. The result was duplicate chunks in the index, which then dominated search results because they had identical embeddings ranked near each other. The fix is to make ingestion idempotent: always run the delete-then-upsert dance described above, or compute a hash of the body and skip the work if it hasn't changed.

async function ingestIfChanged(env: Bindings, docId: string, body: string) {
  const newHash = await sha256(body);
  const oldHash = await env.KV.get(`doc:${docId}:hash`);
  if (oldHash === newHash) {
    return { changed: false };
  }
  await reindexDoc(env, docId, body);
  await env.KV.put(`doc:${docId}:hash`, newHash);
  return { changed: true };
}

A content hash short-circuits the most expensive part of the pipeline (embedding generation) and saves real money over time, especially when your ingestion job re-scans an entire corpus on a schedule.

7. Underestimating Eval Discipline

The hardest part of RAG isn't building the pipeline—it's knowing whether your changes made things better. Without an eval set, you'll convince yourself that every tweak is an improvement, when in fact several of them silently regress quality.

I keep a small spreadsheet of 30 representative questions and the article ID I expect to be the top hit for each. Every time I touch the chunker, embedding model, reranker, or topK, I run the eval suite and compare the recall@5 number. The discipline is unglamorous but it's what separates "I think this works" from "I can prove it works."

// src/lib/eval.ts
export type EvalCase = { question: string; expectedDocId: string; tags?: string[] };
 
export async function runEval(env: Bindings, cases: EvalCase[]) {
  let recall1 = 0;
  let recall5 = 0;
  const failures: { case: EvalCase; got: string[] }[] = [];
 
  for (const c of cases) {
    const results = await hybridSearch(env, c.question, 10);
    const ids = results.map((r) => r.id.split("#")[0]);
    if (ids[0] === c.expectedDocId) recall1 += 1;
    if (ids.slice(0, 5).includes(c.expectedDocId)) recall5 += 1;
    else failures.push({ case: c, got: ids.slice(0, 5) });
  }
 
  return {
    recall1: recall1 / cases.length,
    recall5: recall5 / cases.length,
    failures,
  };
}

Run this in a Workers Cron daily, store the metrics in Analytics Engine, and you have a regression alarm bell you actually trust.

A Real-World Scenario: A Solo Developer Setup

Putting it all together at a realistic personal-product scale.

Scenario: Site Search and Related-Article Recommendations for a Tech Blog

Target scale:

• 500 articles (250 in each language)
• ~6 chunks per article → ~3,000 vectors total
• 1,000 queries/day

Architecture:

• Cloudflare Workers (Hono): API surface
• Vectorize: vector search (768-dim, cosine)
• D1: FTS index + article metadata
• KV: query cache (TTL 24h)
• Workers AI: embeddings (bge-m3) + reranker (bge-reranker-base) + LLM (llama-3.1-8b-instruct)
• Analytics Engine: usage logs
• R2: backup of original Markdown

Jobs:

• Cron Trigger (every 5 min): health check + warmup
• Cron Trigger (daily 03:00 UTC): golden query regression test
• Manual webhook on article add/update: reindex

In practice, this stack costs me about $5–7/month, p95 latency is ~60 ms for retrieval and ~1.4 s for retrieval + answer generation, and the health check passes 99.8% of the time. Compared to my previous Pinecone-based setup, operating cost is roughly half and latency is ~4× faster.

When to Plan for Scale

Past 100K vectors, Vectorize queries get noticeably heavier. Past 200K, if your metadata filters aren't selective, the Workers AI reranker starts to dominate end-to-end latency. That's roughly the line where it pays to think about your next optimization.

Ordered by impact:

1. Re-tune chunk size: bigger chunks, fewer total vectors
2. Filter early via metadata indexes: scope by category or lang
3. Make reranking optional: skip reranking when the top-5 score gap is large
4. Split into multiple indexes: e.g., one per language or category

If you want the broader Cloudflare context, the related articles Cloudflare Edge SaaS architecture guide and Cloudflare AI Gateway guide cover the upstream (request routing, caching) and downstream (cost management, monitoring) sides of the RAG pipeline described here.

What to Do Next

This article covered a lot of ground, but the action you can take tomorrow is simple: take 10–20 documents you already work with—blog posts, Notion exports, Markdown from a wiki—and ingest them into Vectorize. Hit /query with a few questions and see whether the right things come back.

The minimal scaffold is in the first half of this article. Run it as-is on your own data first. Only after you've felt the gap between the minimal version and what you actually need will hybrid search and reranking feel worth the effort. Trying to build the perfect setup from day one almost always stalls.

My own first RAG was sad. Retrieval was bad enough that I genuinely wondered whether the whole approach was viable. Having a scaffold to iterate on—chunking, model selection, reranking, one piece at a time—is what saved it. The biggest reason to use Antigravity is exactly this: the cost of producing the first working scaffold drops to near-zero.

Edge-deployed RAG is one of the most cost-effective ways for an indie developer to deliver search quality competitive with much larger teams. I hope your next product makes use of it.