Antigravity × Cloudflare Vectorize: Build a Production RAG Pipeline That Runs at the Edge

Ingestion Pipeline: Chunking, Embeddings, and Metadata

The minimal version works, but it's not production. The single biggest factor in retrieval quality is chunking strategy.

I started with fixed 1,000-character chunks. The result: chunks that broke mid-heading, with "title-only fragments" frequently ranking near the top. After much iteration, I settled on a Markdown-aware chunker that respects heading hierarchy and aims for 600–900 chars per chunk on paragraph boundaries.

A Markdown-Aware Chunker

// src/lib/chunker.ts
type Section = { heading: string; level: number; body: string };
 
export function parseMarkdown(md: string): Section[] {
  const lines = md.split("\n");
  const sections: Section[] = [];
  let current: Section = { heading: "", level: 1, body: "" };
 
  for (const line of lines) {
    const headingMatch = line.match(/^(#{1,6})\s+(.+)$/);
    if (headingMatch) {
      if (current.body.trim() || current.heading) sections.push(current);
      current = {
        heading: headingMatch[2].trim(),
        level: headingMatch[1].length,
        body: "",
      };
    } else {
      current.body += line + "\n";
    }
  }
  if (current.body.trim() || current.heading) sections.push(current);
  return sections;
}
 
export function chunkSections(
  sections: Section[],
  target = 800,
  overlap = 100
): { text: string; heading: string }[] {
  const out: { text: string; heading: string }[] = [];
 
  for (const section of sections) {
    const fullText = section.heading
      ? `## ${section.heading}\n${section.body.trim()}`
      : section.body.trim();
 
    if (fullText.length <= target) {
      out.push({ text: fullText, heading: section.heading });
      continue;
    }
 
    const paragraphs = fullText.split(/\n{2,}/);
    let buf = "";
    for (const p of paragraphs) {
      if ((buf + "\n\n" + p).length > target && buf) {
        out.push({ text: buf.trim(), heading: section.heading });
        const tail = buf.slice(Math.max(0, buf.length - overlap));
        buf = tail + "\n\n" + p;
      } else {
        buf = buf ? `${buf}\n\n${p}` : p;
      }
    }
    if (buf.trim()) out.push({ text: buf.trim(), heading: section.heading });
  }
  return out;
}

Switching to this chunker boosted my site search click-through (the rate at which a search result actually got clicked) by about 1.6×. The reason is unglamorous: keeping the heading at the top of each chunk preserves enough surrounding context for the embedding to reflect what the section is actually about.

Choosing an Embedding Model

The main embedding models available on Workers AI:

• @cf/baai/bge-small-en-v1.5 (384 dim, English)
• @cf/baai/bge-base-en-v1.5 (768 dim, English)
• @cf/baai/bge-large-en-v1.5 (1024 dim, English)
• @cf/baai/bge-m3 (1024 dim, multilingual)

If you're indexing Japanese content, bge-m3 is the only sensible choice. I lost a full day in my early days trying to debug poor retrieval on Japanese queries before realizing I was running an English-only model. Lesson: check the language coverage of your embedding model first.

That said, bge-m3 is 1024-dim, which directly increases Vectorize storage costs and query latency. For predominantly English content, bge-base-en-v1.5 (768-dim) is more economical.

Metadata Design You'll Be Glad You Made

The metadata you attach at upsert time becomes the lever you pull later for filtering. The fields I wish I'd added from day one:

• docId: stable ID of the source document (essential for updates and deletes)
• title: for display
• category / tag: for filtered search
• lang: to scope queries by language
• updatedAt (unix timestamp): for re-indexing decisions
• text: the chunk body itself (Vectorize doesn't return body text by default, so you must store it)

You must create metadata indexes ahead of time with wrangler vectorize create-metadata-index. Adding them later doesn't backfill existing vectors, so be exhaustive before going to production.

Three Places the Scaffold Breaks First in Production: Dimensions, Concurrency, Response Shape

Having just spent a section on chunking, I should admit that the things which actually cost me time came earlier than retrieval quality. Three of them. All are documented, and none of them are visible the moment you run the scaffold. Here's what each one looks like when it hits you, in the hope that it saves someone the same afternoon.

1. Dimension mismatch: "somehow only upsert fails"

This was the first one. I created the index at 768 dimensions and embedded with bge-small-en-v1.5. Embedding generation in /query worked fine; only the upsert in /ingest failed:

VectorizeError: vector dimension mismatch:
index expects 768 dimensions, but got 384

The cause is mundane: bge-small-en-v1.5 outputs 384 dimensions. I had assumed "small" meant roughly half of something bigger and created the index with --dimensions=768. Because the failure surfaces at upsert rather than at embedding time, my first instinct was to inspect the chunk contents — which is exactly why it took as long as it did.

You cannot infer a Workers AI model's output dimension from its name. Measuring it once, before you create anything, is worth the thirty seconds:

# Probe the embedding dimension before creating the index
curl -s -X POST \
  "https://api.cloudflare.com/client/v4/accounts/$CF_ACCOUNT_ID/ai/run/@cf/baai/bge-m3" \
  -H "Authorization: Bearer $CF_API_TOKEN" \
  -d '{"text":["dimension probe"]}' | jq '.result.data[0] | length'
# => 1024

That one line removes an entire index-rebuild round trip. This article standardizes the scaffold on bge-m3 (1024 dimensions) because any pipeline handling non-English content ends up there anyway. An English-only corpus can start on bge-base-en-v1.5 (768 dimensions) — but note that an index's dimension cannot be changed after creation. When you do have to rebuild, the painful part isn't the vectors themselves; it's paying Workers AI to re-embed the entire corpus.

2. Concurrency: Promise.all bites at around 30 chunks

Embedding every chunk via chunks.map() + Promise.all reads naturally and works fine for ten chunks. Mine broke when a long article pushed past forty:

Error: Too many API requests by single worker invocation.

Workers cap the number of concurrent subrequests per invocation, and Promise.all spends that budget all at once. Worse, short articles never reproduce it, so the symptom presents as "ingestion fails for this one specific document" — the hardest kind of failure to read.

The fix is the one now in /ingest above: split into batches of roughly eight and run them sequentially. That sounds like giving up throughput, but the measured difference on a 60-chunk article was 3.1s fully parallel (and failing) versus 4.4s batched. Paying a second to actually finish is clearly the right trade for an ingestion job.

The upsert call itself accepts up to 1,000 vectors, so batching the embeddings while keeping a single upsert at the end works fine.

3. Reranker response shape: id is an index into what you sent

The third one is smaller and, because it fails silently, more annoying. Workers AI's @cf/baai/bge-reranker-base returns { response: [{ id, score }] }, where id is the index into the contexts array you passed in — not your chunk ID.

I had written it as index, so candidates[r.index] evaluated to undefined and the entire reranking step was quietly producing garbage. It never threw. You get back an array that looks reordered, and nothing tells you otherwise until you read the results yourself.

// ✅ Correct: id indexes into contexts
const ranked = result.response.map((r: { id: number; score: number }) => ({
  id: candidates[r.id].id,
  score: r.score,
}));
 
// ❌ Common mistake: there is no `index` property
// candidates[r.index] is always undefined

Defects that fail silently instead of throwing are exactly what the evaluation suite below earns its keep on. I only caught this one because recall@5 dropped unnaturally against the previous day — not by reading the code.

Lifting Search Quality to Production: Hybrid Search and Reranking

Pure vector search struggles with proper nouns and version numbers. A query like "Antigravity 1.20" embeds near a lot of vaguely-related content, and unrelated articles bubble up.

I solved this with hybrid search (vector + full-text) + reranking in two stages.

Hybrid Search with D1 as a Full-Text Index

// src/lib/hybrid-search.ts
type HybridResult = {
  id: string;
  score: number;
  source: "vector" | "fts" | "both";
  text: string;
  title: string;
};
 
export async function hybridSearch(
  env: { VECTORIZE_INDEX: VectorizeIndex; DB: D1Database; AI: Ai },
  query: string,
  topK = 10
): Promise<HybridResult[]> {
  // Run vector and FTS in parallel to minimize wall-clock
  const [vectorRes, ftsRes] = await Promise.all([
    (async () => {
      const { data } = await env.AI.run("@cf/baai/bge-m3", { text: [query] });
      return env.VECTORIZE_INDEX.query(data[0], {
        topK,
        returnMetadata: "all",
      });
    })(),
    env.DB.prepare(
      `SELECT id, title, text, rank
       FROM chunks_fts
       WHERE chunks_fts MATCH ?1
       ORDER BY rank LIMIT ?2`
    )
      .bind(query, topK)
      .all<{ id: string; title: string; text: string; rank: number }>(),
  ]);
 
  // Reciprocal Rank Fusion (RRF) blends the two
  const map = new Map<string, HybridResult>();
  vectorRes.matches.forEach((m, i) => {
    map.set(m.id, {
      id: m.id,
      score: 1 / (60 + i),
      source: "vector",
      text: String(m.metadata?.text ?? ""),
      title: String(m.metadata?.title ?? ""),
    });
  });
  (ftsRes.results ?? []).forEach((r, i) => {
    const existing = map.get(r.id);
    const ftsScore = 1 / (60 + i);
    if (existing) {
      existing.score += ftsScore;
      existing.source = "both";
    } else {
      map.set(r.id, {
        id: r.id,
        score: ftsScore,
        source: "fts",
        text: r.text,
        title: r.title,
      });
    }
  });
 
  return Array.from(map.values()).sort((a, b) => b.score - a.score);
}

On the D1 side, set up an FTS5 virtual table:

-- migrations/0001_chunks_fts.sql
CREATE VIRTUAL TABLE chunks_fts USING fts5(
  id UNINDEXED,
  title,
  text,
  tokenize='unicode61 remove_diacritics 2'
);

Reciprocal Rank Fusion is simple but powerful. It merges results from vector search and FTS using only their ranks, so you don't have to worry about score normalization between two scoring systems with very different magnitudes.

Re-Rank the Top N with Workers AI Reranker

Pull a wider net (top 20) with hybrid search, then have a reranker decide which are actually most relevant.

// src/lib/rerank.ts
export async function rerank(
  env: { AI: Ai },
  query: string,
  candidates: { id: string; text: string }[]
): Promise<{ id: string; score: number }[]> {
  const result = await env.AI.run("@cf/baai/bge-reranker-base", {
    query,
    contexts: candidates.map((c) => ({ text: c.text })),
  });
 
  // response[].id is the *index into the contexts array you sent*,
  // not your own chunk ID. (I got this wrong once — more on that below.)
  return result.response
    .map((r: { id: number; score: number }) => ({
      id: candidates[r.id].id,
      score: r.score,
    }))
    .sort((a, b) => b.score - a.score);
}

Adding a reranker visibly improved the factual quality of LLM answers in my system. Before reranking, the LLM occasionally "filled in" facts that weren't in the documents at all. After reranking, those hallucinations almost vanished.

Cost Design: Make Tokens, Calls, and Storage Visible

Vectorize cost has three axes:

• Queries (vector × topK)
• Stored vectors (GB/month)
• Workers AI calls (embeddings, LLM, and reranker each metered separately)

For a small SaaS scale (10K queries/month, 50K stored chunks), my bill stays under about $7/month. But if you're not careful, Workers AI tokens are where bloat creeps in.

Logging Usage to Make Cost Visible

// src/lib/cost-logger.ts
type UsageEvent = {
  type: "embedding" | "llm" | "rerank" | "vectorize_query";
  inputTokens?: number;
  outputTokens?: number;
  vectorCount?: number;
  durationMs: number;
  ts: number;
};
 
export class UsageLogger {
  private events: UsageEvent[] = [];
  constructor(private analytics: AnalyticsEngineDataset) {}
 
  log(event: UsageEvent) {
    this.events.push(event);
    this.analytics.writeDataPoint({
      indexes: [event.type],
      blobs: [JSON.stringify(event)],
      doubles: [event.durationMs, event.inputTokens ?? 0],
    });
  }
 
  summary(): Record<string, { calls: number; tokens: number; ms: number }> {
    return this.events.reduce((acc, e) => {
      const k = e.type;
      acc[k] ??= { calls: 0, tokens: 0, ms: 0 };
      acc[k].calls += 1;
      acc[k].tokens += (e.inputTokens ?? 0) + (e.outputTokens ?? 0);
      acc[k].ms += e.durationMs;
      return acc;
    }, {} as Record<string, { calls: number; tokens: number; ms: number }>);
  }
}

Analytics Engine is Cloudflare's pay-as-you-go log fabric, free for hundreds of thousands of events per month in practice. I thread UsageLogger through every endpoint and surface a usage summary in the response headers during development.

Three Cost Wins That Actually Mattered

The optimizations that moved the needle in my deployment:

1. Cache layer: Hash the query, look it up in KV before doing anything else. FAQ-shaped questions hit cache 70%+ of the time
2. Early return on weak matches: If the top vector score is below a threshold (I use 0.55), don't even call the LLM
3. Model downgrade: Route easy questions to llama-3.2-3b-instruct instead of llama-3.1-8b-instruct

The cache alone cut Workers AI costs by about 40%. The implementation is trivial:

// src/lib/hash.ts
// Workers don't ship Node's crypto by default, so use Web Crypto here.
// (If you want node:crypto, add the nodejs_compat flag to wrangler.toml.)
export async function sha256(input: string): Promise<string> {
  const buf = await crypto.subtle.digest(
    "SHA-256",
    new TextEncoder().encode(input)
  );
  return Array.from(new Uint8Array(buf))
    .map((b) => b.toString(16).padStart(2, "0"))
    .join("");
}
 
// src/lib/query-cache.ts
import { sha256 } from "./hash";
 
export async function getCachedAnswer(
  kv: KVNamespace,
  query: string
): Promise<string | null> {
  return await kv.get(`q:${(await sha256(query)).slice(0, 32)}`);
}
 
export async function setCachedAnswer(
  kv: KVNamespace,
  query: string,
  answer: string,
  ttlSec = 86400
) {
  await kv.put(`q:${(await sha256(query)).slice(0, 32)}`, answer, {
    expirationTtl: ttlSec,
  });
}

Observability: Logs, Metrics, and Surfacing Failure Modes

What matters most in production is being able to identify the cause when accuracy drops, costs spike, or latency creeps up. If you want to keep everything inside Cloudflare, this is the stack I recommend:

• Workers Logs (built-in): error stack traces, cold-start timing
• Analytics Engine: custom metrics (query rate, average latency, reranker score distribution)
• Tail Workers: forward failed requests to a separate Worker for long-term storage and Slack alerts

Monitoring the distribution of reranker scores is what lets you catch quality regressions early. I once bulk-updated some articles and the chunk boundaries shifted; the reranker's top score average fell from ~0.6 to ~0.3 overnight. Without dashboards I'd have only learned about it from a user complaint days later.

// src/lib/health-check.ts
export async function searchHealthCheck(env: Bindings) {
  const goldenQueries = [
    { q: "How do I get started with Antigravity?", expectedDocId: "antigravity-getting-started" },
    { q: "What does Cloudflare Vectorize cost?", expectedDocId: "antigravity-cloudflare-vectorize-rag-production-guide" },
  ];
 
  const failures: { q: string; got: string[] }[] = [];
  for (const { q, expectedDocId } of goldenQueries) {
    const results = await hybridSearch(env, q, 5);
    const ids = results.map((r) => r.id.split("#")[0]);
    if (!ids.includes(expectedDocId)) failures.push({ q, got: ids });
  }
  return { passed: failures.length === 0, failures };
}

Run this from a Cron Trigger every hour and post failures to Slack. You'll catch regressions long before users do.

Common Pitfalls

You can read this entire article, feel prepared, and still trip on something. Here are the traps I (and other indie developers I know) have actually fallen into, with fixes.

1. Forgetting to Create Metadata Indexes

returnMetadata: "all" lets you read metadata from results, but using metadata in a filter clause requires that you create a metadata index up front. I missed this initially: my "filtered" queries appeared to ignore the filter entirely. Half a day was lost to it. The Vectorize docs are explicit, but it's an easy thing to glide past.

# Without this, filter: { docId: "xxx" } won't work
wrangler vectorize create-metadata-index blog-knowledge \
  --property-name=docId --type=string

2. Switching Embedding Models Mid-Index

If you decide to swap embedding models because accuracy isn't great, you can't keep using the old index. Different dimensions can't even upsert; even with the same dimension, the vector spaces are different and similarity becomes meaningless. Always rebuild the index when you change the embedding model. I run a parallel index with a _v2 suffix, then flip the binding on cutover day, like a DNS change.

3. Regretting Your Chunk ID Scheme

I started with simple IDs like ${docId}_${i}. The problem: every time a document was updated, I needed to delete the old chunks before upserting new ones—but Vectorize's deleteByIds requires exact ID matches, so I needed to know how many old chunks existed. I now store the current chunk count in KV under doc:${docId}:chunkCount, and on update I bulk-delete the old range first.

async function reindexDoc(env: Bindings, docId: string, body: string) {
  const oldCount = parseInt((await env.KV.get(`doc:${docId}:chunkCount`)) ?? "0");
  if (oldCount > 0) {
    const ids = Array.from({ length: oldCount }, (_, i) => `${docId}#${i}`);
    await env.VECTORIZE_INDEX.deleteByIds(ids);
  }
  const chunks = chunk(body);
  // embedAll is the batched embedding loop from /ingest, extracted into a function
  const vectors = await embedAll(env, chunks, docId);
  await env.VECTORIZE_INDEX.upsert(vectors);
  await env.KV.put(`doc:${docId}:chunkCount`, String(chunks.length));
}

4. Embedding Cold Starts Are Real

The first request to a Workers AI model after an idle period can take a few seconds. If that happens on a user-driven query, the experience suffers. I run a Cron Trigger every 5 minutes that hits a /health endpoint to keep the embedding model warm. Costs are pennies a month, and p99 latency drops by seconds.

5. Japanese Queries Don't Work with Default FTS

D1's FTS5 with the unicode61 tokenizer doesn't recognize word boundaries in Japanese, so it returns much less than you'd expect. For Chinese/Japanese/Korean documents, switch to the trigram tokenizer (tokenize='trigram', available in SQLite 3.34+). Doing so roughly doubled recall on Japanese queries in my deployment.

6. Forgetting Idempotent Ingestion

Early on, I let /ingest be called multiple times on the same document without any guard. The result was duplicate chunks in the index, which then dominated search results because they had identical embeddings ranked near each other. The fix is to make ingestion idempotent: always run the delete-then-upsert dance described above, or compute a hash of the body and skip the work if it hasn't changed.

async function ingestIfChanged(env: Bindings, docId: string, body: string) {
  const newHash = await sha256(body);
  const oldHash = await env.KV.get(`doc:${docId}:hash`);
  if (oldHash === newHash) {
    return { changed: false };
  }
  await reindexDoc(env, docId, body);
  await env.KV.put(`doc:${docId}:hash`, newHash);
  return { changed: true };
}

A content hash short-circuits the most expensive part of the pipeline (embedding generation) and saves real money over time, especially when your ingestion job re-scans an entire corpus on a schedule.

7. Underestimating Eval Discipline

The hardest part of RAG isn't building the pipeline—it's knowing whether your changes made things better. Without an eval set, you'll convince yourself that every tweak is an improvement, when in fact several of them silently regress quality.

I keep a small spreadsheet of 30 representative questions and the article ID I expect to be the top hit for each. Every time I touch the chunker, embedding model, reranker, or topK, I run the eval suite and compare the recall@5 number. The discipline is unglamorous but it's what separates "I think this works" from "I can prove it works."

// src/lib/eval.ts
export type EvalCase = { question: string; expectedDocId: string; tags?: string[] };
 
export async function runEval(env: Bindings, cases: EvalCase[]) {
  let recall1 = 0;
  let recall5 = 0;
  const failures: { case: EvalCase; got: string[] }[] = [];
 
  for (const c of cases) {
    const results = await hybridSearch(env, c.question, 10);
    const ids = results.map((r) => r.id.split("#")[0]);
    if (ids[0] === c.expectedDocId) recall1 += 1;
    if (ids.slice(0, 5).includes(c.expectedDocId)) recall5 += 1;
    else failures.push({ case: c, got: ids.slice(0, 5) });
  }
 
  return {
    recall1: recall1 / cases.length,
    recall5: recall5 / cases.length,
    failures,
  };
}

Run this in a Workers Cron daily, store the metrics in Analytics Engine, and you have a regression alarm bell you actually trust.

A Real-World Scenario: A Solo Developer Setup

Putting it all together at a realistic personal-product scale.

Scenario: Site Search and Related-Article Recommendations for a Tech Blog

Target scale:

• 500 articles (250 in each language)
• ~6 chunks per article → ~3,000 vectors total
• 1,000 queries/day

Architecture:

• Cloudflare Workers (Hono): API surface
• Vectorize: vector search (1024-dim, cosine — matching bge-m3)
• D1: FTS index + article metadata
• KV: query cache (TTL 24h)
• Workers AI: embeddings (bge-m3) + reranker (bge-reranker-base) + LLM (llama-3.1-8b-instruct)
• Analytics Engine: usage logs
• R2: backup of original Markdown

Jobs:

• Cron Trigger (every 5 min): health check + warmup
• Cron Trigger (daily 03:00 UTC): golden query regression test
• Manual webhook on article add/update: reindex

In practice, this stack costs me about $5–7/month, p95 latency is ~60 ms for retrieval and ~1.4 s for retrieval + answer generation, and the health check passes 99.8% of the time. Compared to my previous Pinecone-based setup, operating cost is roughly half and latency is ~4× faster.

When to Plan for Scale

Past 100K vectors, Vectorize queries get noticeably heavier. Past 200K, if your metadata filters aren't selective, the Workers AI reranker starts to dominate end-to-end latency. That's roughly the line where it pays to think about your next optimization.

Ordered by impact:

1. Re-tune chunk size: bigger chunks, fewer total vectors
2. Filter early via metadata indexes: scope by category or lang
3. Make reranking optional: skip reranking when the top-5 score gap is large
4. Split into multiple indexes: e.g., one per language or category

The upstream half of this pipeline — request routing, caching, and auth — is covered in the Cloudflare Edge SaaS architecture guide. Settling "who calls this search, and how" before you tune the search itself saves a surprising amount of rework.

What to Do Next

This article covered a lot of ground, but the action you can take tomorrow is simple: take 10–20 documents you already work with—blog posts, Notion exports, Markdown from a wiki—and ingest them into Vectorize. Hit /query with a few questions and see whether the right things come back.

The minimal scaffold is in the first half of this article. Run it as-is on your own data first. Only after you've felt the gap between the minimal version and what you actually need will hybrid search and reranking feel worth the effort. Trying to build the perfect setup from day one almost always stalls.

My own first RAG was sad. Retrieval was bad enough that I genuinely wondered whether the whole approach was viable. Having a scaffold to iterate on—chunking, model selection, reranking, one piece at a time—is what saved it. The biggest reason to use Antigravity is exactly this: the cost of producing the first working scaffold drops to near-zero.

Edge-deployed RAG is one of the most cost-effective ways for an indie developer to deliver search quality competitive with much larger teams. I hope your next product makes use of it.