Agent Memory Design Patterns in Antigravity — Persisting Context Across Sessions

What Does "Memory" Mean for an Agent?

One of the first walls developers hit when building AI agents is maintaining context across sessions. Conversational LLMs can only hold information within their context window — when a user returns the next day, the previous conversation is gone.

Antigravity is a flexible platform for building agent architectures, but without proper memory design, even the most sophisticated agent will give users a frustrating "starting from zero" experience every time they open it.

The Three Layers of Agent Memory

Organizing agent memory into three distinct layers makes the design much cleaner:

1. Working Memory (Short-term) The conversation history within the current session. This is what the context window provides. Antigravity's standard conversation history handles this.

2. Episodic Memory (Medium-term) Specific past interactions like "this user mentioned X last week" or "they chose option Y in this project." These need to persist across sessions.

3. Semantic Memory (Long-term) Abstracted facts and knowledge like "this user prefers React" or "this project is TypeScript." Referenced over the longest timeframes.

Pattern 1: Episodic Memory — Structured Storage of Past Sessions

The simplest approach is to save the key points of each session as structured data in a database, and load them at the start of the next session.

Implementation: Episodic Memory with Supabase

// memory/episode-store.ts
import { createClient } from '@supabase/supabase-js';
 
interface Episode {
  id: string;
  userId: string;
  sessionId: string;
  summary: string;
  keyFacts: string[];
  timestamp: string;
}
 
const supabase = createClient(
  process.env.SUPABASE_URL!,
  process.env.SUPABASE_KEY!
);
 
export async function saveEpisode(episode: Omit<Episode, 'id'>): Promise<void> {
  await supabase.from('episodes').insert(episode);
}
 
export async function recallRecentEpisodes(
  userId: string,
  limit = 5
): Promise<Episode[]> {
  const { data } = await supabase
    .from('episodes')
    .select('*')
    .eq('userId', userId)
    .order('timestamp', { ascending: false })
    .limit(limit);
 
  return data || [];
}

Agent Skill: Summarize and Save at Session End

// skills/summarize-and-save.ts
export async function summarizeAndSave(
  conversationHistory: Message[],
  userId: string,
  sessionId: string
): Promise<void> {
  const summary = await callClaude(`
    Summarize the following conversation in under 200 words and extract
    key facts as a JSON array.
 
    Conversation:
    ${conversationHistory.map(m => `${m.role}: ${m.content}`).join('\n')}
 
    Response format:
    {
      "summary": "...",
      "keyFacts": ["...", "..."]
    }
  `);
 
  const parsed = JSON.parse(summary);
 
  await saveEpisode({
    userId,
    sessionId,
    summary: parsed.summary,
    keyFacts: parsed.keyFacts,
    timestamp: new Date().toISOString(),
  });
}

Pattern 2: Semantic Memory — Meaning-based Search with Vector DB

While episodic memory captures "what happened when," semantic memory stores "what we know about the user." Text is embedded into vectors and stored for semantic similarity search.

Implementation with pgvector

// memory/semantic-store.ts
 
interface SemanticMemory {
  id: string;
  userId: string;
  content: string;
  embedding: number[];
  createdAt: string;
}
 
export async function embedAndStore(
  userId: string,
  fact: string
): Promise<void> {
  const embedding = await getEmbedding(fact); // OpenAI Embeddings API
 
  await supabase.from('semantic_memories').insert({
    userId,
    content: fact,
    embedding,
    createdAt: new Date().toISOString(),
  });
}
 
export async function recallRelevant(
  userId: string,
  query: string,
  topK = 5
): Promise<SemanticMemory[]> {
  const queryEmbedding = await getEmbedding(query);
 
  // Cosine similarity search using pgvector's <=> operator
  const { data } = await supabase.rpc('match_memories', {
    query_embedding: queryEmbedding,
    match_user_id: userId,
    match_count: topK,
  });
 
  return data || [];
}

Supabase Custom Function (SQL)

CREATE OR REPLACE FUNCTION match_memories(
  query_embedding vector(1536),
  match_user_id text,
  match_count int DEFAULT 5
)
RETURNS TABLE(
  id uuid,
  content text,
  similarity float
)
LANGUAGE sql
AS $$
  SELECT
    id,
    content,
    1 - (embedding <=> query_embedding) AS similarity
  FROM semantic_memories
  WHERE user_id = match_user_id
  ORDER BY embedding <=> query_embedding
  LIMIT match_count;
$$;

Pattern 3: Memory-Augmented Prompt Construction

Having memory means nothing if you don't inject it into the agent's prompt effectively. Here's how to do it in Antigravity:

// agent/memory-augmented-agent.ts
export async function buildSystemPrompt(
  userId: string,
  currentQuery: string
): Promise<string> {
  // Fetch both memory types in parallel
  const [recentEpisodes, relevantMemories] = await Promise.all([
    recallRecentEpisodes(userId, 3),
    recallRelevant(userId, currentQuery, 5),
  ]);
 
  const episodeContext = recentEpisodes.length > 0
    ? `## Recent Session History\n${recentEpisodes.map(e => `- ${e.summary}`).join('\n')}`
    : '';
 
  const memoryContext = relevantMemories.length > 0
    ? `## Relevant Memories\n${relevantMemories.map(m => `- ${m.content}`).join('\n')}`
    : '';
 
  return `You are the user's personal AI assistant.
 
${episodeContext}
 
${memoryContext}
 
Use the memories above to provide consistent, personalized assistance.
If new important facts emerge, suggest adding them to memory.`;
}

Memory Maintenance Strategies

Memory needs more than just writing — it also requires updating, pruning, and weighting. Here are three strategies to prevent stale information from causing problems:

1. TTL (Time-to-Live) for Episodes

const EPISODE_TTL_DAYS = 30;
 
export async function cleanOldEpisodes(userId: string): Promise<void> {
  const cutoff = new Date();
  cutoff.setDate(cutoff.getDate() - EPISODE_TTL_DAYS);
 
  await supabase
    .from('episodes')
    .delete()
    .eq('userId', userId)
    .lt('timestamp', cutoff.toISOString());
}

2. Deduplication Before Storage

Before saving new memories, check whether a similar memory already exists:

export async function storeIfNovelFact(
  userId: string,
  fact: string
): Promise<boolean> {
  const similar = await recallRelevant(userId, fact, 1);
 
  if (similar.length > 0 && similar[0].similarity > 0.92) {
    console.log('Skipping duplicate memory:', fact);
    return false;
  }
 
  await embedAndStore(userId, fact);
  return true;
}

3. Importance Scoring

interface MemoryWithScore extends SemanticMemory {
  importanceScore: number;
  accessCount: number;
  lastAccessedAt: string;
}
 
// Boost importance score every time a memory is accessed
export async function accessMemory(memoryId: string): Promise<void> {
  await supabase.rpc('increment_importance', { memory_id: memoryId });
}

Antigravity Integration: Defining a Memory Skill in SKILL.md

Using Antigravity's skill system, you can let the agent handle its own memory autonomously:

# Memory Manager Skill
 
## Description
Extract important information from user interactions and store/retrieve
it from persistent memory.
 
## Triggers
- When the user says "remember this" or "make a note"
- When a new user preference is revealed in conversation
- At session start and end
 
## Actions
1. **recall**: Retrieve memories relevant to the current query
2. **store**: Save a new fact to semantic memory
3. **summarize**: Save an episode summary at session end
4. **forget**: Delete a memory when the user says "forget that"

Where to Start

You do not need all three layers at once. In my experience as an indie developer building operations agents for my own sites since 2014, the order of impact is: episodic memory → prompt augmentation → semantic memory. Saving a session summary on exit (Pattern 1) and loading it on start (Pattern 3) removes most of the "starting from zero" friction by itself. Vector-based semantic recall earns its keep only once memories number in the hundreds and "the latest five" stops being enough.

One operational item worth designing up front: the deletion path — letting a user say "forget that" — is hard to retrofit, so put it in the first schema rather than the second. I hope this saves you a refactor.