Browser Sub-Agent Practical Guide — AI Autonomously Operating the Web

What is Browser Sub-Agent?

Google Antigravity IDE's Browser Sub-Agent gives your AI agent "hands" to operate the web. Unlike traditional web scraping or test automation tools, this AI understands natural language and controls browsers like humans would—clicking, typing, scrolling, and waiting.

Key capabilities:

Web Scraping: Extract data from JavaScript-heavy sites (not just static HTML)
Form Automation: Handle multi-step authentication, complex forms, conditional UI elements
E2E Testing: Run user scenarios automatically, capture screenshots, verify outcomes
Business Automation: Price monitoring, report generation, daily workflow execution

Browser Sub-Agent Architecture

Internal Flow

┌─────────────────────────────────────┐
│ AI Agent (Natural Language Input)   │
└──────────────┬──────────────────────┘
               │
               ↓
┌─────────────────────────────────────┐
│ Intent Parser                       │
│ Decompose into Actions              │
│ - "Click button" → click(xy)        │
│ - "Enter text" → type(selector)     │
└──────────────┬──────────────────────┘
               │
               ↓
┌─────────────────────────────────────┐
│ DOM Interpreter                     │
│ Resolve selectors & compute         │
│ element positions                   │
└──────────────┬──────────────────────┘
               │
               ↓
┌─────────────────────────────────────┐
│ Chromium Browser Instance           │
│ (Headless or Headful mode)          │
│ - Load pages                        │
│ - Execute interactions              │
│ - Capture screenshots               │
└──────────────┬──────────────────────┘
               │
               ↓
┌─────────────────────────────────────┐
│ Output Processor                    │
│ - DOM snapshots                     │
│ - Screenshots                       │
│ - Text extraction                   │
└─────────────────────────────────────┘

Important Limitations & Workarounds

Limitation	Workaround
JS rendering delay (3–5 sec)	Use `waitForTimeout`, `waitForSelector`
Network speed dependent	Set longer timeout values (default 30 sec)
CORS blocks API calls	Implement server-side proxy
Session management	Enable `persistCookies: true`

Web Scraping: Implementation

Basic Pattern

// browser-scraper.ts
import { launch } from 'puppeteer';
 
interface ScrapedProduct {
  title: string;
  price: number;
  rating: number;
  url: string;
}
 
class EcommerceScraper {
  private browser: any;
 
  async initialize() {
    this.browser = await launch({
      headless: true,
      args: [
        '--no-sandbox',
        '--disable-setuid-sandbox',
        '--disable-blink-features=AutomationControlled', // Avoid bot detection
      ],
    });
  }
 
  async scrapeProductListing(
    baseUrl: string,
    pageCount: number = 3
  ): Promise<ScrapedProduct[]> {
    const results: ScrapedProduct[] = [];
    const page = await this.browser.newPage();
 
    // Spoof User-Agent
    await page.setUserAgent(
      'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
    );
 
    for (let i = 1; i <= pageCount; i++) {
      const url = `${baseUrl}?page=${i}`;
      console.log(`Scraping: ${url}`);
 
      try {
        // Wait for page load
        await page.goto(url, { waitUntil: 'networkidle2', timeout: 30000 });
 
        // Wait for dynamic content
        await page.waitForSelector('.product-item', { timeout: 10000 });
 
        // Extract from DOM
        const products = await page.evaluate(() => {
          const items = document.querySelectorAll('.product-item');
          return Array.from(items).map((item) => {
            const titleEl = item.querySelector('.product-title');
            const priceEl = item.querySelector('.product-price');
            const ratingEl = item.querySelector('.product-rating');
            const linkEl = item.querySelector('a.product-link');
 
            return {
              title: titleEl?.textContent?.trim() || 'N/A',
              price: parseFloat(
                priceEl?.textContent?.replace(/[^\d.]/g, '') || '0'
              ),
              rating: parseFloat(ratingEl?.textContent || '0'),
              url: linkEl?.getAttribute('href') || '',
            };
          });
        });
 
        results.push(...products);
        console.log(`Page ${i}: ${products.length} products found`);
 
        // Rate limiting: Add delay between pages
        await page.waitForTimeout(2000 + Math.random() * 3000);
      } catch (error) {
        console.error(`Error scraping page ${i}:`, error);
      }
    }
 
    await page.close();
    return results;
  }
 
  async close() {
    await this.browser.close();
  }
}
 
// Usage
async function main() {
  const scraper = new EcommerceScraper();
  await scraper.initialize();
 
  const products = await scraper.scrapeProductListing(
    'https://example-ecommerce.com/products',
    3
  );
 
  console.log(`Total products scraped: ${products.length}`);
  console.log(JSON.stringify(products, null, 2));
 
  await scraper.close();
}
 
main().catch(console.error);

Rate Limiting

class RateLimitedScraper {
  private requestTimes: number[] = [];
  private maxRequestsPerMinute: number = 10;
 
  async ensureRateLimit() {
    const now = Date.now();
    const oneMinuteAgo = now - 60 * 1000;
 
    // Count requests in last minute
    this.requestTimes = this.requestTimes.filter((t) => t > oneMinuteAgo);
 
    if (this.requestTimes.length >= this.maxRequestsPerMinute) {
      // Rate limited: wait
      const waitTime = 60 * 1000 - (now - this.requestTimes[0]);
      console.log(`Rate limit reached. Waiting ${waitTime}ms...`);
      await new Promise((r) => setTimeout(r, waitTime));
    }
 
    this.requestTimes.push(now);
  }
 
  async scrapePage(url: string) {
    await this.ensureRateLimit();
    // Scraping logic...
  }
}

Sample Output:

[
  {
    "title": "High-Performance AI Laptop",
    "price": 1580,
    "rating": 4.8,
    "url": "https://example.com/product/ai-laptop-pro"
  },
  {
    "title": "USB-C Docking Station",
    "price": 25800,
    "rating": 4.5,
    "url": "https://example.com/product/usb-c-dock"
  }
]

Form Automation & Login

Complex Forms with Conditional Fields

class FormFiller {
  async fillComplexForm(page: any) {
    // Step 1: Text input fields
    await page.type('#name-input', 'Masaki Hirokawa');
    await page.type('#email-input', 'user@example.com');
 
    // Step 2: Radio button selection
    await page.click('input[value="business"]');
 
    // Step 3: Conditional reveal (fields appear based on selection)
    await page.waitForSelector('#company-name-field', { timeout: 5000 });
    await page.type('#company-name-field', 'Dolice Labs');
 
    // Step 4: Multi-select (Ctrl/Cmd+click for multiple)
    const tags = ['AI Development', 'Web Automation', 'Agent System'];
    for (const tag of tags) {
      await page.click(`label:contains("${tag}")`, { modifiers: ['Control'] });
    }
 
    // Step 5: File upload
    const uploadInput = await page.$('input[type="file"]');
    await uploadInput.uploadFile('/path/to/resume.pdf');
 
    // Step 6: Accept terms
    await page.click('#terms-checkbox');
    await page.click('#privacy-checkbox');
 
    // Step 7: Submit
    await Promise.all([
      page.waitForNavigation({ waitUntil: 'networkidle0' }),
      page.click('#submit-button'),
    ]);
 
    console.log('Form submitted successfully');
  }
}

Session Persistence & Auth State

class AuthenticatedSession {
  private cookiesFile = './session-cookies.json';
 
  async saveCookies(page: any) {
    const cookies = await page.cookies();
    require('fs').writeFileSync(this.cookiesFile, JSON.stringify(cookies, null, 2));
  }
 
  async restoreCookies(page: any) {
    if (require('fs').existsSync(this.cookiesFile)) {
      const cookies = JSON.parse(
        require('fs').readFileSync(this.cookiesFile, 'utf8')
      );
      await page.setCookie(...cookies);
      console.log('Cookies restored from previous session');
    }
  }
 
  async login(page: any, email: string, password: string) {
    await page.goto('https://example.com/login');
 
    await page.type('#email', email);
    await page.type('#password', password);
    await page.click('button[type="submit"]');
 
    await page.waitForNavigation({ waitUntil: 'networkidle0' });
 
    // Save cookies for future use
    await this.saveCookies(page);
  }
 
  async accessProtectedPage(url: string) {
    const browser = await launch();
    const page = await browser.newPage();
 
    // Restore previous session
    await this.restoreCookies(page);
 
    await page.goto(url);
    return await page.content();
  }
}

E2E Test Automation

User Scenario-Based Testing

class E2ETestRunner {
  async testCheckoutFlow(baseUrl: string) {
    const browser = await launch();
    const page = await browser.newPage();
 
    try {
      // Scenario 1: Search for product
      console.log('Test 1: Search for product');
      await page.goto(`${baseUrl}/`);
      await page.type('#search-input', 'AI notebook');
      await page.click('#search-button');
      await page.waitForSelector('.product-grid', { timeout: 10000 });
 
      // Scenario 2: Navigate to product detail
      console.log('Test 2: Navigate to product detail');
      await page.click('a.product-link:first-child');
      await page.waitForSelector('.product-detail', { timeout: 5000 });
 
      // Scenario 3: Add to cart
      console.log('Test 3: Add to cart');
      await page.click('#add-to-cart-button');
      const cartBadge = await page.$eval(
        '.cart-count',
        (el) => el.textContent
      );
      console.assert(cartBadge === '1', 'Cart should have 1 item');
 
      // Scenario 4: Proceed to checkout
      console.log('Test 4: Proceed to checkout');
      await page.click('a[href="/checkout"]');
      await page.waitForSelector('#shipping-form', { timeout: 5000 });
 
      // Scenario 5: Fill shipping info
      console.log('Test 5: Fill shipping info');
      await page.type('#address', '123 Tech Street, San Francisco');
      await page.select('#country', 'US');
 
      // Scenario 6: Payment info (use Stripe test keys in production)
      console.log('Test 6: Fill payment info');
      const stripeFrame = page.frames().find((f) =>
        f.url().includes('stripe.com')
      );
      if (stripeFrame) {
        await stripeFrame.type('[placeholder="Card number"]', 'YOUR_TEST_CARD_NUM');
      }
 
      // Scenario 7: Complete order
      console.log('Test 7: Complete order');
      await Promise.all([
        page.waitForNavigation({ waitUntil: 'networkidle0' }),
        page.click('#place-order-button'),
      ]);
 
      // Confirmation screenshot
      const screenshot = await page.screenshot({
        path: './test-results/order-confirmation.png',
      });
      console.log('✓ Checkout flow completed successfully');
 
      return true;
    } catch (error) {
      console.error('✗ Test failed:', error);
      await page.screenshot({ path: './test-results/error-screenshot.png' });
      return false;
    } finally {
      await browser.close();
    }
  }
}
 
// Run tests
const runner = new E2ETestRunner();
runner.testCheckoutFlow('https://example-ecommerce.com')
  .then((success) => process.exit(success ? 0 : 1));

Production Security Design

Secure Credential Management

class SecureAutomation {
  private credentialsManager: CredentialsManager;
 
  async runSecurelyWithCredentials(task: string) {
    // Fetch from environment or Vault
    const credentials = await this.credentialsManager.get(task);
 
    // In-memory encryption
    const decrypted = await this.credentialsManager.decrypt(credentials);
 
    // Clear memory after use
    try {
      return await this.executeTask(task, decrypted);
    } finally {
      // Secure wipe (overwrite)
      decrypted.password = Buffer.alloc(decrypted.password.length, '\0');
    }
  }
}

IP Rotation & Proxy Support

class ProxyRotatingBrowser {
  private proxyList = [
    'http://proxy1.example.com:8080',
    'http://proxy2.example.com:8080',
    'http://proxy3.example.com:8080',
  ];
 
  async launchWithProxy(proxyIndex: number = 0) {
    const proxy = this.proxyList[proxyIndex % this.proxyList.length];
 
    return await launch({
      args: [`--proxy-server=${proxy}`],
    });
  }
 
  async scrapeWithRotation(urls: string[]) {
    for (let i = 0; i < urls.length; i++) {
      const browser = await this.launchWithProxy(i);
      const page = await browser.newPage();
 
      try {
        await page.goto(urls[i]);
        // Scraping logic...
      } finally {
        await browser.close();
      }
    }
  }
}

Request Header Spoofing

async function setupRealisticHeaders(page: any) {
  // Realistic HTTP headers
  await page.setExtraHTTPHeaders({
    'Accept-Language': 'en-US,en;q=0.9,ja;q=0.8',
    'Accept-Encoding': 'gzip, deflate, br',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Cache-Control': 'max-age=0',
    'DNT': '1',
    'Connection': 'keep-alive',
    'Upgrade-Insecure-Requests': '1',
  });
 
  // Prevent automation detection
  await page.evaluateOnNewDocument(() => {
    Object.defineProperty(navigator, 'webdriver', {
      get: () => undefined,
    });
 
    Object.defineProperty(navigator, 'plugins', {
      get: () => [1, 2, 3, 4, 5], // Fake plugins
    });
  });
}

Best Practices

Error Handling is Critical
- Network failures, timeouts, CAPTCHA
- Implement retry logic and logging
Wait Properly for Content
- waitForNavigation(): Page transitions
- waitForSelector(): Dynamic content loading
- Avoid waitForTimeout() (last resort only)
Prevent Memory Leaks
- Properly close browser instances
- Clear large Cookie/localStorage
Logging & Debugging
- Log each step with timestamps
- Screenshots on failures