n8n × AI Agents — Self-Hosting Gotchas and Parsing the JSON That Comes Back Broken

Setting Up the Self-Hosted Environment

A Minimal Docker Compose

# docker-compose.yml
services:
  n8n:
    image: docker.n8n.io/n8nio/n8n:2.35.5
    restart: unless-stopped
    ports:
      - "127.0.0.1:5678:5678"
    environment:
      - N8N_HOST=your-domain.com
      - N8N_PORT=5678
      - N8N_PROTOCOL=https
      - WEBHOOK_URL=https://your-domain.com/
      - GENERIC_TIMEZONE=Asia/Tokyo
      - TZ=Asia/Tokyo
      # Always supply your own key from .env (see below)
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
      # Trust exactly one reverse proxy hop for X-Forwarded-*
      - N8N_PROXY_HOPS=1
    volumes:
      - n8n_data:/home/node/.n8n
 
volumes:
  n8n_data:

# Generate the key and keep it in .env — skipping this line costs you later
echo "N8N_ENCRYPTION_KEY=$(openssl rand -hex 32)" > .env
chmod 600 .env
 
docker compose up -d
docker compose logs -f n8n

On first boot, open http://localhost:5678 and create the owner account. That account is your administrator.

Three Settings That No Longer Work

A docker-compose.yml I copied from elsewhere refused to start, and tracking down why took longer than it should have. These three lines are the usual survivors from older guides.

The second one is the dangerous one. It fails silently, so nothing tells you the instance is unauthenticated. If you set those variables and never see a login prompt behave the way you expect, assume they are no longer being read.

Pinning the image tag instead of using :latest also pays off later. Without a pinned version, every docker compose pull can change behavior, and a flow that breaks becomes much harder to attribute.

VPS Placement and Nginx

I run mine on a $6/month VPS (1 vCPU / 1 GB RAM) behind Nginx with Let's Encrypt. n8n starts on 512 MB, but if you push large payloads through AI nodes, plan on 1 GB.

Because Compose binds the port to 127.0.0.1:5678, the only path in from outside is through Nginx.

# /etc/nginx/sites-available/n8n
server {
    listen 443 ssl http2;
    server_name your-domain.com;
 
    ssl_certificate     /etc/letsencrypt/live/your-domain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/your-domain.com/privkey.pem;
 
    # Raise the ceiling for flows that handle attachments
    client_max_body_size 32m;
 
    location / {
        proxy_pass http://127.0.0.1:5678;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
        # Editor execution results stream over SSE — don't buffer them
        proxy_buffering off;
        proxy_read_timeout 300s;
    }
}
 
server {
    listen 80;
    server_name your-domain.com;
    return 301 https://$host$request_uri;
}

Pasting a server block with no listen directive leaves nothing bound on 443, and connections simply never land. X-Forwarded-Proto and the N8N_PROXY_HOPS value above are a pair — drop either one and n8n will hand out webhook URLs that still say http://.

Losing the Encryption Key Means Losing the Credentials

The worst self-hosting incident is not the server going down. It is restoring from backup and finding that the credentials will not decrypt.

n8n encrypts credentials before storing them in the database. The key that decrypts them does not live in the database — it lives in /home/node/.n8n/config, or in the N8N_ENCRYPTION_KEY environment variable. If you never set that variable, n8n generates a random key on first boot and writes it into config.

So recreating the volume produces a new key. Restoring only the database leaves every credential encrypted under a key that no longer exists. Typing the -v in docker compose down -v puts you there immediately.

There are two defenses. Choose the key yourself and pass it from .env. And verify at backup time that the backup is restorable, rather than at restore time. For the second, I run this at the end of the nightly backup job.

#!/usr/bin/env bash
# backup-verify.sh — check that an extracted backup can actually be restored
set -uo pipefail
DIR="${1:?usage: backup-verify.sh <dir>}"
fail=0
chk() { if [ -e "$2" ]; then echo "  OK   $1"; else echo "  MISS $1 ($2)"; fail=$((fail+1)); fi; }
 
echo "[1] All three required pieces present"
chk "database"          "$DIR/database.sqlite"
chk "config file"       "$DIR/config"
chk "workflow export"   "$DIR/export/workflows.json"
 
echo "[2] Encryption key readable"
KEY=$(python3 -c "import json;print(json.load(open('$DIR/config')).get('encryptionKey',''))" 2>/dev/null)
if [ -z "$KEY" ]; then echo "  MISS no encryptionKey in config"; fail=$((fail+1))
else echo "  OK   key found (fingerprint: $(printf %s "$KEY" | sha256sum | cut -c1-12))"; fi
 
echo "[3] Key matches the previous backup"
FP_FILE="$DIR/.key-fingerprint"
NOW=$(printf %s "$KEY" | sha256sum | cut -c1-12)
if [ -f "$FP_FILE" ]; then
  PREV=$(cat "$FP_FILE")
  if [ "$PREV" = "$NOW" ]; then echo "  OK   unchanged ($NOW)"
  else echo "  WARN key changed ($PREV -> $NOW). Older credentials cannot be decrypted"; fail=$((fail+1)); fi
else echo "  INIT first run, recording fingerprint"; fi
printf %s "$NOW" > "$FP_FILE"
 
echo "[4] Credentials exported with their payloads"
CRED="$DIR/export/credentials.json"
if [ -f "$CRED" ]; then
  N=$(python3 -c "import json;print(len(json.load(open('$CRED'))))" 2>/dev/null || echo 0)
  D=$(grep -c '"data"' "$CRED" 2>/dev/null || echo 0)
  echo "  count $N / with data field $D"
  [ "$N" -gt 0 ] && [ "$D" -eq 0 ] && { echo "  WARN exported without --decrypted"; fail=$((fail+1)); }
else echo "  MISS credentials.json"; fail=$((fail+1)); fi
 
echo
[ "$fail" -eq 0 ] && echo "RESULT: restorable" || echo "RESULT: $fail issue(s)"
exit $fail

Feed it a backup whose key was regenerated and you get this:

[3] Key matches the previous backup
  WARN key changed (a99e5428f925 -> 783283448b2f). Older credentials cannot be decrypted

RESULT: 1 issue(s)

It records only the first 12 characters of the SHA-256 digest in .key-fingerprint, never the key itself — I would rather not leave plaintext keys sitting in a backup bucket. It returns an exit code, so || notify in cron is all the wiring you need.

The check may look paranoid, but the difference in how I feel about that backup, before and after adding it, is substantial. A backup should be judged by whether it restores, not by whether it ran.

Wiring In the AI Nodes

n8n's AI node coverage grew quickly through late 2024. Claude, Gemini, and OpenAI are all available as nodes, and orchestration can stay entirely inside n8n.

Connecting Claude

Register your Anthropic API key under Credentials, then drop an "Anthropic Chat Model" node into the flow. When the prompt needs to be built dynamically, put a Code node in front of it.

// Code node: build a classification prompt from an incoming email
const emailBody = $json.email_body ?? '';
const sender = $json.sender_name ?? '(unknown)';
 
// Long threads inflate both tokens and latency — trim to the head
const excerpt = emailBody.slice(0, 4000);
 
return {
  json: {
    prompt: [
      'Classify the following email.',
      'Output exactly one JSON object. Do not add any prose before or after it.',
      'category must be support / sales / other. priority must be high / medium / low.',
      '',
      `Sender: ${sender}`,
      `Body: ${excerpt}`,
      '',
      'Format: {"category":"...","priority":"...","summary":"..."}',
    ].join('\n'),
  },
};

Workflow 1: Email Classification

This is the shape I actually run.

Gmail Trigger (new email)
  │
  ├─ Filter: drop spam and promotions
  │
  ├─ Claude Node: analyze and classify
  │
  ├─ Code Node: extract JSON from the response (function below)
  │
  ├─ Switch Node: branch on category
  │   ├─ Support → create a ticket in a Notion database
  │   ├─ Sales   → post to the Slack #sales channel
  │   └─ Other   → apply a Gmail label only
  │
  └─ Completion notice

Workflow 2: Weekly Report Generation

Scheduler (Mondays, 09:00)
  │
  ├─ Google Analytics Node: last week's traffic
  ├─ Google Search Console: search performance
  ├─ Notion Node: tasks completed last week
  │
  ├─ Code Node: format the numbers into the prompt
  │
  ├─ Gemini Node: draft the report body
  │
  └─ Gmail Node: send it to myself

For long-form generation I prefer Gemini; for short structured output like classification I reach for Claude. Mixing models inside one flow is one of the quiet luxuries of self-hosting.

Workflow 3: Social Post Scheduling

Notion Trigger (new content item)
  │
  ├─ Claude Node: adapt the copy per platform
  │   ├─ X: under 280 characters
  │   ├─ LinkedIn: professional tone
  │   └─ Instagram: hashtag optimization
  │
  ├─ Approval step (review in Slack → button approval)
  │
  └─ Scheduling node: queue each platform

The approval step is the one thing I have never removed. Ten seconds of human eyes on the copy is cheap insurance against a post you cannot take back.

"Return JSON" Is Not Enough on Its Own

The part of the classification flow that refused to stabilize was the parsing. However strictly the prompt specifies the format, responses still arrive with "Sure! Here is the JSON:" in front, wrapped in a code fence, or carrying a trailing comma. Every one of those crashes the step before the Switch node.

So I collected the failure shapes into 13 fixtures and ran four extraction strategies against them.

• S1 bare JSON.parse
• S2 strip the code fence, then JSON.parse
• S3 slice from the first { to the last }
• S4 walk the braces while tracking string literals and escapes, repair smart quotes, single quotes, and trailing commas, then JSON.parse

The results:

Bare JSON.parse clearing only 4 was expected. What surprised me was S3 plateauing at 9. "First brace to last brace" is the shortcut nearly everyone reaches for, but the moment a trailing explanation mentions something like a set of allowed values in braces, the slice swallows it and breaks. I fell into that once, and switched to walking the braces myself afterward.

Here is the extraction function I use. It drops straight into a Code node.

// Code node: pull a JSON object out of an LLM response
function extractJsonObject(text) {
  const t = String(text).replace(/^\uFEFF/, '');
  const start = t.indexOf('{');
  if (start < 0) throw new Error('no object found');
 
  // Find the matching brace while respecting string literals and escapes
  let depth = 0, inStr = false, esc = false, raw = null;
  for (let i = start; i < t.length; i++) {
    const c = t[i];
    if (esc) { esc = false; continue; }
    if (c === '\\') { esc = true; continue; }
    if (c === '"') { inStr = !inStr; continue; }
    if (inStr) continue;
    if (c === '{') depth++;
    else if (c === '}' && --depth === 0) { raw = t.slice(start, i + 1); break; }
  }
  if (raw === null) throw new Error('unbalanced braces');
 
  const repaired = raw
    .replace(/[\u201c\u201d]/g, '"')          // smart double quotes
    .replace(/[\u2018\u2019]/g, "'")          // smart single quotes
    .replace(/'([^'\\]*)'(\s*[:,}])/g, '"$1"$2')
    .replace(/([{,]\s*)'([^'\\]*)'(\s*:)/g, '$1"$2"$3')
    .replace(/,(\s*[}\]])/g, '$1');           // trailing comma
 
  return JSON.parse(repaired);
}
 
const ALLOWED_CATEGORY = ['support', 'sales', 'other'];
const ALLOWED_PRIORITY = ['high', 'medium', 'low'];
 
return $input.all().map((item) => {
  let parsed;
  try {
    parsed = extractJsonObject(item.json.text ?? item.json.response ?? '');
  } catch (e) {
    // Don't throw — route it to "other" and keep the raw text for diagnosis
    return { json: { category: 'other', priority: 'low', summary: '', parse_error: e.message, raw: item.json.text } };
  }
  return {
    json: {
      category: ALLOWED_CATEGORY.includes(parsed.category) ? parsed.category : 'other',
      priority: ALLOWED_PRIORITY.includes(parsed.priority) ? parsed.priority : 'low',
      summary: String(parsed.summary ?? '').slice(0, 500),
    },
  };
});

The other thing worth noticing is that a parse failure does not throw. Throwing inside an n8n Code node fails the entire execution. One malformed email stopping the flow means every email behind it queues up too. Letting the broken item fall out of classification, tagged with parse_error, made the whole thing far quieter to operate.

The ALLOWED_CATEGORY guard follows the same logic. If the model answers "Support" or a localized label, the Switch node silently misses every branch. Funneling the unexpected into one visible place is what lets you notice it at all.

When the AI Agent Node Earns Its Cost

Recent n8n versions include an "AI Agent" node that goes beyond a single LLM call, letting the model pick tools as it works.

AI Agent Node
  ├─ System Prompt: "You are a data analysis agent. Use the tools to..."
  ├─ Tools: Calculator / HTTP Request / Code Executor
  └─ Model: Anthropic Chat Model (set under Credentials)

The agent decides for itself that a calculation is needed, or that it should fetch more data. I use this for research-gathering flows.

The flip side: pointing an agent node at a fixed, known procedure inflates both runtime and cost. If you can already draw the branches, Switch and IF nodes are faster and far more reproducible. The agent is the tool for work you cannot script in advance — that is the line I hold.

Three Things I Watch on a Self-Hosted Instance

1. Make the webhook paths hard to guess Leaving the default /webhook/ exposed invites scanners. Even adding a random segment like /webhook/a7f3k9m2b6/ cuts the log noise dramatically.

2. Never write API keys into the workflow itself Use Credentials or .env. You will eventually export a workflow's JSON to share it, and that is the moment the shortcut costs you.

3. Don't let execution data pile up By default, execution records accumulate until SQLite gets sluggish. Setting EXECUTIONS_DATA_PRUNE and EXECUTIONS_DATA_MAX_AGE keeps a small VPS comfortable.

A Note From an Indie Developer

Cap the flows at three, and consolidate before adding a fourth

Early on I added a flow every time I had an idea, and lost track of which one owned what. Now I keep one rule: once three flows are running, look for a merge before building a new one. Folding similar work into a single flow with a Switch means fewer places to look when something changes.

Build the flow without AI first, then swap the node in

Putting an LLM in from the start makes it impossible to tell whether the output is drifting or the wiring is wrong. I run the flow end to end with a Code node returning fixed values, then replace that node with the AI node. Since adopting that order, the time it takes me to localize a problem has dropped noticeably.

Keep the "is it still running" check outside the box

n8n's error workflows are useful, but they cannot notify you when n8n itself is down. I keep a container health check and a once-a-day external webhook ping somewhere other than that server. Not putting the watcher inside the thing being watched is obvious in hindsight, and easy to skip.

Where to Go Next

Start with the three lines in docker-compose.yml: the image name, any leftover BASIC_AUTH variables, and an explicit N8N_ENCRYPTION_KEY. Getting those right is what keeps a backup from being unrestorable.

After that, look at whatever parses the output of your AI nodes. If it is still a bare JSON.parse, swapping in the extraction function above will recover executions that have been failing quietly.

Automation is something you grow rather than something you finish. I am still working much of this out myself, and I hope these sticking points save someone else the detour. Thank you for reading.

Configuration examples were verified on n8n 2.35.5. If you are staying on 1.x, the encryption key and image name sections still apply as written.