Automated Lesson Extractor

The learning pipeline relies on the agent self-reporting its mistakes. But agents miss things — especially when the conversational pull to move on is strong. The lesson extractor is an automated cron that scans recent session transcripts for undocumented mistakes.

How It Works

The cron runs on a schedule (every 4 hours), spinning up an isolated session that:

1. Finds recent sessions — scans session log files from the last 4 hours
2. Searches for trigger phrases — regex scan for mistake indicators in assistant messages ("my bad", "should have", "missed that", "forgot to", etc.) and correction indicators in user messages ("you forgot", "why didn't you", "that's wrong", etc.)
3. Cross-references with daily notes — checks whether each identified mistake was actually documented in that day's Mistakes/Lessons section
4. Fills gaps — writes missing entries to the daily note, adds Lesson Candidates for repeatable patterns, triggers post-mortems for significant failures
5. Reports or stays silent — announces findings to the relevant channel, or returns NO_REPLY if nothing was missed

Configuration

{
  "name": "lesson-extractor",
  "schedule": { "kind": "every", "everyMs": 14400000 },  // every 4 hours
  "sessionTarget": "isolated",
  "payload": {
    "kind": "agentTurn",
    "model": "sonnet",  // mid-tier model is sufficient for text scanning
    "timeoutSeconds": 300,
    "message": "// scans sessions, cross-references daily notes, fills gaps"
  },
  "delivery": { "mode": "announce" }
}

Lessons from Running It

• Timeout must be generous. Session log scanning + jq parsing + daily note cross-referencing takes time. The initial 180s timeout caused consecutive failures. 300s is safer.
• False positive filtering matters. The prompt explicitly says: "Don't create noise. Only flag genuine mistakes. Ignore humor, casual self-deprecation, or routine conversation." Without this, every "oops, typo" becomes a logged lesson.
• It catches real gaps. The document-before-acknowledge rule gets violated regularly. The extractor has found cases where mistakes were verbally acknowledged in conversation but never written to the daily note — exactly the gap it's designed to close.
• Automation that stays broken is worse than no automation. If the extractor fails and nobody re-enables it, you assume coverage you don't have. Monitor the monitor.