AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

authoring-scouts

Write, edit, and adapt PostHog Signals scouts: scheduled agents that scan a project and report findings. Use to change a canonical scout's scope, thresholds, schedule, or dry-run settings; add a scout for an uncovered surface; record structured output from scheduled LLM scoring; or give feedback through notes. Covers SKILL.md structure, report requirements, follow-up report checks, structured output, duplicate detection, scratchpad memory, scout notes, per-team skills-store and canonical repository edits, and private prompt trials with saved rubrics. Trigger on "write/edit/customize a signals scout", "new scout for X", "tune my scout schedule", "make a scout that watches <event>", "score/judge/measure X with a scout", "test/compare scout prompts", "generate scout rubrics", "structured output from a scout", "scout output to Slack", "leave a note for / give feedback to a scout".

Original source
posthog
License
MIT
Pinned revision
cac7c80a9e5b73484d18eda1bffb47297c938e0c

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

Ask your assistant

Use Agentic Memory skills to read posthog--authoring-scouts at revision cac7c80a9e5b73484d18eda1bffb47297c938e0c, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 8

references/scout-anatomy.md

Scout anatomy

A scout is a single SKILL.md (its body is loaded verbatim as the agent's system prompt) plus optional references/ files read on demand. Keep the body lean and push depth into references — every line of the body is a recurring token cost on every run.

Contents

  • Naming
  • Frontmatter
  • Body structure (the ten canonical sections)
  • References
  • Skeleton — specialist scout
  • Skeleton — broad / cross-product scout

Naming

Any valid skill name works: lowercase letters, numbers, and hyphens. The SignalScoutConfig row is what makes a skill a scout. Name it in lowercase kebab-case after the surface or question the scout watches: error-tracking, checkout-funnel, mcp-feedback. The canonical fleet keeps the signals-scout- prefix, and a per-team scout can use it too. The prefix only controls whether the coordinator auto-registers a config for a skill that has none, so a report-producing scout named anything else comes in through scout-create, which writes the skill and its config in one call. A scout with a product-specific output comes in through skill-create and scout-config-create, because scout-create always grants the report tools; see Scouts with a product-specific output.

Frontmatter

This template is for a report-producing scout. A scout with a product-specific output omits both report tools and avoids the signals-scout- prefix; see Scouts with a product-specific output for how to create it.

---
name: signals-scout-<scope>
description: >
  One or two sentences, third person: the surface it watches and the specific shapes it
  looks for (bursts, regressions, clusters, drops). Keep it tight. Don't restate the
  fleet-wide boilerplate every scout shares (files reports above the bar, writes
  memory, closes out empty, self-contained peer) — that's assumed, and repeating it
  across the fleet burns the caller's token budget and gets truncated in AI plugins.
allowed_tools:
  - emit_report
  - edit_report
compatibility: >
  Designed for the PostHog Signals agent in a Claude sandbox with PostHog MCP scopes
  (read-only analytics plus signal_scout_report:write for reports and
  signal_scout_internal:write for scratchpad).
  Assumes the signals-scout MCP family (scout-project-profile-get, scout-runs-list,
  scout-runs-retrieve, scout-scratchpad-search, scout-scratchpad-remember,
  scout-scratchpad-forget, scout-emit-report, scout-edit-report)
  plus whatever query tools the scope needs (e.g. execute-sql, read-data-schema,
  query-error-tracking-issues-list, inbox-reports-list).
metadata:
  owner_team: signals # or the team that owns the scope
  scope: <scope> # short machine label, e.g. error_tracking, csp_violations
---

name and description are required and validated at build time. allowed_tools with emit_report / edit_report is what puts the scout on the report channel — every report-producing scout needs it (without it the scout falls back to a deprecated legacy signal-emitting channel and can't write reports). A scout with a product-specific output omits both tools; see Scouts with a product-specific output. compatibility and metadata are optional but conventional — compatibility documents the scopes/tools the scout assumes; metadata.scope gives downstream tooling a short label. A canonical scout can also declare scout-precheck-query, the default scheduled-run gate. The harness reads it only from the shipped canonical file, so a custom scout sets precheck_query on its config instead. See scheduled-precheck.md.

The description does double duty: beyond skill discovery, it is surfaced verbatim as the scout's description on the config API (scout-config-list / -create / -update responses) — it's how the fleet roster reads to agents and the UI without opening each scout's body. Write it to stand alone in that listing, and keep it short: it's also loaded alongside every other scout's into a caller's AI plugin, where a wordy description wastes token budget and gets truncated. A sentence or two that names the surface and the shapes is the whole job.

Body structure

The canonical body is a workflow, not a script — it reads like how an experienced analyst would approach the surface, and trusts the agent to adapt. (One variant departs from it: a recurring measurement / LLM-judge scout on the structured-output channel replaces the discriminator + Decide sections with a rubric and a sample → judge → record loop — see that pattern in scout-patterns.md; orient and memory stay the same, and so does close-out — except its quick early-exit fires only on an empty eligible population, never at a steady baseline, since the scout samples and records every verdict (the unremarkable ones are the denominator) on any run with items to judge.) The fleet's specialists all share this shape:

  1. Identity + discriminator (the most important lines). One sentence on what the scout is, then name the signal-vs-noise discriminator explicitly and tell the agent to internalize it. This is the cheap profile-shape read that separates "worth a look" from "baseline". Examples: count vs distinct_users ratio (error tracking); reach over raw count (CSP); negative+mixed share vs baseline (MCP feedback). Without this, the scout wastes every run re-deciding what "normal" means.

  2. Quick close-out. A cheap early-exit so a quiet run costs almost nothing: if the surface is not in use, or the watched event sits at baseline (no fresh 24h activity), write one scratchpad entry and stop. This keeps idle scouts cheap. Decide "not in use" from the profile's surfaces entry for the scout, never from absence in top_events: top_events keeps only the top events by volume, so a quiet surface that is in use falls off it. A canonical scout gets a surfaces entry when it ships a scout-precheck-query. A scout with no entry, or an entry that reads in_use: null, checks with one bounded execute-sql count. top_events counts are windowed (each row carries window_days), not lifetime — a project whose ingestion recently went dark reads identically to one that never had traffic. Before closing out a busy-looking project as empty on top_events thinness alone, rule out a capture gap with a direct execute-sql over a longer window (e.g. 30d); only close out when the low volume holds there.

    key:     not-in-use:<scope>:team{team_id}     # if the surface is absent entirely
         or  pattern:<scope>:baseline-team{team_id} # if it fires at a steady baseline
    content: "<surface> baseline ~{count}/day, no fresh 24h burst at {timestamp}"
    
  3. Orient. Three cheap reads cold-start every run — bake them into the body:

    • scout-scratchpad-search (text=<scope keyword>) — durable steering from past runs; the pattern: / noise: / addressed: / dedupe: entries tell the scout what's normal and what's already covered.
    • scout-runs-list (last 7d) — what prior runs of this scout found and ruled out. Pull -runs-retrieve only for a summary worth drilling into. The fleet-wide read (siblings' runs, and following an interesting summary into the report it produced) is already in the harness prompt for every scout, so don't restate it in your body.
    • scout-project-profile-get — the deterministic snapshot; read the discriminator metrics off the relevant top_events row.
  4. Profile shape / discriminator table. A small table mapping the discriminator's shapes to what they usually mean, so the agent triages fast. (See the error-tracking scout's count-vs-distinct_users table for the canonical example.)

  5. Explore patterns. 2–4 named investigation patterns — starting points, not a checklist. Each names the concrete tools/queries to run and the shape that confirms it. E.g. "Burst with broad reach" → list active issues, SQL hourly breakdown, look for the one-occurrence-per-distinct-user shape. Give the agent real queries, not generic advice.

  6. Save memory as you go. Tell the scout to write scratchpad entries continuously, encoding the category in the key prefix (see dedupe-and-memory.md). Give 2–3 worked example entries scoped to this surface so the agent matches the format.

  7. Decide. Author / edit / remember / skip, calibrated against the report contract (see report-contract.md) and the four-states classifier (see dedupe-and-memory.md). State the surface-specific "report-worthy" thresholds (e.g. "a broad-reach burst with concrete entity ids and counts in the evidence"). Tell it to cross-check inbox-reports-list before authoring — an existing report on the topic gets an edit_report, not a duplicate.

  8. Disqualifiers. The known noise for this surface that should be skipped (single-user quirks, dev-env bursts, allowlisted domains, known upstream provider errors). "When in doubt, write memory instead of filing a report."

  9. MCP tools. List the direct (read-only) calls and the harness-level tools the scout uses, so the agent doesn't rediscover them each run.

  10. Close out. One paragraph: looked at what, filed/edited what, remembered what, ruled out what. The harness saves this as the run summary; future runs read it via scout-runs-list. Tell it not to write a separate "run metadata" scratchpad entry — the summary already serves that role. "Looked but found nothing meaningful" is a real outcome.

Not every scout needs all ten sections, but every scout needs 1 (discriminator), 2 (quick close-out), 3 (orient), 7 (decide), 8 (disqualifiers), and 10 (close out). Sections 4–6 and 9 are where a specialist earns its keep.

References

The generalist carries references/conventions.md (the four-states author/edit classifier + scratchpad vocab); the report-channel contract itself rides in the harness prompt (injected into every report-channel scout), so a scout bundles no copy of it. For a per-team scout you usually don't need to bundle your own copies — the canonical scout already encodes the conventions inline, and your scout body can too. Bundle a reference only when you have genuinely surface-specific depth (a long SQL cookbook, a taxonomy of fingerprints) that would bloat the body. Attach bundled files to a per-team scout with posthog:skill-file-create; in the repo, drop them in references/ and they're collected automatically.

Skeleton — specialist scout

---
name: signals-scout-<scope>
description: >
  Signals scout for PostHog <surface>. Watches <event/metric> for <the shapes: bursts /
  regressions / clusters / drops>.
allowed_tools:
  - emit_report
  - edit_report
compatibility: >
  Designed for the PostHog Signals agent in a Claude sandbox with PostHog MCP scopes
  (read-only analytics plus signal_scout_report:write and signal_scout_internal:write).
  Assumes the signals-scout MCP family plus <the query tools this scope needs>.
metadata:
  owner_team: <team>
  scope: <scope>
---

# Signals scout: <surface>

You are a focused <surface> scout. Spot meaningful changes in <event/metric> — <the
shapes> — and file a report only when a finding clears the report bar.

<Name the discriminator here.> The relationship between <X> and <Y> is the most important
signal-vs-noise discriminator. Internalize that shape.

## Quick close-out: is <surface> even loud?

If `surfaces["<skill-name>"]` reads `in_use: false`, or <event> is at baseline (no fresh 24h activity), <surface>
isn't where the signal is today. Absence from `top_events` alone proves nothing. Cheap scratchpad entry + close out empty.

## How a run works

Cycle between these moves; skip what's not useful.

### Get oriented

- `scout-scratchpad-search` (`text=<scope keyword>`) — durable steering.
- `scout-runs-list` (last 7d) — what prior runs found and ruled out.
- `scout-project-profile-get` — read the discriminator metrics off `top_events`.

### Profile shape

| Pattern   | What it usually means           |
| --------- | ------------------------------- |
| <shape A> | <meaning A — investigate first> |
| <shape B> | <meaning B — usually noise>     |

### Explore

Patterns to watch — starting points, not a checklist.

#### <Pattern 1>

<the concrete queries/tools and the confirming shape>

#### <Pattern 2>

<...>

### Save memory as you go

Write a scratchpad entry whenever you observe something a future run should know. Encode the
category in the key prefix — `pattern:`, `noise:`, `addressed:`, `dedupe:`.

- key `pattern:<scope>:baseline` — "<normal shape for this project>"
- key `dedupe:<scope>:<entity>` — "<surfaced when, with what condition for next run>"

### Decide

- **Author** a report via `scout-emit-report` above the bar (a well-formed
  finding you'd own end-to-end, concrete entity ids + counts in evidence).
  Cross-check `inbox-reports-list` first — an existing report on the topic gets a
  `scout-edit-report` instead of a duplicate.
- **Remember** if below the bar but worth carrying forward.
- **Skip** if a `noise:` / `addressed:` / `dedupe:` entry already covers it.

### Close out

One paragraph: looked at what, filed/edited what, remembered what, ruled out what.

## Disqualifiers (skip these)

- <surface-specific noise: single-user, dev-env, allowlisted, known-upstream>

## MCP tools

Direct (read-only): <list>. Harness-level: scout-project-profile-get, scout-scratchpad-search,
scout-runs-list, scout-runs-retrieve, scout-emit-report, scout-edit-report,
scout-scratchpad-remember.

Skeleton — broad / cross-product scout

Start from signals-scout-general instead. Its job is cross-product correlations and surfaces no specialist covers — it deliberately leaves single-surface deep dives to the specialists and rotates investigative lenses across runs to avoid lens-lock. Use this shape when your scout's question spans products (e.g. "deploy → error burst → revenue dip") rather than living inside one surface.