SKILL.md
Authoring Signals scouts
A scout is a scheduled agent that wakes on its own interval, looks at one PostHog project, decides what's genuinely worth surfacing, and writes it into the Signals inbox as a report — or closes out empty, which is a real outcome. PostHog ships a fleet of canonical scouts (a cross-product generalist plus per-surface specialists). This skill helps you and your agent adapt those canonical scouts to a specific project, or author new scouts from scratch for a use case the fleet doesn't cover.
A report-producing scout's output is the report channel: it lists emit_report / edit_report in its frontmatter allowed_tools and authors or edits full inbox reports 1:1 directly.
Most of the canonical fleet runs this way, and every new report-producing scout should too — include the allowed_tools opt-in when you author one.
A scout whose output is a product surface (for example, suggestions on a workflow page) omits both tools; see Scouts with a product-specific output.
Where that output lands is a separate, per-scout config decision: the report goes to the Signals inbox, and the same report can be delivered to a Slack channel or DM at the same time (output_destinations under Run posture) — so don't rule a scout out of a job because the user wants the result in Slack.
(A historical signal-emitting channel — weak emit-signal findings a pipeline consolidated — still exists in the harness for scouts that never opted in, but it is deprecated: don't author new scouts on it, and opt an old one in rather than extending it.)
A scout is an LLMSkill that holds a SignalScoutConfig.
The harness loads the body verbatim as the agent's system prompt, and progressively reads any bundled reference files on demand.
The config row is what makes a skill a scout. Any valid skill name works, so the signals-scout- prefix is optional.
The prefix controls one thing: the coordinator globs signals-scout-* to auto-register a config for a skill that has none.
A skill with any other name needs its config created alongside it, which is what scout-create does.
The job before the writing
Don't write a scout in the abstract.
Ground it in the target project first — a scout is only as good as its fit to the data it watches.
(The scout tools were recently renamed from signals-scout-* to scout-*; if a scout-* name comes back unknown, the server may still expose it under the legacy signals-scout-* name — search the tool catalog and call whichever name it returns.)
- Read the project.
posthog:scout-project-profile-getreturns the deterministic snapshot the scout itself cold-starts from: products in use, top events with reach/burst metrics, integrations, existing inbox counts. If the scout watches a specific event, confirm it exists and check its shape withposthog:read-data-schema. A scout for an event the project doesn't capture is dead on arrival. - See what already runs.
posthog:scout-config-listlists every existing scout on the project with its schedule,enabled, andemitposture, plus each scout'sdescription(pulled from the skill's frontmatter) so you can tell what a scout watches without loading its body. Don't duplicate a surface a canonical scout already covers — adapt that one instead. - Read the closest canonical scout. It's your template and your reference shape.
Pull it with
posthog:skill-get {"skill_name": "signals-scout-<x>"}(per-team rows) or read it from the repo atproducts/signals/skills/signals-scout-*/. The generalist (signals-scout-general) is the broad template; if your scope is domain-tight, pick the specialist closest to your surface — list the live roster withposthog:skill-list {"search": "signals-scout"}(specialists exist for most product surfaces: error tracking, logs, AI observability, experiments, feature flags, session replay, web analytics, surveys, and more). - Skim the inbox.
posthog:inbox-reports-listshows what reports are actually landing — calibrate so your scout adds signal, not noise.
Choose the path
There are two independent decisions: what you're building, and where it lives.
What
| Situation | Approach | | ---------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | A canonical scout is close but too broad / too noisy / missing a disqualifier for this project | Adapt it — narrow the scope, add disqualifiers, retune thresholds. | | You want a surface no canonical scout covers (a custom event, a product-specific funnel) | New scout from scratch — copy the closest canonical scout as scaffolding, replace the domain discriminator + explore patterns. | | You want to compare prompts, models, or reasoning efforts before applying a change | Private trial — keep the saved scout intact, review its rubric, and compare variants through the Test loop below. | | You only want to change when / whether a scout runs | No authoring — just tune the config (see Run posture). | | You have one-off feedback, a pointer, or short-lived context for a scout | No authoring — leave a note (see Steering with notes). |
Where
| Path | Mechanism | Use when |
| ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| Per-team (the common user path) | Create a new runnable scout via posthog:scout-create; edit its prompt or files later via posthog:skill-update / -file-create, and tune its runtime config via posthog:scout-config-update. | Customizing for one project. The harness globs the row in on the next tick; canonical sync leaves your edited ("diverged") row alone. |
| Canonical (PostHog contributors) | Edit disk under products/signals/skills/signals-scout-*/, lint/build, open a PR. | Improving a scout for every enrolled project. lazy_seed mirrors it onto all enrolled teams on the next tick. |
Adapting-in-place tradeoff: editing a canonical scout's row for your team marks it diverged — you stop receiving upstream improvements to that scout.
If you only need an additional behavior, prefer authoring a new, differently-named scout (signals-scout-<your-scope>) and leaving the canonical one intact.
See references/lifecycle-and-testing.md for the exact skills-store calls, the build/lint commands, and how seeding works.
Write the scout
First pick the shape.
references/scout-patterns.md is a cookbook of the reference architectures scouts fall into — anomaly watcher, liveness/absence watcher, zero-result/unmet demand, watchlist explore/exploit, cross-product correlation, recommendation/gap, warehouse-backed source, custom single-event, open-text theme, adversarial/abuse concentration, external-tool/code, state∩code intersection, custom issue-tracker/work-queue, daily digest/roll-up, triage over a pre-detected stream, first-person dogfooding/probe, recurring measurement/LLM-judge, maintainer/steward, owner-scoped book/queue, trigger-to-brief enrichment, dispatcher/campaign, fleet meta-scout/reviewer — each mapped to a canonical scout or a proven custom shape you can copy as scaffolding.
It also makes the key point that a scout can watch any source PostHog ingests into the data warehouse, not just analytics events (a Slack channel sync, a billing system, a CRM, a support inbox), plus external systems reachable from the sandbox, other agents' output, and the scout fleet itself.
And where a built-in signals source already covers the surface (GitHub and Linear issues), the issue-tracker pattern says where that source stops and a scout starts paying for itself.
A scout does not have to stop at describing a fix: the maintainer / steward pattern is how a scout holding write_scopes keeps a family of dashboards, alerts, warehouse views, or scanner prompts healthy itself, with a write ladder that says which changes it applies, which it verifies and reports, and which it hands to a human.
Find the closest pattern, then write the body.
Follow references/scout-anatomy.md — it has the frontmatter schema (including the allowed_tools report-channel opt-in every report-producing scout needs; a scout with a product-specific output omits it), the canonical body structure (quick close-out → orient → domain discriminator → explore patterns → save-memory → decide → disqualifiers → close-out), the lean-body rule, and copy-ready skeleton templates for both a specialist and the generalist.
Write the body feature-forward. The body is a prompt a run reads in full, every run, for as long as the scout lives — so it states what the scout watches and how it decides, and nothing else. Four things do not belong in it:
- No rollout state. A preview flag, an early-access gate, a "recently renamed" tool, a port that is happening one scout at a time. It is true this month and misleading next month, and a run cannot tell which. Write the behavior the scout should follow now; if a capability may be absent, say how the run tells, not when it is due.
- No backlog. Work that is planned, a field that does not exist yet, a lane somebody means to add. A run cannot act on it, so it only spends tokens and invites the run to wait for something.
- No project-only facts. A metric, dashboard, event, id, or reviewer that exists in one project. A canonical scout ships to every project, so a fact from one of them is wrong for almost every run. Text about an optional product says how the run tells whether the project uses it, and never assumes it does.
- No duplicated harness mechanics. The report-channel contract, the run gates, and the close-out format come from the harness prompt, which every report-channel scout is given. A copy in the body drifts from the real one and the run then holds two versions of the same rule. Keep the body to your scout's own domain framing.
The same four apply to a bundled reference: it is read at run time too, so it carries no more rollout state or backlog than the body.
Two craft references the whole fleet reasons in terms of — a good scout's Decide and memory sections are built on them, so read them before writing those sections:
references/report-contract.md— the report tools (scout-emit-report/scout-edit-report), the report bar (author 1:1 only for a finding you'd own end-to-end),suggested_reviewersrouting, the dedup-via-report_iddiscipline (the channel isn't idempotent — reconcile against existing reports via the vanillainbox-reports-list/inbox-reports-retrievebefore authoring), and the accepted caveat that the pipeline may later rewrite an authored title/summary. This is how your scout decides what clears the bar and how to file it.references/dedupe-and-memory.md— the four-states classifier (net-new / material-update / already-covered / addressed-or-noise), the scratchpad key-prefix vocabulary, and the cross-project noise patterns. This is how your scout avoids re-filing and learns across runs.references/report-checks.md— the follow-up checks a scout attaches to a report so "did the fix hold?" is measured later instead of remembered. This is how your scout closes the loop on a finding after somebody acts on it.
A report is backward-looking; a check is the opposite direction — an expectation plus a time to test it, written onto a report with scout-report-check-create.
Give your scout a checks section when its findings are the kind whose fix shows up in data later.
Three rules belong in the body, and the reference has the rest:
- List the report's existing checks first (
scout-report-check-list), every time. An open check for the same claim makes a second one noise, and a report holds at most five open checks. metric_thresholdwherever one number settles the claim and an event or action series can carry it. The coordinator measures it itself, with no scout run.agentwhen no single number settles it, or when the number lives outside events — a log rate, a fix whose effect shows in which entities fire rather than how many, a claim that needs a stack trace read.
The single most important design decision in any scout is its signal-vs-noise discriminator — the cheap profile-shape read that separates "worth investigating" from "baseline".
For error tracking it's the count vs distinct_users ratio; for CSP it's reach over raw count.
Your new scout needs its own.
Name it explicitly near the top of the body so every run anchors on it.
(The one exception: a measurement scout on the structured-output channel holds no bar — it applies a rubric to every sampled item, and the rubric takes the discriminator's slot as the design surface to name, dogfood, and calibrate. See the recurring measurement / LLM-judge pattern in references/scout-patterns.md.)
A second design consideration applies to a metric-shaped scout — one that scores, ranks, or reports a named, reusable measure, whether a business measure (MRR, churn risk, usage revenue, activation) or operational telemetry it computes every run to monitor or report (cost per run, failure or error rates, latency, throughput).
If the project has an approved metric for that measure (metric-list shows what exists), name it in the body and run it with data-catalog-metric-run, so the scout's number matches the one the team already reports.
Run posture (config)
A scout's schedule and emit behavior live on its SignalScoutConfig, separate from the skill body.
For a brand-new scout, pass these settings in the nested config object of the posthog:scout-create call, including creating it disabled or in dry-run before it ever runs.
When creating a scout to test an idea, set config.enabled=false so scheduled runs cannot interfere with the trial.
The endpoint creates the skill and config atomically, always opts the scout into the report channel, and safely re-applies config fields when the same definition is retried.
A scout with a product-specific output must stay off the report channel, so create it with skill-create and scout-config-create instead; see Scouts with a product-specific output.
Otherwise the coordinator auto-registers an enabled config on the default every-24-hours schedule on its next tick (up to ~30 min).
For an existing scout, tune with posthog:scout-config-update (find the id via -config-list):
run_interval_minutes— 30 to 43200. Default 1440 (every 24 hours). Slow a chatty or expensive scout by raising this.run_cron_schedule: a five-field cron expression ('30 9 * * *','0 9 * * 1-5') evaluated in the project's timezone; occurrences must be at least 30 minutes apart. When set it takes precedence overrun_interval_minutes, so a "slow it down" edit on a cron scout has to change or clear the cron (setnullto return to the rolling interval). A new or edited schedule anchors on the edit time and waits for its next slot rather than catching up on a past one. Reach for it when the run should land at a wall-clock time: a digest before standup, a weekday-only watch.- A scheduled run starts inside a window, not at an exact minute.
Each scout has a fixed offset in that window, so its cadence does not change from run to run.
A cron scout starts within 20 minutes of its cron time, so a
0 9 * * *scout starts at the same minute between 9:00 and 9:20 every day. An interval scout starts within the same 30-minute coordinator tick, andlast_run_atrecords the tick, not the start. A scout's first run, a manual run, a workflow run, and a report-check run start at once. When you explain or write a schedule, describe the time as a window. enabled—falsepauses the scout entirely (coordinator skips it).precheck_query— defaults to null: a HogQLSELECTthat a scheduled run evaluates before it starts, with{since},{now}, and{interval_minutes}placeholders. No rows, or one row with one false value (such as a zerocount()), skips the run before it creates a run row or a sandbox. Any other result starts the run, and the rows enter the run prompt. A query error also starts the run. Only scheduled runs evaluate it: a manual, workflow, or report-check run always starts. Set it for a scout that has nothing to do on most ticks. Test the query withposthog:scout-config-precheck-testbefore you save it, because a scheduled check runs as the scout's acting user and the dry run runs as you. A canonical scout can ship a default in itsscout-precheck-queryfrontmatter.precheck_queryoverrides that default, null restores it, andprecheck_disabled: trueturns off both. The full recipe, the placeholder semantics, and how to diagnose a tick with no run row are inreferences/scheduled-precheck.md.emit— defaults totrue: the scout writes its reports straight to the inbox. Setemit=false(dry-run) to inspect an ordinary run's reasoning without publishing reports to the inbox. Private trials capture reports and memory separately and work while the source scout is paused or hasemit=false.network_access— defaults totrusted: the scout's sandbox can only reach the platform's trusted-domain allowlist (PostHog, GitHub, common package registries), which covers the MCP loop andghbut blocks everything else. Setfullfor a scout whose skill needs to read arbitrary external sites, e.g. documentation, papers on arxiv.org, or a vendor status page. Applies from the scout's next run, and changes are activity-logged.auto_pause_exempt— defaults tofalse. A scout whose reports nobody engages with (no open, rating, or action — the cloud web inbox records reads; other clients don't yet) is warned and then paused automatically (pause_reason=ignored) — every run costs a sandbox agent, so a scout producing output no human consumes shouldn't keep running forever. A scout that is merely quiet is only flagged (pause_reason=no_output, a warning that never advances to a pause), since a watch scout's silence can be its job.-config-listshows the warning asstatus=pending_pauseand the pause asstatus=paused_by_system; settingenabled=trueagain resumes the scout with a fresh grace window before the sweep may judge it again. Setauto_pause_exempt=trueup front for a watchdog scout whose whole job is to stay quiet, so it never even picks up the quiet flag.write_scopes— defaults to["notebook:write"]: the scout reads the project and writes its findings, its memory, and notebooks.notebook:writereaches every notebook in the project, not only the ones the scout made. Send[](or drop it from the list) for a scout that should write nothing but its findings and memory. Unlike the other scopes, a run keeps it when its author no longer resolves and it acts as another project member. Grantdashboard:write,insight:write,annotation:write,alert:write,customer_task:write,llm_skill:write,warehouse_view:write,warehouse_table:write,replay_scanner:write,canvas:write, orexperiment:writeto a scout whose job is to maintain one of those things rather than only describe what it would change. The body of such a scout follows the maintainer / steward pattern inreferences/scout-patterns.md: a curated inventory, a write ladder (do-and-log, do-verify-report, hand-to-a-human), a per-run change cap, and a rolling maintenance report naming every object changed. Each scope is project-wide and covers update and delete of every object of its kind, not only the ones the scout made, so grant only what the scout's body actually tends, and say in the body what it may change and when.llm_skill:writeis the one to think twice about: custom scouts are skills in the same store, so a scout holding it can edit a sibling scout's body, or the body it runs from itself. Grant it to a scout whose job really is tending a set of skills, name that set in the body, and say there that the scouts are off limits unless tending them is the job.warehouse_view:writeandwarehouse_table:writeare separate on purpose: a scout that keeps a set of views healthy does not also need to create tables. Take both rows only when the scout tends both.replay_scanner:writepermits scanner maintenance. A scout must set acredit_limitwhen it creates, copies, or enables a scanner, and before it changes targeting, sampling, or the model of an enabled scanner. It cannot clear a limit, delete a scanner, or start manual scans, retries, or backfills. Useenabled: falseto stop a scanner and keep its observations. Change a shared rating only to record an explicit user verdict, because ratings steer how the scanner judges later sessions.customer_task:writecovers every Customer analytics task in the project, including tasks assigned to other people: create, update (status, due date, assignee), archive, and restore archived tasks, with no delete. Say in the body whose tasks the scout tends, and whether it may reassign one. Customer analytics tasks are still behind the `custom