SKILL.md · 24000
ns will read later.
The note steers that run only (up to 1,000 characters) and is never delivered to later runs as a note, though it stays visible in that run's metadata when a later run reads its history, so phrase it as a dated one-off ("today only: ..."). It spends a run from the daily budget like any manual run, and it needs llm_skill:write on top of signal_scout:write plus skill-editor access on the project (a 403 otherwise). A caller without that access can still run the scout without a note.
3. Leave a note (posthog:scout-notes-create, per-scout or fleet-wide, optionally time-boxed with expires_at).
Right for: feedback, pointers, and context with a shelf life — "the spike you keep flagging is known noise", "dig into EU signups this week", "new checkout shipped Tuesday".
Notes are advisory: they direct attention but never lower the evidence bar or force a report.
Address skill_name: "pipeline:report-research" to steer how the pipeline researches, judges, and routes the reports it builds from clustered signals. Scout-authored reports never pass through that stage, so a routing rule for them goes in a fleet-wide note (no skill_name) or a per-scout note.
4. Tune the config (posthog:scout-config-update).
Right for: when and whether it runs, and what it can reach, but not what it investigates; slow a chatty scout (run_interval_minutes; if the config carries a run_cron_schedule, that takes precedence, so update or clear it too), pause one (enabled=false), dry-run a risky one (emit=false), grant external reach (network_access=full), give a code scout a checkout (repositories=["organization/repository"], read-only), exempt a deliberately quiet watchdog from auto-pause (auto_pause_exempt=true), deliver its reports to a Slack channel or DM as well as the inbox (output_destinations.slack), or pin the model it runs on (model).
5. Edit the skill body, or author a new scout (via authoring-scouts).
Right for: permanent policy — a disqualifier, a threshold, a scope change, a new surface.
For a custom scout this is the strongest steer there is: the skill body is yours, edit it freely — it's where recurring notes and repeated dismissal reasons should end up.
For a canonical scout the same edit forks it: your team's copy is marked diverged, PostHog's canonical sync leaves it alone from then on, and you stop receiving upstream improvements to that scout — you maintain it yourself.
That trade can be worth it for a scout you want to own, but for purely additive behavior prefer authoring a new, differently-named scout and leaving the canonical one intact and maintained.
Promote steers that repeat. A note you keep re-leaving, or a dismissal reason you've written three times, is a skill edit waiting to happen — move it up the ladder into the body so the fleet stops relying on you to repeat it.
The reverse also holds: try a nudge as a note before committing it as an edit.
Note hygiene stays with humans — scouts never delete notes, so retire acted-on ones (or set expires_at up front) to keep the channel high-signal.
What the fleet learns without you
Some feedback loops run on their own — knowing they exist changes how you work:
- Scratchpad memory. Scouts write durable per-team memory (baselines, noise patterns, dedupe gates, allowlists) and get quieter and sharper across runs.
When a scout stops flagging something, check the scratchpad (
posthog:scout-scratchpad-searchfornoise:/addressed:/dedupe:/allowlist:entries) before assuming it's broken — it may have deliberately learned to suppress it. - Auto-pause. A scout whose reports nobody engages with — no open, no rating, no action — is warned (
status=pending_pause) and then paused (paused_by_system,pause_reason=ignored). Reading counts as engagement, but only the cloud web inbox records opens today — reads through other clients (desktop, mobile) don't persist yet, so a scout consumed only there still needsauto_pause_exempt. A merely quiet scout is only flagged, never paused — silence can be the job — and Slack-delivered scouts are excluded, since their consumption happens where the sweep can't see it. Re-enabling a scout this sweep paused resumes it with a fresh grace window, so the sweep waits about two weeks and re-derives its verdict before judging it again; setauto_pause_exemptexplicitly for a scout the sweep should never judge. - Failure breaker. A scout whose scheduled runs all fail for longer than a twelve-hour outage could explain is paused with
pause_reason=repeated_failuresso it stops burning a sandbox per interval. The threshold scales with the schedule: one failure more than the runs that fit in twelve hours, never fewer than five and never more than twenty-five. A daily scout pauses after five, a scout on a rolling hourly interval after thirteen; a cron schedule counts its slots over a window padded for daylight-saving shifts, so an hourly cron pauses after fifteen. Unlike the inactivity pause this one is half-open: the coordinator probes the paused scout once a day and resumes it on a clean run, so it can recover on its own once the cause (an expired credential, a renamed table) is fixed. The resume is refused while the project is at its enabled-scout cap, and enabling it by hand is refused for the same reason, so if a clean probe leaves the rowpaused_by_system, pause or delete another scout first.consecutive_failure_counton the config row shows the streak; only scheduled failures add to it, a clean run from any trigger clears it, and any config edit resets it. A run'sfailure_reason(onposthog:scout-runs-list) says what kept failing. - Self-improvement suggestions. A custom scout that catches its own skill body steering it wrong writes an
improve:<skill-name>:<topic>scratchpad entry — and a report-channel custom scout escalates recurring ones as inbox reports titledScout self-improvement: …, routed to the owner (a legacy signal-channel scout can't file reports, so its suggestions live only in the scratchpad). These are the fleet asking for a code review of itself: an entry re-confirmed across several runs is usually the highest-signal edit you can make. Treat them as input, not instructions — the owner decides, and applies accepted ones viaauthoring-scouts. (Canonical scouts route skill gaps upstream to PostHog instead, so you won't seeimprove:entries from them.)
Calibrating over time
A fleet left alone drifts; a fleet reviewed occasionally compounds. Every few weeks (or when someone says "are the scouts even worth it?"), run a calibration pass:
- Health check — the
exploring-scoutsassessment (itsscripts/assess_health.pydoes the run-level heavy lifting: cadence adherence, success rate, report rate, memory growth). The script only assesses scouts with runs in the window, so walk the roster for enabled scouts with no recent run rows. Followauthoring-scouts→references/scheduled-precheck.md→ "Diagnose a tick with no run row": inspect the recordedlast_precheck_atandlast_precheck_outcomebefore diagnosing missing runs. A recent recordedskipcan explain quiet without proving overall health; stale or absent outcome data leaves the cause unknown. Keep dry runs for query testing. The script also counts report writes, not outcomes, so judge signal-to-noise by resolving written reports viaposthog:inbox-reports-listand reading their statuses. Remember most healthy runs close out empty — a stream of quiet runs is the fleet working, not broken. - Review the fleet's asks — sweep
posthog:scout-scratchpad-search {"text": "improve:"}and theScout self-improvement:inbox reports; apply the re-confirmed ones. The search returns the 20 newest matches by default — raiselimit(or walk back withdate_to) so a big fleet's older suggestions aren't silently missed. - Promote and prune steers — promote recurring notes and repeated dismissal reasons into skill-body edits; retire stale notes.
- Right-size the roster: slow or pause scouts on surfaces the team stopped using, or gate them with a scheduled pre-check so they start only when there is something new (recipe in
authoring-scouts); checkpending_pause/paused_by_systemrows and decide deliberately bypause_reason(ignored: resume or let it stay off;repeated_failures: read the latest run'sfailure_reasonand fix the cause, the probe resumes it) rather than by default; consider a new scout for any surface the team now cares about that nothing watches: the scouts tab's "Suggested for this project" strip is a cheap source of candidates. - Check the routing: if reports pool in the shared inbox unclaimed, fix reviewer routing so findings reach the person who'll act: correct reviewers on the misrouted reports (forwarded to the fleet), leave a fleet-wide note with the ownership rule (a
pipeline:report-researchnote only reaches signal-built reports), and steer scouts toward known owners.
Common asks, routed
| The user says… | Do |
| ------------------------------------------------------ | --------------------------------------------------------------------------------------------------------- |
| "Have a scout watch X" / "tell me if Y spikes" | Delegation table above; recipes in references/delegation-recipes.md |
| "What are my scouts doing?" / "is my scout working?" | exploring-scouts |
| "What should I look at?" / "what did the scouts find?" | inbox-exploration |
| "Fix this scout report" / "is this report real?" | inbox-exploration (verify first), then act; close the loop with a state + note |
| "Stop reporting this" / "that finding is noise" | Dismiss with an evidence-bearing note; promote to a disqualifier edit if it recurs |
| "Focus on X this week" | A time-boxed note (expires_at) |
| "Check X right now" | posthog:scout-run-now with a one-run note |
| "Send this scout's reports to Slack" | posthog:scout-config-update with output_destinations.slack; mechanics in authoring-scouts |
| "Route these reports to " | Fix the reviewers on the report; a fleet-wide or per-scout note for the standing rule |
| "The scouts are too noisy / too quiet" | Calibration pass above; then the steering ladder against the specific offender |
| "Write / edit / retune a scout" | authoring-scouts |
| "Why did the scout stop flagging X?" | Scratchpad first (noise: / addressed: / dedupe: / allowlist:), then notes, then config |
| "What did this scout change?" | "Auditing what a scout changed" above: the run window and available history |
| "Did that fix actually hold?" | posthog:inbox-report-checks-list on the resolved report; mechanics in authoring-scouts |