AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

authoring-scouts

Write, edit, and adapt PostHog Signals scouts: scheduled agents that scan a project and report findings. Use to change a canonical scout's scope, thresholds, schedule, or dry-run settings; add a scout for an uncovered surface; record structured output from scheduled LLM scoring; or give feedback through notes. Covers SKILL.md structure, report requirements, follow-up report checks, structured output, duplicate detection, scratchpad memory, scout notes, per-team skills-store and canonical repository edits, and private prompt trials with saved rubrics. Trigger on "write/edit/customize a signals scout", "new scout for X", "tune my scout schedule", "make a scout that watches <event>", "score/judge/measure X with a scout", "test/compare scout prompts", "generate scout rubrics", "structured output from a scout", "scout output to Slack", "leave a note for / give feedback to a scout".

Original source
posthog
License
MIT
Pinned revision
cac7c80a9e5b73484d18eda1bffb47297c938e0c

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

Ask your assistant

Use Agentic Memory skills to read posthog--authoring-scouts at revision cac7c80a9e5b73484d18eda1bffb47297c938e0c, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 8

references/report-contract.md · 48000

heaper paths above don't resolve an owner, and the way a team slug becomes reviewers. It returns this project's members, each with user_uuid, email, name, a resolved github_login, and the teams they're on. Pass search= to narrow by name or email. Match the owner and route with user_uuid. The org-scoped org-members-list / org-member-get-github-login tools are not available in a scout run — a scoped-team token can't reach the org-nested endpoint, so don't build a scout's reviewer recipe around them.

Resolving a team slug to reviewers. Call scout-members-list with team=<slug> (bare slug, no @your-org/ prefix, case-insensitive). It returns the members of that team with its maintainers first, so:

  • Take the first 1 to 3 rows and route them. Three is the cap suggested_reviewers enforces anyway, and past the maintainers the order carries no ownership signal, so a longer list dilutes rather than widens.
  • Prefer one reviewer when a maintainer is clearly the owner of the area. Add the next one or two only when the work spans the team.
  • Route each with user_uuid, the same as any other reviewer.

Two things the roster can't tell you, which change what you should do rather than what you should report:

  • A slug with no rows means "not synced here", not "no such team". The GitHub teams and team_members schemas are off by default and need the organization Members permission, so coverage is partial on most projects. The tool returns an error saying which case it hit. Fall back to matching the owner by name or email, and don't write a report claiming the team doesn't exist.
  • The roster is a snapshot, so it can lag the live team. Someone who joined or left since the last sync is wrong here. Treat a surprising result as stale data, and cross-check against a recent author or an inbox precedent before routing on it alone.

If you can't confidently identify a reviewer, leave suggested_reviewers empty — the report still surfaces for a human to grab. Never guess a handle: a wrong login mis-assigns the report (or silently fails to assign), which is worse than leaving it open. And remember edit_report can set reviewers on a report later — so a report that surfaced routed to no one isn't stuck; once you resolve an owner, edit it in (which also re-runs autostart).

Cache for next time. After you confidently tie an area to an owner, write a reviewer:<domain>:<area> scratchpad entry with the bare lowercase login so the next run — and sibling scouts — route faster. The fleet's reviewer map should compound over time.

edit_report — update an existing report

Rewrite title/summary, append evidence or a note, set suggested_reviewers, correct the repository, change the work decisions (actionability, priority), and/or replace metrics / charts / suggested_prompts on a report that already exists. Pass run_id (the current run) and report_id, plus at least one of title, summary, append_note, append_evidence, suggested_reviewers, repository, actionability, priority, metrics, charts, suggested_prompts. An edit that supplies content (title, summary, metrics, charts, suggested_prompts, append_note, append_evidence, or a reviewer reason) passes the same safety judge as emit_report; an unsafe edit is rejected whole and the report keeps what it had.

edit_report can target any of the team's inbox reports — not just ones a scout authored. That makes it the right tool when a later run learns something about a report the pipeline (or another scout) created. Rules of good behavior:

  • Use append_evidence for a new observation that a reader can check. It takes the same {description, source_id} items as emit_report, and each one lands in the report's evidence rail as a bound signal, so the report's signal_count and total_weight grow with it.
  • Use append_note for commentary — a reading of the report that adds nothing to check, such as the owning team already knowing, or a deploy having fixed it. Send both in one call when an observation needs a reading alongside it.
  • A recovery is a note, not evidence. signal_count and total_weight only grow, and both feed the inbox ranking, so evidence that an issue is over would rank the report as stronger.
  • At the cap, the note is the channel that still lands. Emit plus every append share the report's 50 evidence rows, and the grouping pipeline can raise the count too, so a long-lived report can fill up. An append past the cap is rejected and the report keeps what it had.
  • Prefer these additive fields over rewriting title/summary on a report you didn't author. A note is additive and audit-friendly (it carries your scout as the author); a rewrite silently overwrites a human- or pipeline-authored headline.
  • Don't fight an in-flight pipeline. A report the summary/research workflow is mid-run on can have its fields overwritten under you. If a report is actively being worked, append a note rather than rewriting.
  • Take the questions down when you replace the prose they answer. Rewriting summary leaves the report's suggested_prompts in place, and they were written against the summary you just replaced — send a fresh set in the same call, or [] to clear them.
  • Use suggested_reviewers to rescue an unrouted report. Setting reviewers (same {github_login?, user_uuid?} shape as emit_report) replaces the report's reviewer list and re-runs autostart — so a report that surfaced routed to no one can be assigned to an owner you resolved later, and a now-actionable report with a repo + priority can open a draft PR. An empty list is a no-op (it never clears existing reviewers).
  • Use repository to fix a report that points at the wrong codebase. Pass owner/repo to replace the target, or the NO_REPO sentinel when nothing under version control can change. Omit the field to keep the current target. Correct the report in place instead of filing a duplicate. The change re-runs autostart, so a report that had no repository can now open a draft PR.

Changing the work decisions

edit_report can replace the two decisions that control autostart. Each decision is a set of paired fields, and you send the full set together:

| Decision | Fields to send together | | ------------- | --------------------------------------------------------------------- | | Actionability | actionability, actionability_explanation, and already_addressed | | Priority | priority (P0–P4) and priority_explanation |

  • An explanation is 2–3 sentences of evidence. The edit fails when you set a decision without its explanation.
  • already_addressed is part of the actionability decision, so it needs actionability and actionability_explanation in the same call. When you omit it, the decision records false.
  • A decision change re-runs autostart. It does not change the report's inbox status. immediately_actionable with a priority can open a draft PR. requires_human_input and not_actionable stop autostart from opening one.
  • decision_fields_set in the response names the decisions the edit replaced. It is empty when you re-send the decisions the report already holds.

Mark in-flight work so that autostart does not compete with it. When a fix has landed, or a PR for the issue is already open, set already_addressed: true with the current actionability and an explanation that names the fix. Autostart then does not open a second, competing implementation. Do this instead of a note alone: a note records the fix for a reader, but autostart does not read it.

Replacing the report's pull request

A report that autostarted has an open draft PR built from the summary as it read at the time. When your rewrite changes what the fix should be, set supersedes_implementation: true alongside the title / summary you are changing. This records a replacement decision for a ready report. Autostart checks policy and eligibility before starting a replacement from your new summary. Technical failures retry automatically; a policy block waits for a new edit or research trigger. The existing PR stays open until the replacement succeeds with a verified open PR.

Set it only when the fix itself changed: a different root cause, a different file or layer, a materially wider or narrower scope. More evidence for the same fix is not a reason — the open PR already implements it, and replacing it throws away review someone may already have done. An append_note is the right move there instead.

Two things bound it, and the response tells you which one applied:

  • It is only honored alongside a rewrite that actually changed the title or summary. Restating the text the report already holds is not a revision, and neither is a note or a reviewer change. is_content_revision in the response says whether yours counted.
  • Only the first four content revisions can request replacements. content_revision_count counts every title or summary rewrite, including ones that did not request replacement. Past four your rewrite still lands, but it cannot request a replacement. supersedes_implementation in the response is true only when the decision was recorded.

Re-confirming a report you already filed

Appending a note that says the finding still holds is worth doing, and it is not a revision — it leaves content_revision_count alone. Free-form append_note text always remains in the work log, including recovery details and observations beyond the evidence cap. Set corroboration_only: true only for a confirmation with no new information. A report keeps its first four confirmations as separate entries and counts later confirmations; the web and desktop inboxes show the collapsed count. The call still succeeds, and corroboration_collapsed in the response tells you it happened.

A replacement request that cannot bind verified predecessor PRs, or whose report changes during verification, fails without saving the edit. Retry the same edit to resolve the context again.

Finding "the report I made last time"

There is no scout-specific report search — use the vanilla inbox tools the scout already has. Before authoring, list the team's existing reports so you reconcile against one instead of filing a duplicate:

  • inbox-reports-list — filter by title/summary free-text (search), status, source_product, or your own task_id; newest-updated first.
  • inbox-reports-retrieve — fetch a single report by id (use the report_id you stashed in the scratchpad last run).

Dedup: the retry is covered, the near-duplicate is not

emit_report carries an emit key, so resending a call that timed out returns the report the first one authored (idempotent_replay true) rather than a twin. The key is the idempotency_key you pass, or the report's own content when you pass none, and it is scoped to your run. Pass one when a retry might reword the report, since a reworded report is a different content key.

That barrier covers the transport failure and nothing else. A report on a topic an earlier run already filed is a fresh emission with a fresh key, so the cross-run dedup is still two-sided and the scout owns it:

  1. Before authoring, inbox-reports-list for a prior report on the same topic. Found one? edit_report it instead of authoring a new one.
  2. After authoring, write a report:<domain>:<entity> scratchpad entry recording the report_id so the next run finds it (via inbox-reports-retrieve) without a title-search guess. (This is the report-channel member of the scratchpad key-prefix vocabulary — see dedupe-and-memory.md.)

edit_report has no such barrier: never retry an edit_report call that may have succeeded, since a transport error after the write commits, retried, appends a second note. If you're unsure whether an edit landed, inbox-reports-retrieve to check before retrying.

The pipeline may rewrite what you authored (accepted)

An authored report is a first-class SignalReport that coexists with pipeline reports. When future signals consolidate around the same topic, the pipeline may re-promote and re-research the report, overwriting your authored title/summary. This is accepted behavior, not a bug — there is no pin. Don't author a report assuming your exact prose is immutable; author the finding, and let the inbox stay the source of truth for how it's currently framed. Your durable record of "I filed this" is the report: scratchpad entry and the report_id, not the title text.

Granting the tools

In the scout's SKILL.md frontmatter, list the report tools under allowed_tools:

allowed_tools:
  - emit_report
  - edit_report

Every report-producing scout needs this. A report-producing scout that omits it falls back to a deprecated legacy channel (weak emit-signal findings a pipeline consolidated) and can't write reports at all. Don't author a new report-producing scout without the opt-in. If you find an existing report-producing scout missing it, add it and rework the scout's Decide section onto this contract. The canonical fleet runs on this channel; signals-scout-anomaly-detection's references/report-contract.md keeps a worked, surface-specific shape (its notebook write-up + embedded-chart recipe). Add a short body section telling the scout what's report-shaped for its surface. Keep it lean — the field-level detail lives here (and in the harness prompt), not in the body.

Rollout posture: for a chatty or high-stakes new scout, start in dry-run (emit=false on its SignalScoutConfig) so it runs and logs what it would author without writing to the inbox. Inspect via scout-runs-retrieve, calibrate, then flip emit=true. The channel files a full inbox item on the first hit, so the cautious loop is worth it when in doubt.

Scouts with a product-specific output

Some scouts write their output to a product surface, not to the inbox. Such a scout omits both emit_report and edit_report, and it does not call emit_signal.

signals-scout-workflows is the example. It files suggestions through the workflows suggestions API, and the suggestion on the workflow page is its output. Its frontmatter description and its "Why this scout files no reports" section state that this is deliberate. A report on a workflow change could autostart a code PR for a change that is only PostHog configuration.

Before you add the report tools to a scout that does not have them, read its description and output section. If the scout states a product-specific output, keep it off the report channel.

To create such a scout per team, do not use scout-create, because it always grants both report tools. Make both calls below on the parent project, not on a child environment. skill-create stores the skill in the active environment, but scout-config-create looks for the skill in the parent project. From a child environment, registration fails, or it registers a different skill with the same name from the parent project. If the active context is a child environment, select the parent project with switch-project first. Some MCP contexts do not expose switch-project, for example a connection with a filtered tool set. In such a context, connect to the parent project directly. If you cannot, do not use this path from that context. Create the skill with skill-create and leave allowed_tools empty. Then register it with scout-config-create, and set its schedule, enabled, and emit on that call. Also set write_scopes on that call to the scope that writes the product output, for example ["hog_flow_proposal:write"] for workflow suggestions. Without that grant, the scout cannot write its output. A dry run (emit: false) never holds the grant, so the scout writes no output in a dry run. Give the skill a name without the signals-scout- prefix. The coordinator auto-registers an enabled config for a prefixed skill that has no config, so a prefixed scout can run on the defaults before your scout-config-create call.