AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

authoring-scouts

Write, edit, and adapt PostHog Signals scouts: scheduled agents that scan a project and report findings. Use to change a canonical scout's scope, thresholds, schedule, or dry-run settings; add a scout for an uncovered surface; record structured output from scheduled LLM scoring; or give feedback through notes. Covers SKILL.md structure, report requirements, follow-up report checks, structured output, duplicate detection, scratchpad memory, scout notes, per-team skills-store and canonical repository edits, and private prompt trials with saved rubrics. Trigger on "write/edit/customize a signals scout", "new scout for X", "tune my scout schedule", "make a scout that watches <event>", "score/judge/measure X with a scout", "test/compare scout prompts", "generate scout rubrics", "structured output from a scout", "scout output to Slack", "leave a note for / give feedback to a scout".

Original source
posthog
License
MIT
Pinned revision
cac7c80a9e5b73484d18eda1bffb47297c938e0c

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

Ask your assistant

Use Agentic Memory skills to read posthog--authoring-scouts at revision cac7c80a9e5b73484d18eda1bffb47297c938e0c, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 8

references/report-contract.md · 24000

d on both, because the total comes from the series math alone.

Keeping the query live and bounded

The query is the source of truth; it runs again every time a reader opens the report. So attach a query you ran successfully in this session, the same rule the charts carry.

  • One InsightVizNode wrapping one TrendsQuery. Any other node kind is refused at write time.
  • Event and action series only. Every source series must be an EventsNode with a non-empty event, or an ActionsNode with a positive integer id. This is what lets the server check a reader's access without an unbounded query fan-out.
  • A relative window that advances with time. dateRange.date_from must be relative (-13d, -30d), at most 366 days, and date_to must be empty. Default to date_from: "-13d" with interval: "day", which gives 14 inclusive daily buckets including today. interval accepts second through year.
  • At most 1,000 estimated longitudinal points, counting the current partial bucket. One hourly series over roughly six weeks fits.
  • Exactly one output series. Without a formula that means exactly one source series. A rate may combine up to 10 source series with exactly one formula. A breakdown and compare mode are both refused, because either can multiply the output at run time.
  • Keep the filters that reproduce the observation, so the figure measures the thing the report describes.

Consumers own the display. The report derives two shapes from the stored query: BoldNumber for the whole-window aggregated_value, and ActionsBar for the longitudinal buckets. The display you author does not control how the report draws the metric. Run the total-value shape when you measure a snapshot, because a bar or line response gives no whole-window total. Never sum distinct-user buckets, since one person can appear in several.

Keeping semantics separate from formatting

kind says what the metric measures. value_format says how to print it.

  • A non-currency unit is one lowercase word that completes the figure, because the report prints it next to the number: users, sessions, events, runs, calls.
  • For a rate, name what the share means: failure for an error rate, conversion for a conversion rate. % is redundant and is dropped.
  • Use percentage for percentage points (34 means 34%) and percentage_scaled for 0–1 ratios (0.34 means 34%).
  • A percentage query must set aggregationAxisFormat to exactly the same value as value_format. A missing or numeric axis format is invalid.

Titling and captioning

The tile prints the figure, the unit, the title, and the window together, so never state one fact twice across them.

Leave caption empty unless it carries something the reader needs and cannot see:

  • a filter that narrows the count (Production only, excluding internal users),
  • why a longer window was needed,
  • a caveat on the data (Person context is missing on about a third of these events).

A caption that restates the title, the unit, or the window is noise. The strip shows at most the trailing 14 buckets, so a window longer than 14 days needs the caption that explains it.

Snapshots are optional cached fallbacks, not estimates

A snapshot keeps an inbox list read cheap. It never replaces the required live query.

  • Send a snapshot only for a value you measured in this run. Zero is a real measurement; null means unavailable.
  • A count snapshot and series must be non-negative whole numbers. A rate snapshot must sit inside its format's range (0–100 for percentage, 0–1 for percentage_scaled).
  • Never author a snapshot-only or queryless row. Those shapes are legacy or malformed, and the server always redacts them.
  • Leave comparison unset. The server does not yet keep an adjacent comparison window live, so an authored comparison is stored and never shown.

A refresh replaces value, value_at, and series, and clears comparison. It runs only when a client posts to the report's refresh_metrics endpoint; reading a report does not refresh anything on its own. Listing reports never executes a metric query: a row carries the metric metadata and any readable cached snapshot, but no query definition.

A reader may see the tile without its data

A metric snapshot is measured without a requesting user, while a report read is authorized as the viewer. When the server cannot prove the two are equivalent, the metric stays in the response with its data-bearing fields redacted.

  • A cohort reference, including a nested one, hides the metric from every reader. There is no cohort object access policy to prove access against, so this is an intentional safety limit rather than a display bug. Keep a cohort filter out of a metric query.
  • A reader can also lose the query, the snapshot, or both for access reasons you cannot see while authoring: a property restriction, a missing token scope, an action they cannot read, or an unfamiliar filter shape.

So a metric is not a way to deliver a number to a reader who could not run the query themselves. Keep the figure the report's argument depends on in the summary prose, where every reader gets it.

Caps, and what an edit does

At most one affected_users metric, and 60,000 characters of query JSON across the report. Prefer the few measurements that change the decision.

metrics on an edit is the report's whole set, not an addition, exactly like charts. Omit the field to keep the metrics the report has, send the complete replacement list to change them, or send metrics: [] to clear them. Re-send a metric_id with a newer query or snapshot to replace that metric. Read the report first (inbox-reports-retrieve returns its metrics) when you mean to add one. A metric that comes back with a null query is redacted, not empty. You cannot re-send it, because the write shape requires a query, and leaving it out deletes that tile. Omit metrics altogether on such a report, and leave the change to a reader who can see every row.

When the flag permits a metric write, every metric title, caption, snapshot, and query goes before the safety judge, the same as the report prose.

Attaching charts

charts puts the data next to the claim, so a reader sees the move instead of taking the number on trust. Worth it when the shape is the point — a trend that broke, a distribution that shifted, a funnel step that collapsed. A chart restating one number the summary already gives is noise; just write the number.

| Field | Type | Notes | | ---------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | chart_id | string, required | Your own slug (lowercase letters, numbers, _, -). How the summary points at the chart, and the key a later edit refreshes it under. Unique within the report. | | title | string, required | Heading above the chart. | | query | object, required | An InsightVizNode, DataVisualizationNode (a HogQLQuery source, plus display and chartSettings for a graph), or SavedInsightNode (by shortId). Any other kind is refused at write time. | | caption | string, optional | One line on what to look at. | | size | enum, optional | small / medium / large. Leave it out unless the default looks wrong — the inbox sizes a chart from its query (a big single number gets a short box, a retention grid a tall scrolling one). |

A trends chart and a graph built from SQL, as they arrive in charts:

[
  {
    "chart_id": "exceptions-daily",
    "title": "Exceptions per day",
    "caption": "The step up starts on 18 June.",
    "query": {
      "kind": "InsightVizNode",
      "source": {
        "kind": "TrendsQuery",
        "dateRange": { "date_from": "2026-06-01", "date_to": "2026-07-02" },
        "interval": "day",
        "series": [{ "kind": "EventsNode", "event": "$exception", "math": "total" }],
        "trendsFilter": { "display": "ActionsLineGraph" }
      }
    }
  },
  {
    "chart_id": "exceptions-by-type",
    "title": "People affected, by exception type",
    "query": {
      "kind": "DataVisualizationNode",
      "source": {
        "kind": "HogQLQuery",
        "query": "SELECT exception_type, uniq(distinct_id) AS people FROM ... GROUP BY exception_type ORDER BY people DESC"
      },
      "display": "ActionsBar",
      "chartSettings": { "xAxis": { "column": "exception_type" }, "yAxis": [{ "column": "people" }] }
    }
  },
  {
    "chart_id": "exceptions-by-type-daily",
    "title": "Exceptions per day, by type",
    "query": {
      "kind": "DataVisualizationNode",
      "source": {
        "kind": "HogQLQuery",
        "query": "SELECT toDate(timestamp) AS day, exception_type, count() AS occurrences FROM ... GROUP BY day, exception_type ORDER BY day"
      },
      "display": "ActionsLineGraph",
      "chartSettings": {
        "xAxis": { "column": "day" },
        "yAxis": [{ "column": "occurrences" }],
        "seriesBreakdownColumn": "exception_type",
        "showLegend": true
      }
    }
  }
]

A graph from SQL needs its axes named. Setting display without chartSettings draws an empty box; chartSettings.xAxis.column and chartSettings.yAxis[].column say which columns of the result are which. Omit display altogether and the node renders the result table, which reads better than a chart for a handful of rows.

A graph from SQL needs one row per x-axis value. The x axis is built from the result rows in the order they arrive, so a query that also groups by a second dimension puts several rows at the same x position and the line zigzags instead of trending. Either aggregate the query down to one row per x value, or name the second dimension in chartSettings.seriesBreakdownColumn, which pivots those rows into one series per value of that column. For a time series per segment, an InsightVizNode wrapping a TrendsQuery with a breakdownFilter is usually cleaner than SQL.

Only the node's kind and its serialized size are checked on write. A well-formed node of an allowed kind carrying a broken query is stored without complaint, then fails to draw when a reader opens the report, and nothing reports that back to the scout. So a scout should attach a query it has already run in the same session, or point at an insight that already exists via SavedInsightNode, rather than composing a node from memory. This is the single most useful thing to reinforce in a scout body that leans on charts.

A chart query must not carry anything executable. HogVM bytecode (what conditional formatting compiles to), a nested HogQuery, and sendRawQuery are each refused with a 400 wherever they sit in the node, because a chart renders data rather than running code in the reader's session. A nested SuggestedQuestionsQuery is refused the same way, for cost rather than execution: its runner calls an LLM, so a chart carrying one buys a completion every time a reader opens the report. A query over a warehouse connection is fine as long as it goes through HogQL: keep connectionId, drop sendRawQuery. So a direct-warehouse query you ran with the raw-SQL bypass has to be rewritten before it can be attached.

Placement comes from the summary. A markdown link with a chart: target — [Daily signups](chart:signups-drop) — draws the chart at that point in the body; a chart you never reference still renders, after the prose. Reference each chart once: a repeated reference reads as pointing back at the chart, not as asking for a second copy of it. Two references in one paragraph sit side by side, so put a pair you want compared in a paragraph of their own. A reference inside a code span, a table cell, or a heading has no room to draw — its chart falls to the end of the report instead.

The summary has to read without the charts. A report can also be delivered to Slack, where each reference degrades to the plain label it was given and the charts follow the prose as images rather than sitting inline. Only InsightVizNode and SavedInsightNode charts render there, at most three per report with referenced charts first; a DataVisualizationNode chart shows only in the inbox. "Signups fell 60% over the week" survives that; "the chart below shows the drop" leaves a Slack reader with nothing.

Pin the window to absolute dates wherever the node supports it, so a reader opening the report days later sees the data you wrote about rather than whatever a relative range resolves to then. This holds for charts alone. A metric and a follow-up check measure the period before each run, so each one needs a relative dateRange.date_from and an empty date_to. An absolute window is refused there.

charts on an edit is the report's whole set, not an addition. It replaces what the report had, the way summary replaces the summary — so send every chart you want kept, and re-send an id under a newer window to refresh that chart. Leave charts out entirely and the report keeps the ones it has; read the report first (inbox-reports-retrieve returns its charts) when you mean to add to them. Send charts: [] to take every chart down, for when the finding has moved on and the old chart would now mislead. Cap is 20 charts per report (and a combined query-size budget), which is far more than most reports should use. Each chart runs its query when the report is opened, so attach the ones that carry the argument rather than everything you looked at: three charts a reader studies beat a dozen they scroll past.

Suggesting follow-up prompts

suggested_prompts are prompts the inbox offers above the report's Ask AI box: follow-up questions, and next-step actions the reader can send as a request. Clicking one fills the box with it; nothing is sent on the click, so the reader can send it as written or edit it first. You did the research and know which threads you left open and what should happen next, so this hands the reader that knowledge instead of leaving them to invent a prompt from an empty box.

Optional, and worth it only when you can name a prompt worth an agent run. Write none rather than pad to the cap — a report with no suggestions looks exactly as it did before.

Ask what your research left open, not what it already answered. A question the summary answers spends an agent run restating the report. Good ones widen the finding (who else is affected, since when, what changed), test a hypothesis you could not, or ask for the next step you did not have the standing to take.

Offer the action your report recommends, so acting on it is one click. The prompt reaches an agent run that can investigate, carry out the report's recommendation, and work the report itself — its work log and its state — so a good action prompt names the concrete work: "Create the alert the report recommends, then mark this report resolved". Fold in "mark this report resolved" only when the action completes in place — an action that lands as a pull request must not resolve the report, because a caller resolve closes the report's open PR and the merge resolves the report on its own. Suggest only actions your report's own recommendation makes concrete; leave anything a human should weigh first (deleting data, changing a flag serving live traffic) as a question instead.

Write each prompt as the reader would send it, in their words — a question they would ask or a request they would make — and make each one stand alone: the prompt reaches an agent that gets the report as context but not your run, so it can't point at "the above" or "the second chart".

suggested_prompts on an edit is the report's whole set, not an addition. It replaces what the report had, the way summary replaces the summary, so re-send every prompt you want kept. Leave the field out and the report keeps the ones it has; send suggested_prompts: [] to take them down. Rewriting summary on an edit does not clear them for you, so send the new set (or []) in the same call. The research pipeline does clear them when it rewrites a report it re-researches, since the prompts were written against the prose it replaces.

Cap is 3 prompts per report, each ≤200 characters, and duplicates are refused.

Suggesting a product to turn on

When a report would have had better evidence from a product the project does not use, record a source_suggestion artefact on it. After emit_report or edit_report returns the report id, call inbox-report-artefacts-create with artefact_type: "source_suggestion" and content: {"product": ..., "reason": ...}. The tool needs the task:write scope. If the session does not have it, continue without the suggestion. product is one of logs, session_replay, error_tracking, or llm_analytics. The inbox shows the suggestion under the report's evidence with a link to that product, and hides it once the project uses the product.

Suggest a product only for a gap you hit this run: a question it would have answered that you could not answer. Confirm first that the project does not use it, from a not-in-use: memory or a probe that came back empty. reason is one sentence of at most 300 characters about what the product would have shown for this report, not a generic pitch. A report shows one suggestion, and a newer one replaces the older one.

Opening a draft PR (autostart)

A surfaced, immediately-actionable report can open a draft PR automatically — the same autostart path the pipeline uses. It's opt-in per report via three more emit_report fields; supply them only when the report is a concrete, fixable issue you'd want a PR for:

| Field | Type | Notes | | ---------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | repository | string | "owner/repo" targets that repo; omitting it falls back to free-form selection; the NO_REPO sentinel opts out. See Choosing a repository below. | | priority | P0-P4 | Required for a PR. Pair with priority_explanation. | | priority_explanation | string | Required when priority is set. | | suggested_reviewers | list of obj | Reviewers to consider, each {github_login?, user_uuid?} (at least one per entry; see the section below). A PR opens only if at least one clears their autonomy threshold. |

Choosing a repository. Prefer owner/repo whenever you can say where a fix would land, including on a requires_human_input report — a repository does not open a PR by itself, it is what lets a person open one from the inbox later. Omit the field when your team has several repositories and you can't tell which one, so selection can find it; that is the slow path on a many-repo team, since it spawns a selection sandbox. Keep NO_REPO for the rare report where nothing under version control could change, such as a staffing finding or a data question with no artifact. A skill body, a config file, and a doc all live in a repository, so "not code" is not the test.

Full repo selection only runs when you signal PR intent — an explicit repository, or both priority and suggested_reviewers. A report that supplies none of these just surfaces in the inbox: no repo sandbox, and no PR. It still gets a repo target when its own text links exactly one repository the team has connected on GitHub, so someone reading it can click Create PR. That inferred target is for a person to act on — it never opens a PR by itself, and rewriting the report's title or summary to link a different connected repository moves it. Adding a qualifying reviewer later is a person asking for the PR, so the report can open a draft one from then on. Autostart itself still no-ops unless the report is immediately_actionable, has a repo + priority, and a reviewer qualifies — so these fields are safe to omit for an informational report.

Choosing suggested_reviewers — how a report gets assigned to a human

suggested_reviewers is not just a PR gate — it is the primary way a report gets routed to the right person internally. The inbox orders by is_suggested_reviewer, so a reviewer's own reports float to the top of their inbox; a report with the right reviewer reaches that human even when no PR is involved. Set it whenever you can name a plausible owner — including on informational requires_human_input reports, not only PR-bound ones. A report with no reviewer just sits in the shared inbox hoping someone grabs it.

Each entry identifies one reviewer by github_login, user_uuid, or both:

  • github_login — a bare, lowercase GitHub login (e.g. octocat, not @OctoCat). Internal assignment matches it against each user's linked GitHub login by exact, lowercased comparison, so a mis-cased handle, an @-prefix, a display name, a CODEOWNERS team slug, or an email won't set is_suggested_reviewer for anyone (autostart's PR-selection path is more lenient, but the assignment path is not).
  • user_uuid — a PostHog user UUID. The server resolves it to that org member. It wins if you also pass a github_login. Use this whenever your evidence already names a PostHog user. It works without a linked GitHub account. A user_uuid that is not an org member of this team is rejected, so it never silently drops.

So you have two routes to a reviewer. If you already hold a PostHog user UUID, prefer passing it as user_uuid — it's the most reliable. Otherwise resolve a github_login, cheapest source first:

  1. Scratchpad cache. A reviewer:<domain>:<area> entry you (or a sibling run) recorded before — reuse it. Fastest path, and the reason the caching step at the end of this list exists.
  2. Inbox precedent. inbox-reports-list for a similar/related report on the same surface (same source_product, plus a free-text search for the area), then inbox-reports-retrieve / inbox-report-artefacts-list to see who comparable reports were routed to. Reuse that reviewer for the same area — the safest general recipe, available to every scout.
  3. CODEOWNERS / git (only if the scout has a repo checkout). .github/CODEOWNERS for the owning path, or the last git log author for the file. Neither hands you a reviewer directly: CODEOWNERS entries are often team slugs (@your-org/team-name) and git log gives a name + email. A reviewer is always an individual, so resolve either to people with scout-members-list before you write it.
  4. scout-members-list — the in-run roster lookup, for the cold-start case where the c