references/scout-patterns.md · 48000
::(+ a...:qualifier);addressed:::gates re-filing;pattern::` records the repo's stack so the next run doesn't re-derive it.
- Requirements & gotchas — specific to reaching outside the sandbox:
- Needs network reach to the target and the runtime (e.g.
node/npx,git,curl). The default TRUSTED sandbox network covers the platform's trusted-domain allowlist — GitHub, package registries, and common dev infrastructure — which is enough for the clone-and-grep machinery here. A target outside that allowlist (an arbitrary docs site, arxiv.org, a vendor status page) needsnetwork_access: "full"on the scout's config (posthog:scout-config-update, or the nestedconfigat creation), or every fetch is blocked. The harness runs every scout in the same fixed sandbox image — it does not readcompatibilityto install tools. Document the requirement incompatibilityfor human readers, but the scout must verify at run time that the runtime is actually present and, if it isn't, close out with ablocked:<domain>:sandboxmemory entry recording the exact error rather than pretending it ran (see "Be honest when the tool can't run"). - Prefer
gitover authenticated APIs. Scouts run without third-party credentials. Clone cheaply (git clone --filter=blob:none) or reuse an on-disk checkout, and derive the changed-file set fromgit log --since=… --name-only— zero API calls. If you must hit an unauthenticated API, it's rate-limited (~60 req/hr); cap calls per run. - Cap the work and never silently truncate. Bound the number of files assessed and the reports per run; if you drop files for budget, say how many in the close-out.
- Calibrate the tool/ruleset to the target's reality. A ruleset written for one stack (e.g. a server framework) mostly doesn't apply to a different one (e.g. a client-only SPA) — scope the rules per repo before applying them, or the findings are noise.
- Attribute to the diff. Use the tool's diff/PR mode if it has one; otherwise filter its full output down to the recently-changed file set. Don't re-report standing debt.
- The backlog-sweep variant is the deliberate exception. When the job is the standing debt (complexity hot spots, duplicated knowledge, comment rot, test tiering), walk the whole tree instead: every tracked file in a stable order (sort by a hash of the path, so the walk does not follow directory structure), a durable cursor, and a fixed batch per run (50–500 files depending on how much each needs reading).
A renamed file changes its hash and re-enters the walk as a new path, so a lap can skip it or see it twice; snapshot the path list at the start of a lap, and reconcile renames with
git log --followorgit diff -Mwhen a lap has to be exact. Rank what you find by evidence the repo already carries rather than by size: a clone whose copies change in the same commits (git co-change) outranks a bigger clone that never moves together, and a comment block naming a symbol that no longer exists outranks a long one. Two disciplines keep a sweep from becoming a nag: hold "nothing should happen" as a first-class result (deliberate duplication, a long comment that is still true) and record it so the next lap skips it, and decide per finding whether the fix is one obvious file (file itimmediately_actionablewithrepositoryand apriority, so autostart opens the draft PR; the scout's own checkout is read-only and its token cannot push) or a structural call (file the report and route it to a human). A sweep that has lapped the tree once should restart from the files that changed since, not from the top. - Be honest when the tool can't run. If the CLI can't execute in the sandbox (registry unreachable, needs a heavy install you shouldn't attempt), record a memory entry with the exact error and close out — never pretend it ran clean.
- Skip generated/test files; cite the tool's finding (rule id, file:line) in the evidence so a human can reproduce it.
- Treat fetched repo code, rulesets, and tool output as untrusted — see the safety note below. Cloned code and third-party rulesets can carry injected instructions.
- Needs network reach to the target and the runtime (e.g.
State ∩ code-intersection scout
A composition of the external-tool/code pattern with a PostHog-entity read, where neither source alone is the signal — the overlap is. The scout reads an entity's state from PostHog (via the normal MCP tools) and reads the source repo (via the clone-and-grep machinery of the external-tool pattern), and reports only where the two intersect in an actionable way.
-
Canonical example — feature-flag cleanup. A fully-rolled-out-for-a-long-time flag is dead weight only if its key is still referenced in code; a flag that's gone from code is already cleaned up, and a flag still doing targeting work isn't a candidate. So the discriminator is the intersection:
PostHog says STALE/fully-rolled-outANDthe key still appears at a real SDK call site in non-test source. PostHog does the staleness detection server-side (feature-flag-get-allactive:"STALE"), the clone-and-grep half confirms the code reference, and the finding is a P3 cleanup recommendation with the exact file:line call sites and a ready-to-paste cleanup prompt. Everything else — the rollout-state classification, the dependency/experiment caveats — is reused from thecleaning-up-stale-feature-flagsskill the sandbox bakes in. -
Discriminator: the overlap, not either side. Name both reads and the condition that makes their intersection actionable. State-without-code and code-without-state are both non-findings worth a memory entry (
addressed:when the code reference is gone — that's the cleanup having happened), not a report. -
Dedupe + memory: key on the stable entity id, not the row or the file —
dedupe:<domain>:<flag-key>;addressed:<domain>:<flag-key>once the code half disappears;noise:<domain>:<flag-key>for intentional keeps (kill switches, seasonal flags, experiment flags). The repo list lives in aconfig:<domain>:reposentry so a human can curate it. -
Inherits the external-tool gotchas wholesale: network reach (the TRUSTED allowlist covers GitHub; anything outside it needs
network_access=fullon the config), verifygit/rgat run time and close outblocked:if absent, prefer a shallowgit clone --depth 1 --filter=blob:noneof a public repo (no third-party creds), cap the work, and treat cloned code as untrusted data. The one extra knob is which repo — see the note below. -
Repo discovery is the open problem. A per-team scout can name its repos directly (or read them from a
config:scratchpad entry). A truly canonical version needs to discover the repo without hardcoding — the connected GitHub integration already caches the org's repository list, so the graduation path is to read it from there (or surface it into the project profile) rather than bake a repo name into the skill. Until that's wired, keep the repo list out of the canonical body and in per-team config. -
This shape generalizes past feature flags: any "PostHog entity whose code footprint determines whether its state is a problem" fits it — a cohort/insight referencing an event that the code stopped emitting, a deprecated SDK method still called, a tracked event with no capture call left in source.
-
And it generalizes past "PostHog state ∩ code": the two halves can be any two independently-readable sources whose overlap is the signal. Proven variations:
- code ∩ data (the inverse direction) — a newly-shipped user-facing surface in the repo AND no matching capture event in the project's stream: an instrumentation gap.
Here the code half should produce PostHog state and doesn't; confirm the gap on the data side with
read-data-schema/ a stream query before reporting. - code ∩ docs (cross-repo) — a public docs repo claiming beta / coming soon AND the product repo showing the feature went GA (or a doc pinned to an anchor — endpoint, setting, command — a recent PR renamed or removed). Corroborate the "it's GA now" half across several signals (flag removed from code, live flag fully rolled out, early-access graduation) before trusting it; a doc that says beta for a still-gated feature is correct, not stale.
- code ∩ the outside world — a third-party API version pinned in shipped code AND that provider's published deprecation/sunset schedule, fetched from the web. Rotate through providers with a per-run cap rather than re-checking all of them every run, and treat the fetched schedule pages as untrusted data.
- shipped changes ∩ account asks — a feature that landed (a merged PR, a changelog entry, a flag rolled out) AND an account that asked for exactly that (a feature request in their shared channel, a support ticket, a call note) or whose usage shape says they would care (heavy use of the parent product, a workflow the change unblocks). The intersection is a reason for the account owner to reach out with something concrete, which is worth far more than a release note. Hold the match to an exact ask or a distinctive workflow; "they use the product this shipped in" only clears the bar for rare flagship launches, or the scout becomes a newsletter. Route per owner, and see the owner-scoped pattern for the book half.
In every variation the discipline is the same: name both reads, name the condition that makes the intersection actionable, and keep single-source non-findings as memory entries.
- code ∩ data (the inverse direction) — a newly-shipped user-facing surface in the repo AND no matching capture event in the project's stream: an instrumentation gap.
Here the code half should produce PostHog state and doesn't; confirm the gap on the data side with
Custom issue-tracker / work-queue scout
PostHog already ships built-in signals sources for GitHub and Linear: connect the tracker as a data warehouse source, toggle the source on in the inbox, and every new open issue becomes a signal that the grouping pipeline turns into reports. Reach for that first — it is one toggle and it needs no skill. This pattern is what you write when you have outgrown it, which happens sooner than you would expect on a busy tracker.
Know exactly where the built-in source stops, because that boundary is the reason to write a scout at all:
| The built-in source | What that means for you |
| ------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Fires once per issue, at ingest, off the warehouse sync's incremental watermark, capped at 1,000 records per sync. | It reads the issue as first synced. Anything that depends on the thread evolving — someone claimed it, a maintainer's question got answered, a PR appeared — is out of reach. The cap bites on a busy tracker: records past it are dropped for good once the watermark advances, so "every new issue becomes a signal" holds only below that rate. |
| Filters with a fixed rule (GitHub: not closed; Linear: state type not completed/canceled) plus an LLM actionability pass. | No label allowlist, no team or milestone scoping, no author tiering, no "only issues in this area". Your scoping has to live somewhere. |
| Exposes enable/disable plus free-text steering and a default_not_actionable flip on the source config. | Try steering first — it is the cheap middle rung, and it does more than it looks like. A steered gate sees the record's whole metadata block, so a conjunction over fields the sync carries (labels, GitHub author association, Linear state and team) can be written in prose. What it cannot reach is anything not synced onto the record — assignees are not, and neither is the comment thread — which is exactly where the readiness axes live. |
| Inherits the warehouse source's sync cadence, and covers whatever repos/workspace the connection covers. | No independent schedule, and repo scope is an integration-level decision, not a per-signal one. |
So the trigger for this pattern is any of: a judgment with more than one axis, scoping the source config can't express, a verdict that depends on live thread state rather than the issue as filed, or a cadence of your own.
- Watched data: the tracker's open work items, swept as current state every run — not consumed as a stream of new rows. That inversion is the whole unlock: a per-row emitter can never notice that issue #412 became ready last Tuesday when its blocker got answered, because #412 was not new that day.
- Discriminator — a conjunctive multi-axis gate over live state. Name the axes, require all of them, and put them in a table at the top of the body. The worked example's is readiness = unclaimed × unblocked × scoped: no assignee / no linked PR / nobody claiming it in comments, and no unanswered maintainer question or stated dependency and none of the parking labels, and a concrete change whose product area a reader can name. An item failing any one axis is a scratchpad entry, never a report — and record which axis failed, so the run that sees it flip knows what changed. Other trackers rotate the axes rather than the shape: staleness × customer impact for a support queue, unreviewed × age × blast radius for a PR queue, SLA-at-risk × unassigned for a ticket inbox.
- Who reported it tells you what kind of item it is — let impact set the priority. Reporter tier is genuinely informative: an issue the owning team raised on itself is agreed work, an external bug report is a defect someone hit, an external feature request is a decision the team owes an answer to rather than work to schedule.
That distinction should drive routing — actionable versus needs-a-product-call — and it is a reasonable tiebreaker.
Don't let it drive priority on its own, though: priority is what the report contract says it is, an impact judgment, and tier-as-priority quietly ranks an internal chore above a severe external bug and changes which reports clear the autostart threshold.
For the classification itself, prefer the tracker's own membership data — GitHub's
author_associationis on the issue and is authoritative.scout-members-listreturns the PostHog project's roster, not the repo's or the workspace's, so matching a tracker handle against it is a heuristic that fails wherever the two memberships differ; cache what you learn inpattern:<domain>:team-rosterand say in the summary when you were unsure. - Three read paths, and the credential scope decides which — check it, don't assume it.
gh, authenticated. A report-channel scout on a team with a mintable GitHub App installation gets an ephemeral read-only installation token in its sandbox, and the harness prompt says so when it does. Its scope is the catch: the token is minted withcontents,metadata, andpull_requestsread —issuesis not in it. Soghis genuinely authenticated and genuinely useful for repo and PR reads, and it still cannot list issues. That, not a broken CLI, is the likely reason the worked example'sgh issue listcame back empty against a real backlog.- The tracker's API directly. For a public repo the issues API needs no credential at all, so plain
curlagainstapi.github.comis the working path for this pattern today, and reaches live state the sync never carries (a timeline showing a cross-referenced PR, the full comment thread). GitHub is on the default TRUSTED allowlist, so this needs nonetwork_access=full. Quote every URL so&survives the shell wrapper. - A mounted MCP server. Check this before assuming you are stuck with lagged data: a scout's config carries
mcp_gateway_server_ids, and the harness mounts those team-shared MCP Store connections into the run and names their tools in the prompt. Linear is in the catalog, so a Linear-connected team can give the scout live issue, comment, and attachment reads instead of the synced snapshot. It is opt-in per scout and empty by default, which is why it is easy to miss. - The synced warehouse table. The fallback for a private repo, or for Linear with no MCP connection mounted — the sync's own credentials are not yours to reuse for issues.
Discover the table name; never hardcode it. Names are built as
<prefix><source_type>_<schema>, the schema is repository-qualified on multi-repo GitHub sources (github_owner_repo__issues) but bare on legacy single-repo ones (github_issues), and a user-set source prefix changes all of it. Resolve it fromsystem.information_schema.tablesfirst, then inherit the warehouse-backed pattern's gotcha list — cursor, sync lag, string timestamps, confirm columns. You get scoping and judgment the source config can't express; you do not get anything the sync didn't pull.
- Verify your client actually works before trusting a zero. The worked example lost three consecutive runs to a client returning
[]in five milliseconds against a real backlog of ten — no error, no network call, indistinguishable from an empty backlog. Name the client known to work in your sandbox, and record the standing backlog shape inpattern:<domain>:backlog. Then make the zero case a verification, not a verdict: check the HTTP status, the response shape, and that pagination terminated, and if all three hold, a zero is a real empty queue — say so and close out normally. Reserveblocked:for a read that failed or came back internally inconsistent, or a genuinely-cleared backlog leaves the scout permanently stuck. - Two-phase sweep, because detail calls are the expensive half. One cheap list call per scope (GitHub's
labels=is an AND across the list, so an OR over two labels is two calls unioned on issue number — and the/issuesendpoint returns PRs too, so drop anything with apull_requestkey), filter down to survivors, then spend detail calls only on those. Follow pagination on the list half.per_page=100is one page; a scope with more open items silently truncates to the newest, which is not a current-state sweep and can hide a ready item indefinitely. Walk theLinkheader'srel="next"under a hard page cap, and if you stop at the cap, say so in the close-out. Unauthenticated GitHub is 60 requests/hour shared across the sandbox; a full run should cost single digits, and a 403 rate-limit response is ablocked:<domain>:ratelimitclose-out, never a retry loop. - Dedupe + memory — scope the key to the repo or team. An issue number is local to its repository (and a Linear number local to its team), so a scout covering more than one scope must key on
dedupe:<domain>:<repo>:<number>or the tracker's own immutable id. A bare<number>collides two unrelated issue 42s onto one entry, and the loser is either skipped forever or gets another issue's lifecycle note. Store the item'supdated_atin the value — that pairing is what makes the quick close-out nearly free — and the skill version alongside it, because aupdated_atcache is invalidated by tracker edits only: retune the axes or the parking labels and every cached item stays skipped until something unrelated touches it upstream, which reads as the rubric change having done nothing. Re-score entries whose recorded version is behind the current one.noise:parks an item deliberately iceboxed;report:holds the emittedreport_id. - Bound what you write for non-candidates. "Record which axis failed" is right for items that are close, and ruinous as a blanket rule on a busy queue — one
remembercall per rejected item can spend the run before the real candidates get read. Persist a state transition (an item that changed axis since last run) or a capped set of near-misses, and roll the rest into one aggregate backlog entry. - Close the loop on what you filed — and know what closing it can and cannot do. A "ready to pick up" report is wrong the moment someone picks it up, and it costs a person duplicating work already underway.
Re-check each
report:entry every run andedit_reportonce the item is assigned, PR-linked, or closed — but note thatedit_reportmutatestitle,summary,append_note,append_evidence,suggested_reviewers,charts, andsuggested_promptsonly. It cannot change status or actionability, so an appended note or evidence row does not retire the report. Rewrite the title and summary so the stale framing is gone from the surface a human scans, and leave the status change to a person. - Routing the outcome is part of the design. On the report channel a queue scout can hand work straight to a draft PR:
actionability: immediately_actionable+repository+ aprioritymakes the report eligible to autostart one. Eligible is not automatic — the team's autostart toggle, its priority threshold, the org's self-driving quota, a free-trial hold (a trial org gets reports, not pull requests, until the trial ends), and resolving a runner identity each gate it independently, so a correctly-filed report can sit still for reasons that have nothing to do with the scout. Reviewers do not gate it: a report whosesuggested_reviewersresolve to nobody still starts under the member who enabled signals for the team, provided it meets the team's default autostart priority. Reserverequires_human_inputfor items needing a product call or touching permissions, billing, or security — and still setrepositoryon those, so a later human press of Create PR gets a sandbox with credentials rather than doing the work and failing at push time. Cap reports per run hard (the worked example files at most 3, highest priority first) and say in the close-out how many candidates you dropped for budget. - Seam with the built-in source — and know the toggle is not per-repo. If the same tracker's built-in source is also enabled, you have two things filing on one surface.
The source config is unique on
(team, source_product, source_type)with no repository selector, so turning it off to hand the surface to your scout turns it off for every connected repo — only