references/verification-checklist.md
Verification Checklist — Post-Analysis Quality Gates
This file is the single source of truth for all verification rules that must pass before a threat model report is finalized. It is designed to be handed to a verification sub-agent along with the output folder path.
Authority hierarchy: This file contains CHECKING rules (pass/fail criteria for quality gates). The AUTHORING rules that produce the content being checked are in
orchestrator.md. Some rules appear in both files for visibility — if they ever conflict:orchestrator.mdtakes precedence for authoring decisions (how to write), this file takes precedence for pass/fail criteria (what constitutes a valid output). Do NOT remove rules from either file to "deduplicate" — the overlap is intentional for visibility.
When to use: After ALL output files are written (0.1-architecture.md through 0-assessment.md), run every check in this file. If any check fails, fix the issue before finalizing.
Sub-agent delegation: The orchestrator can delegate this entire file to a verification sub-agent with the prompt:
"Read verification-checklist.md. For each check, inspect the named output file(s) and report PASS/FAIL with evidence. Fix any failures."
Inline Quick-Checks (Run Immediately After Each File Write)
Purpose: These are lightweight self-checks the WRITING agent runs immediately after creating each file — NOT deferred to Step 10. Since the agent just wrote the file, the content is still in active context, making these checks highly effective.
How to use: Before writing each file, read the corresponding skeleton from
skeletons/skeleton-*.md. After eachcreate_filecall, scan the content you just wrote for these patterns. If any check fails, fix the file immediately before proceeding to the next step.Skeleton compliance rule: Every output file MUST follow its skeleton's section order, table column headers, and heading names. Do NOT add sections/tables not in the skeleton. Do NOT rename skeleton headings.
After writing 3-findings.md:
- [ ] First finding heading starts with
### FIND-01:(notF01,F-01, orFinding 1) - [ ] Every finding has these exact row labels:
SDL Bugbar Severity,Remediation Effort,Mitigation Type,Exploitability Tier,Exploitation Prerequisites,Component - [ ] Every CVSS value contains
CVSS:4.0/prefix - [ ] Every
Related Threatscell contains](2-stride-analysis.md#(hyperlink, not plain text) - [ ] Every finding has
#### Description,#### Evidence,#### Remediation, and#### Verificationsub-headings (notRecommendation, notImpact, notMitigation, not bold**Description:**paragraphs) — exactly 4 sub-headings, no extras - [ ] Every
#### Descriptionsection has at least 2 sentences of technical detail (not single-sentence stubs) - [ ] Every
#### Evidencesection cites specific file paths, line numbers, or config keys (not generic statements like "found in codebase") - [ ] Every finding has ALL 10 mandatory attribute rows:
SDL Bugbar Severity,CVSS 4.0,CWE,OWASP,Exploitation Prerequisites,Exploitability Tier,Remediation Effort,Mitigation Type,Component,Related Threats - [ ] Every CWE value is a hyperlink: contains
](https://cwe.mitre.org/(not plain text likeCWE-79) - [ ] Every OWASP value uses
:2025suffix (not:2021) - [ ] Findings organized by TIER (Tier 1/2/3 headings), NOT by severity (no
## Critical Findings) - [ ] Tier-Prerequisite consistency (inline): For each finding, use canonical mapping:
None→T1;Authenticated User/Privileged User/Internal Network/Local Process Access→T2;Host/OS Access/Admin Credentials/Physical Access/{Component} Compromise/combos→T3. ⛔Application AccessandHost Accessare FORBIDDEN. - [ ] Count finding headings — they must be sequential: FIND-01, FIND-02, FIND-03...
- [ ] No time estimates: search for
~,Sprint,Phase,hour,day,week— must not appear - [ ] Threat Coverage Verification table present at end of file with columns
Threat ID | Finding ID | Status - [ ] Coverage table status values use emoji prefixes:
✅ Covered (FIND-XX),✅ Mitigated (FIND-XX),🔄 Mitigated by Platform— NOT plain text like "Finding", "Mitigated", "Covered" - [ ] Coverage table column names are exactly
Threat ID | Finding ID | Status— NOTThreat | Finding | Status
After writing 0-assessment.md:
- [ ] First
##heading is## Report Files - [ ] Count
##headings — exactly 7 with these exact names: Report Files, Executive Summary, Action Summary, Analysis Context & Assumptions, References Consulted, Report Metadata, Classification Reference - [ ] Heading contains
¬and: search forAnalysis Context & Assumptions - [ ] Count
---separator lines — at least 5 - [ ]
### Quick Winsheading exists - [ ]
### Priority by Tier and CVSS Scoreheading exists under Action Summary, BEFORE Quick Wins - [ ] Priority table has max 10 rows: Count data rows in Priority by Tier and CVSS Score table — must be ≤ 10
- [ ] Priority table sort order: All Tier 1 findings come first, then Tier 2, then Tier 3. Within each tier, higher CVSS scores come first. ❌ T2 finding appearing before a T1 finding → FAIL
- [ ] Priority table Finding hyperlinks: Every Finding cell is a hyperlink
[FIND-XX](3-findings.md#find-xx-title-slug). Search for](3-findings.md#in every row — must be present. ❌ Plain textFIND-XXwithout link → FAIL - [ ] Priority table anchor resolution: For each hyperlink, verify the anchor slug matches the actual
### FIND-XX:heading in 3-findings.md AS WRITTEN. Compute the anchor from the heading text (lowercase, spaces to hyphens, strip special chars). ❌ If any heading contains status tags like[STILL PRESENT]or[NEW], that is a FAIL — status tags must NOT appear in headings (see Phase 2 check). Anchors should be computed from clean, tag-free heading text. - [ ] Action Summary tier hyperlinks: Tier 1, Tier 2, Tier 3 cells in the Action Summary table are hyperlinks to
3-findings.md#tier-Nanchors - [ ]
### Needs Verificationheading exists - [ ]
### Finding Overridesheading exists - [ ] Action Summary has exactly 4 data rows: Tier 1, Tier 2, Tier 3, Total. Search for
| Mitigated |or| Platform |or| Fixed |in the Action Summary table — FAIL if found. These are NOT separate tiers. - [ ] Git Commit includes date: The
| Git Commit |row must contain both the SHA and the commit date (e.g.,f49298ff(2026-03-04)). If only the hash is shown without date → FAIL. - [ ] Baseline/Target Commits include dates (incremental mode):
| Baseline Commit |and| Target Commit |rows must each include a date alongside the SHA. - [ ]
### Security Standardsand### Component Documentationheadings exist (two Reference subsections) - [ ]
| Model |row exists in Report Metadata table - [ ]
| Analysis Started |row exists in Report Metadata table - [ ]
| Analysis Completed |row exists in Report Metadata table - [ ]
| Duration |row exists in Report Metadata table - [ ] Metadata values wrapped in backticks: check for
`in metadata value cells - [ ] Report Files table first row:
0-assessment.mdis the FIRST data row (not0.1-architecture.md) - [ ] Report Files completeness: Every generated
.mdand.mmdfile in the output folder has a corresponding row in the Report Files table (threat-inventory.jsonis intentionally excluded) - [ ] Report Files conditional rows:
1.2-threatmodel-summary.mmdandincremental-comparison.htmlrows present ONLY if those files were actually generated - [ ] Note on threat counts blockquote: Executive Summary contains
> **Note on threat counts:**paragraph - [ ] Boundary count: Boundary count in Executive Summary matches actual Trust Boundary Table row count in
1-threatmodel.md - [ ] Action Summary tier priorities: Tier 1 = 🔴 Critical Risk, Tier 2 = 🟠 Elevated Risk, Tier 3 = 🟡 Moderate Risk. These are FIXED — never modified based on counts.
- [ ] Risk Rating heading has NO emojis:
### Risk Rating: Elevatednot### Risk Rating: 🟠 Elevated
After writing 0.1-architecture.md:
- [ ] Count
sequenceDiagramoccurrences — at least 3 - [ ] First 3 sequence diagrams have
participantlines and->>message arrows (not empty diagram blocks) - [ ] Key Components table row count matches Component Diagram node count
- [ ] Every Key Components table row uses PascalCase name (not kebab-case
my-componentor snake_casemy_component) - [ ] Every Key Components Type cell is one of:
Process,Data Store,External Service,External Interactor— no ad-hoc types likeRole,Function - [ ] Technology Stack table has all 5 rows filled: Languages, Frameworks, Data Stores, Infrastructure, Security
- [ ]
## Security Infrastructure Inventorysection exists (not missing) - [ ]
## Repository Structuresection exists (not missing)
After writing 1.1-threatmodel.mmd:
- [ ] Line 1 starts with
%%{init: - [ ] Contains
classDef process,classDef external,classDef datastore - [ ] No Chakra UI colors (
#4299E1,#48BB78,#E53E3E) - [ ]
linkStyle default stroke:#666666,stroke-width:2pxpresent - [ ] DFD uses
flowchart LR(NOTflowchart TB) — search forflowchartand verify direction isLR - [ ] Incremental DFD styling (incremental mode only): If new components exist, verify
classDef newComponent fill:#d4edda,stroke:#28a745is present AND new component nodes use:::newComponent(NOT:::process). If removed components exist, verifyclassDef removedComponentwith gray dashed styling. ❌newComponent fill:#6baed6(same blue as process) → FAIL (visually invisible).
After writing 2-stride-analysis.md:
- [ ]
## Summaryappears BEFORE any## ComponentNamesection (check line numbers) - [ ] Summary table has columns:
| Component | Link | S | T | R | I | D | E | A | Total | T1 | T2 | T3 | Risk |— search for| S | T | R | I | D | E | A |to verify - [ ] Summary table S/T/R/I/D/E/A columns contain numeric values (0, 1, 2, 3...), NOT all identical 1s for every component
- [ ] Every component has
#### Tier 1,#### Tier 2,#### Tier 3sub-headings - [ ] No
&,/,(,),:in##headings - [ ] No status tags in headings (ANY file): Search ALL
.mdfiles for^##.+\[Existing\],^##.+\[Fixed\],^##.+\[Partial\],^##.+\[New\],^##.+\[Removed\], and same for###headings. Also check old-style:^##.+\[STILL,^##.+\[NEW,^###.+\[STILL,^###.+\[NEW CODE. ❌ Tags in headings break anchor links and pollute ToC. Status must be on first line of section body as a blockquote (> **[Tag]**), not in the heading. - [ ] CRITICAL — A = Abuse, NEVER Authorization: Search for
| Authorization |in the file. If ANY match is a STRIDE category label (not inside a threat description sentence) → FIX IMMEDIATELY by replacing with| Abuse |. The "A" in STRIDE-A stands for "Abuse" (business logic abuse, workflow manipulation, feature misuse). This is the single most common error observed. - [ ] N/A entries not counted: If any component has
N/A — {justification}for a STRIDE category, verify that category shows0(not1) in the Summary table - [ ] STRIDE Status values: Every threat row's Status column uses exactly one of:
Open,Mitigated,Platform. NoPartial,N/A,Accepted, or ad-hoc values. - [ ] Platform ratio: Count threats with
Platformstatus vs total threats. If >20% (standalone) or >35% (K8s operator) → re-examine each Platform entry. - [ ] STRIDE column arithmetic: For every Summary table row, verify S+T+R+I+D+E+A = Total AND T1+T2+T3 = Total
- [ ] Full category names in threat tables: Category column uses full names (
Spoofing,Tampering,Information Disclosure,Denial of Service,Elevation of Privilege,Abuse) — NOT abbreviations (S,T,DoS,EoP) - [ ] N/A table present: Every component section has a
| Category | Justification |table listing STRIDE categories with no threats — NOT prose/bullet-point format - [ ] Link column is separate: Summary table 2nd column is
Linkwith[Link](#anchor)values — component names do NOT contain embedded hyperlinks - [ ] Exploitability Tiers 4th column: The tier definition table must have 4th column named
Assignment Rule(NOTExample,Description,Criteria)
After writing incremental-comparison.html (incremental mode only):
- [ ] HTML contains
Trust BoundariesorBoundariesin the metrics bar — search for the text "Boundaries" - [ ] STRIDE heatmap has 13 columns: Component, S, T, R, I, D, E, A, Total, divider, T1, T2, T3 — search for
T1andT2andT3in the HTML - [ ] Fixed/New/Previously Unidentified status information appears ONLY in colored status cards, NOT also as small inline badges in the metrics bar
- [ ] No
| Authorization |as a STRIDE category label in the heatmap — search for "Authorization" in heatmap rows - [ ] HTML counts match markdown counts: The Total threats in the HTML heatmap must equal the Totals row from
2-stride-analysis.md. If they differ, regenerate the HTML heatmap from the STRIDE summary data. T1+T2+T3 totals in HTML must also match. - [ ] Comparison cards present: HTML contains
comparison-cardsdiv with 3 cards: baseline (hash + date + rating), target (hash + date + rating), trend (direction + duration) - [ ] Commit dates from git log: Baseline and target dates in comparison cards must match actual commit dates (NOT today's date, NOT analysis run date)
- [ ] Code Changes box: 5th metrics box shows commit count and PR count (NOT "Time Between")
- [ ] No Time Between box: Search for "Time Between" — must NOT appear in metrics bar
- [ ] Status cards are concise: Each status card's
card-itemsdiv must contain only a short summary sentence. ❌ Threat IDs (T06.S, T02.E), finding IDs (FIND-14), or component names listed in cards → FAIL. Search forT\d+\.andFIND-\d+insidecard-itemsdivs. Detailed item breakdowns belong in the Threat/Finding Status Breakdown section, not in the summary cards.
After writing any incremental report file (incremental mode — inline check):
- [ ] Simplified display tags only: Search ALL
.mdfiles for old-style tags:[STILL PRESENT],[NEW CODE],[NEW IN MODIFIED],[PREVIOUSLY UNIDENTIFIED],[PARTIALLY MITIGATED],[REMOVED WITH COMPONENT],[MODIFIED]. ❌ Any match → FAIL. Replace with simplified tags:[Existing],[Fixed],[Partial],[New],[Removed]. - [ ] Valid display tags: Every finding/threat annotation uses exactly one of the 5 simplified tags:
[Existing],[Fixed],[Partial],[New],[Removed]. Tags must appear as blockquote on first line of body:> **[Tag]**. - [ ] Component status simplified: Component status column uses only:
Unchanged,Modified,New,Removed. ❌Restructured→ FAIL (useModifiedinstead). - [ ] Change Summary tables use simplified tags: Threat Status table has 4 rows (Existing/Fixed/New/Removed). Finding Status table has 5 rows (Existing/Fixed/Partial/New/Removed). ❌ Old-style rows like
Still Present,New (Code),Partially Mitigated→ FAIL.
After writing threat-inventory.json (inline check):
- [ ] JSON threat count matches STRIDE file: Count unique threat IDs in
2-stride-analysis.md(grep^\| T\d+\.). This count MUST equalthreatsarray length in the JSON. If STRIDE has MORE threats than JSON → threats were dropped during serialization. Rebuild the JSON. - [ ] JSON metrics internally consistent:
metrics.total_threatsmust equalthreatsarray length.metrics.total_findingsmust equalfindingsarray length.
After writing 0-assessment.md (count validation):
- [ ] Element count in Executive Summary matches actual Element Table row count (re-read
1-threatmodel.mdif needed) - [ ] Finding count matches actual
### FIND-heading count in3-findings.md - [ ] Threat count matches Total from summary table in
2-stride-analysis.md
Phase 0 — Common Deviation Scan
These are the most frequently observed deviations across all previous runs. After output is generated, scan every output file for these specific patterns. Each check has a WRONG pattern to search for and a CORRECT expected pattern.
How to use: For each check, grep/scan the output files for the WRONG pattern. If found → FAIL. Then verify the CORRECT pattern is present. This phase catches recurring mistakes that the generating model tends to make despite instructions.
0.1 Structural Deviations
- [ ] Findings organized by severity instead of tier — Search for
## Critical Findings,## Important Findings,## High Findings. These must NOT exist. ❌## Critical Findings→ ✅## Tier 1 — Direct Exposure (No Prerequisites) - [ ] Flat STRIDE tables (no tier sub-sections) — Each component in
2-stride-analysis.mdmust have#### Tier 1,#### Tier 2,#### Tier 3sub-headings. ❌ Single flat table per component → ✅ Three separate tier sub-sections - [ ] Missing Exploitability Tier or Remediation Effort on findings — Every
### FIND-block in3-findings.mdmust contain bothExploitability TierandRemediation Effortrows. ❌ Missing either field → ✅ Both MANDATORY - [ ] STRIDE summary missing tier columns — Summary table in
2-stride-analysis.mdmust includeT1,T2,T3columns. ❌ Only S/T/R/I/D/E/A/Total → ✅ Must also have T1/T2/T3/Risk columns - [ ] STRIDE Summary at bottom — Search for the line number of
## Summaryvs first## Component. ❌ Summary after components → ✅ Summary BEFORE all component sections, immediately after## Exploitability Tiers - [ ] Exploitability Tiers table columns — The tier definition table in
2-stride-analysis.mdmust have exactly these 4 columns:Tier | Label | Prerequisites | Assignment Rule. ❌Example,Description,Criteriaas 4th column → ✅Assignment Ruleonly. The Assignment Rule cells must contain the rigid rule text, NOT deployment-specific examples.
0.2 File Format Deviations
- [ ]
.mdwrapped in code fences — Check if any.mdfile starts with```markdownor````markdown. ❌```markdown\n# Title→ ✅# Titleon line 1 - [ ]
.mmdwrapped in code fences — Check if.mmdfile starts with```plaintextor```mermaid. ❌```mermaid\n%%{init:→ ✅%%{init:on line 1 - [ ] Leaked skill directives in output — Search ALL
.mdfiles for⛔,RIGID TIER,Do NOT use subjective,MANDATORY,CRITICAL —,decision procedure. These are internal skill instructions that must NOT appear in report output. ❌ Any match → ✅ Zero matches. Remove any leaked directive lines. - [ ] Nested duplicate output folder — Check if the output folder contains a subfolder with the same name (e.g.,
threat-model-20260307-081613/threat-model-20260307-081613/). ❌ Subfolder exists → ✅ Delete the nested duplicate. The output folder should contain only files, no subfolders. - [ ] STRIDE-A "Authorization" instead of "Abuse" — Search
2-stride-analysis.mdfor| Authorization |or**Authorization**used as a STRIDE category name. The A in STRIDE-A is ALWAYS "Abuse", never "Authorization". ❌ Any match where Authorization is used as a STRIDE category → ✅ Replace with "Abuse". Note: do NOT replace "Authorization" when it appears inside threat descriptions (e.g., "Authorization header", "lacks authorization checks").
0.3 Assessment Section Deviations
- [ ] Wrong section name for Action Summary — Search for
Priority Remediation Roadmap,Top Recommendations,Key Recommendations,Risk Profile. ❌ Any of those names → ✅## Action Summaryonly - [ ] Separate recommendations section — Search for
### Key Recommendationsor### Top Recommendationsas standalone sections. ❌ Separate section → ✅ Action Summary IS the recommendations - [ ] Missing Quick Wins subsection — Search for
### Quick Winsunder Action Summary. ❌ Missing → ✅ Present (with note if no low-effort T1 findings) - [ ] Missing threat count context — Search for
> **Note on threat counts:**blockquote in Executive Summary. ❌ Missing → ✅ Present - [ ] Missing Analysis Context & Assumptions — Search for
## Analysis Context & Assumptions. ❌ Missing → ✅ Present with### Needs Verificationand### Finding Overridessub-sections - [ ] Missing mandatory assessment sections — Verify ALL 7 exist: Report Files, Executive Summary, Action Summary, Analysis Context & Assumptions, References Consulted, Report Metadata, Classification Reference. ❌ Any missing → ✅ All 7 present
0.4 References & Metadata Deviations
- [ ] References Consulted as flat table — Search for
| Reference | Usage |pattern. ❌ Two-column flat table → ✅ Two subsections:### Security Standardswith| Standard | URL | How Used |and### Component Documentationwith| Component | Documentation URL | Relevant Section | - [ ] References missing URLs — Every row in References Consulted tables must have a full
https://URL. ❌ Missing URL column or empty URLs → ✅ Full URLs in every row - [ ] Report Metadata missing Model — Search for
| **Model** |or| Model |row. ❌ Missing → ✅ Present with actual model name - [ ] Report Metadata missing timestamps — Search for
Analysis Started,Analysis Completed,Durationrows. ❌ Any missing → ✅ All three present with computed values
0.5 Finding Quality Deviations
- [ ] CVSS score without vector or missing prefix — Grep each finding's CVSS field. The value MUST match pattern:
\d+\.\d+ \(CVSS:4\.0/AV:. Specifically check for theCVSS:4.0/prefix — the most common deviation is outputting the vector without this prefix (bareAV:N/AC:L/...). ❌9.3(score only) → ❌9.3 (AV:N/AC:L/...)(no prefix) → ✅9.3 (CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:H/VI:H/VA:H/SC:N/SI:N/SA:N) - [ ] CWE without hyperlink — Grep for
CWE-\d+without preceding[. ❌CWE-78: OS Command Injection→ ✅[CWE-78](https://cwe.mitre.org/data/definitions/78.html): OS Command Injection - [ ] OWASP
:2021suffix — Grep for:2021. ❌A01:2021→ ✅A01:2025 - [ ] Related Threats as plain text — Grep
Related Threatsrows for pattern without](. ❌T-02, T-17, T-23→ ✅[T02.S](2-stride-analysis.md#component-name), [T17.I](2-stride-analysis.md#other-component) - [ ] Finding IDs out of order — Check that FIND-NN IDs are sequential: FIND-01, FIND-02, FIND-03... ❌
FIND-06appearing beforeFIND-04→ ✅ Sequential numbering top-to-bottom - [ ] CVSS AV:L or PR:H with Tier 1 — Grep every Tier 1 finding's CVSS vector for
AV:LorPR:H. ❌ Tier 1 with local-only access → ✅ Downgrade to T2/T3 - [ ] Localhost-only or admin-only finding in Tier 1 — Check deployment context: air-gapped, localhost, single-admin services should NOT be Tier 1. ❌ Tier 1 for admin-only → ✅ T2/T3
- [ ] Time estimates in output — Grep for
~1 hour,Sprint,Phase 1,(hours),(days),(weeks),Immediate. ❌ Any scheduling language → ✅ OnlyLow/Medium/Higheffort labels - [ ] "Accepted Risk" in Coverage table — Grep
3-findings.mdforAccepted Risk. ❌ Any match → FAIL. The tool does NOT have authority to accept risks. EveryOpenthreat MUST have a finding. Replace all⚠️ Accepted Riskwith✅ Coveredand create corresponding findings.
0.6 Diagram Deviations
- [ ] Wrong color palette — Grep all
#[0-9a-fA-F]{6}in.mmdfiles and Mermaid blocks. ❌#4299E1,#48BB78,#E53E3E,#2B6CB0,#2D3748,#2F855A,#C53030(Chakra UI) → ✅ Only allowed:#6baed6,#2171b5,#fdae61,#d94701,#74c476,#238b45,#e31a1c,#666666,#ffffff,#000000 - [ ] Custom themeVariables colors — Search