references/incremental-orchestrator.md · 24000
ication Reference (static table copied from skeleton)
Additional incremental sections (insert between Action Summary and Analysis Context):
## Change Summary
### Component Changes
| Status | Count | Components |
|--------|-------|------------|
| Unchanged | X | ComponentA, ComponentB, ... |
| Modified | Y | ComponentC, ... |
| New | Z | ComponentD, ... |
| Removed | W | ComponentE, ... |
### Threat Status
| Status | Count |
|--------|-------|
| Still Present | X |
| Fixed | Y |
| New (Code) | Z |
| New (Modified) | M |
| Previously Unidentified | W |
| Removed with Component | V |
### Finding Status
| Status | Count |
|--------|-------|
| Still Present | X |
| Fixed | Y |
| Partially Mitigated | P |
| New (Code) | Z |
| New (Modified) | M |
| Previously Unidentified | W |
| Removed with Component | V |
### Risk Direction
[Improving / Worsening / Stable] — [1-2 sentence justification based on status distribution]
---
## Previously Unidentified Issues
These vulnerabilities were present in the baseline code at commit `{baseline_sha}` but were not identified in the prior analysis:
| Finding | Title | Component | Evidence |
|---------|-------|-----------|----------|
| FIND-XX | [title] | [component] | Baseline code at `{file}:{line}` |
Report Metadata additions:
| Baseline Report | `{baseline_folder}` |
| Baseline Commit | `{baseline_sha}` (`{baseline_commit_date}` — run `git log -1 --format="%cs" {baseline_sha}`) |
| Target Commit | `{target_sha}` (`{target_commit_date}` — run `git log -1 --format="%cs" {target_sha}`) |
| Baseline Worktree | `{worktree_path}` |
| Analysis Mode | `Incremental` |
4h. incremental-comparison.html
- Read
skeletons/skeleton-incremental-html.mdfirst — use 8-section structure and CSS variables
Generate a self-contained HTML file that visualizes the comparison. All data comes from the change_status fields already computed in threat-inventory.json.
Structure:
<!-- Section 1: Header + Comparison Cards -->
<div class="header">
<div class="report-badge">INCREMENTAL THREAT MODEL COMPARISON</div>
<h1>{{repo_name}}</h1>
</div>
<div class="comparison-cards">
<div class="compare-card baseline">
<div class="card-label">BASELINE</div>
<div class="card-hash">{{baseline_sha}}</div>
<div class="card-date">{{baseline_commit_date from git log}}</div>
<div class="risk-badge">{{old_risk_rating}}</div>
</div>
<div class="compare-arrow">→</div>
<div class="compare-card target">
<div class="card-label">TARGET</div>
<div class="card-hash">{{target_sha}}</div>
<div class="card-date">{{target_commit_date from git log}}</div>
<div class="risk-badge">{{new_risk_rating}}</div>
</div>
<div class="compare-card trend">
<div class="card-label">TREND</div>
<div class="trend-direction">{{Improving|Worsening|Stable}}</div>
<div class="trend-duration">{{N months}}</div>
</div>
</div>
<!-- Section 2: Metrics Bar (5 boxes — NO Time Between, use Code Changes) -->
<div class="metrics-bar">
Components: {{old_count}} → {{new_count}} (±N)
Trust Boundaries: {{old_boundaries}} → {{new_boundaries}} (±N)
Threats: {{old_count}} → {{new_count}} (±N)
Findings: {{old_count}} → {{new_count}} (±N)
Code Changes: {{COMMIT_COUNT}} commits, {{PR_COUNT}} PRs
</div>
<!-- Section 3: Status Summary Cards (colored cards — primary visualization) -->
<div class="status-cards">
<!-- Green card: Fixed (count + list of fixed items) -->
<!-- Red card: New (code + modified) (count + list of new items) -->
<!-- Amber card: Previously Unidentified (count + list) -->
<!-- Gray card: Still Present (count) -->
</div>
<!-- Section 4: Component Status Grid -->
<table class="component-grid">
<!-- Row per component: ID | Type | Status (color-coded) | Source Files -->
</table>
<!-- Section 5: Threat/Finding Status Breakdown -->
<div class="status-breakdown">
<!-- Grouped by status: Fixed items, New items, etc. -->
<!-- Each item: ID | Title | Component | Status -->
</div>
<!-- Section 6: STRIDE Heatmap with Deltas -->
<!-- ⛔ MANDATORY: Heatmap MUST have 13 columns including T1/T2/T3 after a divider -->
<table class="stride-heatmap">
<thead>
<tr>
<th>Component</th>
<th>S</th><th>T</th><th>R</th><th>I</th><th>D</th><th>E</th><th>A</th>
<th>Total</th>
<th class="divider"></th>
<th>T1</th><th>T2</th><th>T3</th>
</tr>
</thead>
<tbody>
<!-- Row per component. Each STRIDE cell: value (▲+N or ▼-N delta from baseline) -->
<!-- The divider column is a thin visual separator between STRIDE totals and tier breakdown -->
</tbody>
</table>
<!-- Section 7: Needs Verification -->
<div class="needs-verification">
<!-- Items where analysis disagrees with old report -->
</div>
<!-- Section 8: Footer -->
<div class="footer">
Model: {{model}} | Duration: {{duration}}
Baseline: {{baseline_folder}} at {{baseline_sha}}
Generated: {{timestamp}}
</div>
Styling rules:
- Self-contained: ALL CSS in inline
<style>block. No CDN links. - Color conventions: green (#28a745) = fixed, red (#dc3545) = new vulnerability, amber (#fd7e14) = previously unidentified, gray (#6c757d) = still present, blue (#2171b5) = modified
- Print-friendly: include
@media printstyles - Use the same CSS color conventions defined above for visual consistency
Phase 5: Verification
5a. Standard Verification
Run the standard verification-checklist.md (Phases 0–9) against the new report. The incremental report must pass ALL standard quality checks since it is a standalone report. Delegate to a sub-agent with the output folder absolute path so it can read the report files.
5b. Incremental Verification
After standard verification passes, run the incremental-specific checks from experiment-history/mode-c-verification-suite.md (Phases 1–9, 33 checks). These verify:
- Structural continuity (every old item accounted for)
- Code-verified status accuracy (e.g., "fixed" actually verified against code diff)
- Previously-unidentified classification (verified against baseline worktree)
- DFD consistency (old nodes present, new nodes distinguished)
- Standalone quality (no dangling references to old report)
- Comparison summary accuracy (counts match inventory)
- Needs Verification completeness
- Edge cases (merges, splits, rewrites)
- Metrics/JSON integrity
5c. Correction Workflow
- Collect all PASS/FAIL results
- For each FAIL → apply the check's "Fail remediation" action
- Re-run failed checks to confirm they pass
- After 2 correction attempts, escalate remaining failures to Needs Verification
- Record end time and generate execution summary
⛔ Rules Specific to Incremental Analysis
These rules supplement (not replace) the 34 mandatory rules from orchestrator.md:
Rule I1: Old Report Assessment Judgments Are Preserved
When the new analysis would assign a different TMT category, component type, tier, or threat relevance than the old report → preserve the old report's value. Log the disagreement in Needs Verification with:
- Old value
- New analysis's proposed value
- 1-2 sentence reasoning
- What the user should check
Exception: Factual corrections (file paths, git metadata, arithmetic) are corrected silently and noted in Report Metadata.
Rule I2: No Silent Overrides
The report body uses the OLD value for assessment judgments. Disagreements go to Needs Verification. The user must explicitly confirm any reclassification.
Rule I3: Previously-Unidentified Must Be Verified
Every previously_unidentified classification MUST include evidence from the baseline worktree. The analyst must actually read the old code at the cited file/line and confirm the vulnerability pattern existed. No guessing based on "it's probably been there."
Rule I4: Fixed Must Be Code-Verified
Every fixed classification MUST cite the specific code change that addressed the vulnerability. Generic statements like "the team fixed this" are not acceptable — show the diff.
Rule I5: new_in_modified Requires Change Attribution
Every new_in_modified finding MUST identify the specific code change that introduced the vulnerability. Cite the diff hunk, new function, new config value, or new dependency that created the issue.
Rule I6: Do Not Delete Baseline Worktree
The baseline worktree may be reused by future incremental analyses. Do NOT run git worktree remove on it. The worktree path is recorded in Report Metadata for reference.
Rule I7: Change Status Consistency
A component's change_status must be consistent with its threats' and findings' statuses:
unchangedcomponent → its threats should bestill_present(orpreviously_unidentifiedfor newly discovered threats in unchanged code)removedcomponent → ALL its threats/findings must beremoved_with_componentmodifiedcomponent → at least one threat should bemodified,fixed, ornew_in_modifiednewcomponent → ALL its threats must benew_code
Rule I8: Carry Forward, Don't Copy
"Carry forward" means regenerating a threat/finding entry that says the same thing — NOT literally copy-pasting old report text. The regenerated entry should:
- Use the same ID
- Reference current file paths (even if unchanged)
- Be phrased in present tense about the current code
- Include the
[STILL PRESENT]annotation
Summary: Phase-by-Phase Checklist
| Phase | Action | Success Criteria |
|-------|--------|-----------------|
| 0 | Setup, validate inputs, worktree | All inputs exist, worktree accessible |
| 1 | Load old inventory skeleton | All arrays populated, metrics match |
| 2 | Per-component change detection | Every component has a change_status |
| 3 | Scan for new components | New components identified, missed components flagged |
| 4 | Generate all report files | 8-9 files written to output folder |
| 5 | Verification (standard + incremental) | All checks pass or escalated to Needs Verification |