AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

mutation-testing

Configures mewt or muton campaigns, analyzes surviving mutants, and investigates bugs exposed by testing gaps. Use when setting up mutation testing, reviewing campaign results, identifying equivalent mutants, or finding bugs from surviving mutations.

Original source
trailofbits
License
CC-BY-SA-4.0
Pinned revision
2be6433b39e009cd2b6fd3be1895f6b1faa5af9a

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read trailofbits--mutation-testing at revision 2be6433b39e009cd2b6fd3be1895f6b1faa5af9a, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 10

references/report-template.md

Mutation Testing Analysis Report

Project: [Project Name] Repository: [Repository URL or Path] Analysis Date: [Date] Mutation Testing Tool: [Tool Name and Version] Analyzer: [Agent and model, if known] with mutation-testing skill


Executive Summary

| Metric | Value | |--------|-------| | Total mutants generated | [N] | | Mutants killed | [N] | | Mutants survived | [N] | | Equivalent mutants (false positives) | [N] | | Real surviving mutants | [N] | | Unresolved survivors | [N] | | Skipped or inconclusive mutants | [N] | | Mutation kill rate (raw) | [N]% | | Adjusted kill rate (excluding equivalent) | [N]% |

Key Findings

[Describe the concrete testing gaps, affected behavior, and recommended tests. Explain the campaign's scope and rate denominators. Do not infer overall test quality from the adjusted kill rate. These findings concern weaknesses in tests, not confirmed vulnerabilities in the original code.]

Severity Distribution

| Severity Tier | Count | Percentage of Real Survivors | |---------------|-------|------------------------------| | Critical (Security-Sensitive) | [N] | [N]% | | High (Financial/State Integrity) | [N] | [N]% | | Medium (Business Logic) | [N] | [N]% | | Low (Observability) | [N] | [N]% |


Equivalent Mutants (False Positives)

[One to two sentences explaining what equivalent mutants are and how many were identified in this analysis.]

| # | File | Line | Mutation | Equivalence Reason | |---|------|------|----------|-------------------| | 1 | [path/to/file] | [N] | [original] → [mutated] | [Brief explanation of why this is a false positive] |


Tier 1: Critical — Security-Sensitive Findings

[One sentence introducing the tier and the number of findings.]

Finding C-1: [Descriptive title]

File: [path/to/file] Line: [N] Mutation: [original code] → [mutated code] Mutation Operator: [operator type, if known]

Context: [One to two sentences describing what the mutated code does and what security property it enforces.]

Responsible test file(s): [path/to/test/file] Relevant test case(s): [test function name(s)]

Why this was not caught: [One to two paragraphs. Reference the existing test code and explain what assertion, branch, or edge case is missing. Be specific about the gap.]

Recommended test improvement: [Specific, actionable description of what to add or modify in the test. Do not include code snippets — describe the change in prose.]


[Repeat for each Tier 1 finding: C-2, C-3, ...]

Tier 2: High — Financial/State Integrity Findings

[One sentence introducing the tier and the number of findings.]

Finding H-1: [Descriptive title]

[Same structure as Tier 1 findings.]


[Repeat for each Tier 2 finding: H-2, H-3, ...]

Tier 3: Medium — Business Logic Findings

[One sentence introducing the tier and the number of findings.]

Finding M-1: [Descriptive title]

[Same structure as Tier 1 findings.]


[Repeat for each Tier 3 finding: M-2, M-3, ...]

Tier 4: Low — Observability Findings

[One sentence introducing the tier and the number of findings.]

Finding L-1: [Descriptive title]

[Same structure as Tier 1 findings.]


[Repeat for each Tier 4 finding: L-2, L-3, ...]

Recommendations

Organize all recommendations by test file, grouping related improvements across severity tiers. For each test file, list the findings it should address and summarize the changes needed.

Unresolved Cases

[For each unresolved survivor, state its location, the uncertain behavior, and the evidence needed to classify it. Omit this section if there are none.]


Appendix

Input Files Analyzed

  • [List of mutation result input files provided by the user]

Source Files with Surviving Mutants

| File | Total Surviving | Critical | High | Medium | Low | |------|----------------|----------|------|--------|-----| | [path/to/file] | [N] | [N] | [N] | [N] | [N] |

Methodology

This analysis follows Trail of Bits' mutation testing analysis methodology. Surviving mutants are classified by the impact type of the mutated code, not by the mutation operator used. Equivalent mutants are identified through type-system analysis and reachability checks. For background on mutation testing, see the Trail of Bits blog post "Use mutation testing to find the bugs your tests do not catch."