AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

property-based-testing

Writes, reviews, and debugs property-based tests — Hypothesis, fast-check, proptest, jqwik, rapid, and Echidna or Medusa for Solidity invariants. Use whenever tests should cover a whole input domain instead of a hand-picked list of examples: encode/decode and serialize/deserialize pairs, parsers, canonicalizers and normalizers, validators, numeric and Decimal types, comparators and sort order, data structures, and smart-contract state invariants. Also use when adding cases to an existing @given, fast-check, or proptest suite, when judging whether existing property tests assert anything real, and when a generator has shrunk a counterexample and you need to tell a wrong property from a genuine bug. Not for coverage-guided binary fuzzing (libFuzzer, AFL), mutation-testing campaigns, static analysis, benchmarking, or end-to-end UI tests.

Original source
trailofbits
License
CC-BY-SA-4.0
Pinned revision
2be6433b39e009cd2b6fd3be1895f6b1faa5af9a

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read trailofbits--property-based-testing at revision 2be6433b39e009cd2b6fd3be1895f6b1faa5af9a, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 7

references/generating.md

Generating Property-Based Tests

Writing the @given decorator is the easy part. These are the decisions that make the difference between a suite that finds bugs and one that just runs.

Put constraints in the strategy, not in assume()

This is the single highest-value habit. assume() discards inputs after generation, so a filter that rejects most candidates wastes the budget and eventually trips Hypothesis's exhausted-filter guard — which surfaces as a warning nobody reads.

# Slow, and mostly discards
@given(st.integers())
def test_positive(x):
    assume(x > 0)
    ...

# Generates only what you want
@given(st.integers(min_value=1))
def test_positive(x):
    ...

Reserve assume() for conditions you genuinely cannot express as a generator — a relationship between two already-generated values, usually.

Build compound inputs with st.builds, and derive dependent fields with st.composite or .flatmap rather than generating independently and filtering:

@st.composite
def sized_list_and_index(draw):
    xs = draw(st.lists(st.integers(), min_size=1))
    i = draw(st.integers(min_value=0, max_value=len(xs) - 1))
    return xs, i

Pin the edge cases you already know about

Random generation finds boundaries eventually; @example finds them on every run and documents that you thought about them.

@given(st.lists(st.integers()))
@example([])
@example([1])
@example([1, 1, 1])
def test_sort(xs): ...

Empty, single-element, all-duplicates, zero, negative, and the maximum representable value are the ones that recur.

Settings

Defaults (100 examples, 200ms deadline) are wrong at both ends of the workflow:

@settings(max_examples=10)                      # local iteration
@settings(max_examples=200)                     # CI
@settings(max_examples=1000, deadline=None)     # nightly

Set deadline=None for anything doing real work — the default deadline turns a slow machine into a failing test, and that flake gets the whole suite deleted.

Determinism as a property

f(x) == f(x) is a tautology for a pure function and a real test for anything else. Serializers over dicts or sets, anything involving hashing, iteration order, or time — those can and do return different output for the same input. Assert it where a broken implementation could falsify it, and not otherwise.

Testing the error path

st.binary() against a decoder is worth writing: the contract is usually "raises DecodeError or succeeds, never IndexError, never hangs". Catch only the documented exception and let everything else fail the test.