references/domain_harness_design.md
Domain Harness Design
How a domain folder full of skills becomes an agent harness: a declarative manifest mapping goals → skills → verifications, plus the repo primitives this skill deliberately reuses instead of rebuilding.
Sources
- AGENTS.md convention — https://agents.md/ (launched Aug 2025; adopted by Codex, Cursor, Devin, Gemini CLI, Copilot; stewarded by the Agentic AI Foundation under the Linux Foundation since Dec 2025). The de-facto standard for declaring "how to build and verify work here" in a file agents read first. The harness manifest is the same idea made machine-readable per domain.
- Anthropic, "Effective harnesses for long-running agents", Nov 2025 —
https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents. Its
feature-list.json— each item carrying description + acceptance criteria + status, looped until all verified — is the closest published goal→tasks→verification manifest and the direct model forplan.v1/state.v1. - Model Context Protocol — https://modelcontextprotocol.io/ (Anthropic, Nov 2024;
multi-vendor stewardship 2025). JSON-schema'd tool registries as the declared action space
of a harness. The manifest's
tools[]blocks follow the same declare-don't-discover philosophy: an agent should read what a skill can do, not grep for it mid-loop. - LangGraph checkpointers — https://langchain-ai.github.io/langgraph/. Durable
execution: every super-step persisted, enabling pause/resume/replay and human-in-the-loop
interrupts. The stdlib equivalent here: an atomically-written JSON state file
(
os.replace), append-only evidence entries, and git as the checkpoint layer. - OpenAI Agents SDK — https://openai.github.io/openai-agents-python/.
max_turnsraisingMaxTurnsExceededand tripwire guardrails: caps are runtime errors, not suggestions. Mirrored by controller exit codes 2 (escalate), 4 (close refused), 5 (iteration cap) — a caller script can branch on them mechanically. - Forward Future, "Loop Library" (MIT, vendored at
loop-library/in this repo) — loop anatomy (Observe/Choose/Act/Verify/Record/Repeat-or-stop) and the terminal-state taxonomy (success · clean no-op · blocked · approval-required · exhausted · stagnated). The harness adopts this vocabulary; "errors and exhausted budgets are never reported as success" is implemented as the no-forceclosegate. - Anthropic, "Effective context engineering for AI agents", Sep 2025 —
https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents.
Compaction, structured note-taking, and subagent summaries for long horizons. The state
file's
evidencelog + the close-time handoff block are the structured notes; a fresh session needs onlyplan.json+state.jsonto continue the loop.
The three-layer architecture
Layer 1 — INVENTORY (per domain, committed, regenerated by tooling)
assets/harnesses/<domain>.json what skills exist, what tools they ship,
what checks prove each tool works,
which agentic signals the skill already has
Layer 2 — PLAN (per goal, generated at run time)
plan.json ordered tasks: skill × objective ×
verification[] × max_attempts × done_when
Layer 3 — STATE (per run, the single source of truth)
.agent-harness/state.json task statuses, attempts, evidence log,
iteration counter, close/handoff record
Layer 1 is diff-stable (--no-timestamp) so CI can regenerate and git diff --exit-code
it — manifest drift against the tree becomes a machine-checkable gate, exactly like this
repo's derive_counters.py --check discipline.
Reuse map (import the pattern, don't rebuild the mechanism)
| Need | Reuse from this repo | The harness adds |
|---|---|---|
| Task/state persistence + handoff schema | engineering/skills/tc-tracker (atomic writes, append-only history, session handoff block) | A goal-scoped, domain-agnostic state file (.agent-harness/), not per-code-change records |
| Anti-overfit locked evaluator | engineering/autoresearch-agent (never modify evaluate.py; KEEP/DISCARD/CRASH) | The same invariant generalized: verify re-runs gates itself |
| N-agent tournament on one task | engineering/agenthub (worktrees, DAG, result ranker) | Nothing — route there when the task wants competing attempts |
| Deterministic fan-out/pipeline topologies | engineering/workflow-builder (Workflow-tool .js + validator) | Nothing — route there when orchestrating Claude Code's Workflow tool |
| Loop design/audit vocabulary | loop-library/ (six-step cycle, stop-state taxonomy) | An executable enforcement of that vocabulary |
| Severity-gated ship decision | engineering/skills/ship-gate (CRITICAL/HIGH/ADVISORY verdict) | Use as a close-time check inside a task's verification[] |
| Honest self-scoring | engineering/skills/self-eval (matrix-locked composite) | Optional close-out step before the handoff |
| Bounded-autonomy STOP triggers | engineering/skills/spec-driven-workflow + focused-fix 3-strike rule | escalate_on defaults in every manifest |
Namespacing (collisions this skill deliberately avoids)
- State directory is
.agent-harness/— never.agenthub/,.autoresearch/, ordocs/TC/, which belong to sibling skills. - Command is
/cs-harness—/agenthub:*,/autoresearch-agent:*,/si:*,/tcare taken. - The runtime agent is
harness-runner— "orchestrator" already denotes thecontext: forkdomain routers, andhub-coordinator/experiment-runnerare taken. - The bare name "loop" is overloaded (Claude Code's built-in
/loopscheduler,loop-library, workflow loop templates) — this skill never claims it.
Extending a domain's harness
- Author or improve skills so they carry the agentic signals (intake, refusal gates,
verification, loop discipline, close-out — see the July 2026 audit's AR1–AR6 rubric in
audit/engineering-agentic-2026-07/RUBRIC.md). - Regenerate the manifest:
python3 scripts/harness_manifest_builder.py --domain <domain> --repo-root . --out-dir assets/harnesses --no-timestamp(run from this skill's directory). - The richer the skill's tools and
--samplesupport, the more executable checks its tasks get for free —manual-evidencetasks are the fallback, not the goal.