AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

chaos-engineering

Use before running a chaos experiment, gameday or fault injection, or after one to write it up. Designs it with steady state and abort criteria in experiment_designer.py, sizes the blast radius with blast_radius_calculator.py and drafts the postmortem.

Original source
alirezarezvani
License
MIT
Pinned revision
e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read alirezarezvani--chaos-engineering at revision e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 7

assets/experiment_template.md

Chaos Experiment

Fill in every section before running. Refuse to run if any section is empty.

Identity

  • Experiment ID: <auto-generated; format: chaos-<target>-<attack>-<unix-ts>>
  • Date: <YYYY-MM-DD>
  • Owner: <your-handle@team>
  • On-call team: <team channel / pager>
  • Reviewer: <peer who reviewed this plan>

1. Hypothesis

When <fault>, <steady-state metric> stays <tolerance>.

Example: When payment-svc is +200ms slow, checkout p99 stays below 500ms.

2. Steady-state metric

  • Metric: <e.g., p99 checkout latency>
  • Baseline window: <e.g., 5 minutes pre-experiment>
  • Tolerance: <e.g., within ±5% of baseline>
  • Dashboard: <URL>

3. Attack

  • Type: [ ] latency [ ] error [ ] cpu [ ] memory [ ] disk [ ] network-partition [ ] dependency-failure [ ] time-skew [ ] kill-instance
  • Magnitude: <e.g., +200ms>
  • Duration: <minutes>
  • Target: <service / pod / instance / region>
  • Tooling: <Chaos Toolkit / Chaos Mesh / Litmus / Gremlin / AWS FIS / Custom>

4. Blast radius

  • Traffic share: <e.g., 5% of US>
  • Expected affected users: <from blast_radius_calculator.py>
  • Error budget consumed: <from blast_radius_calculator.py>
  • Risk score: [ ] GREEN [ ] YELLOW [ ] RED

5. Abort criteria

Auto-trigger experiment termination if ANY of these hit.

  • [ ] <signal 1, e.g., p99 > 1000ms>
  • [ ] <signal 2, e.g., 5xx rate > baseline + 1pp>
  • [ ] <signal 3, e.g., on-call paged SEV1/SEV2>

6. Rollback procedure

  1. <step to disable fault, e.g., "kubectl delete chaos networkchaos/<name>">
  2. Verify steady state recovers within 2 minutes
  3. If not recovering, escalate as incident; restore from backup if needed

7. Learning question

What do you expect NOT to learn? Force yourself to predict.

<your prediction>

Pre-flight checklist

  • [ ] Hypothesis written
  • [ ] Steady-state metric measured for ≥5 min
  • [ ] Blast radius calculated (GREEN or YELLOW only)
  • [ ] Abort criteria documented with thresholds
  • [ ] Rollback procedure tested in staging
  • [ ] On-call team notified
  • [ ] Monitoring dashboards open
  • [ ] Owner identified and reachable
  • [ ] Time-box agreed
  • [ ] Communication plan if abort triggers

Post-experiment

Run experiment_postmortem.py --plan <plan.json> --result-log <results> to generate the postmortem.