AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

secrets-vault-manager

Use when setting up HashiCorp Vault, AWS Secrets Manager, Azure Key Vault or GCP Secret Manager, rotating secrets or auditing secret access. Generates config with vault_config_generator.py, plans rotation with rotation_planner.py and analyzes audit logs.

Original source
alirezarezvani
License
MIT
Pinned revision
e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read alirezarezvani--secrets-vault-manager at revision e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 4

references/emergency_procedures.md

Emergency Procedures Reference

Secret Leak Response Playbook

Severity Classification

| Severity | Definition | Response Time | Example | |----------|-----------|---------------|---------| | P0 — Critical | Production credentials exposed publicly | Immediate (15 min) | Database password in public GitHub repo | | P1 — High | Internal credentials exposed beyond intended scope | 1 hour | API key in build logs accessible to wider org | | P2 — Medium | Non-production credentials exposed | 4 hours | Staging DB password in internal wiki | | P3 — Low | Expired or limited-scope credential exposed | 24 hours | Rotated API key found in old commit history |

P0/P1 Response Procedure

Phase 1: Contain (0-15 minutes)

  1. Identify the leaked secret

    • What credential was exposed? (type, scope, permissions)
    • Where was it exposed? (repo, log, error page, third-party service)
    • When was it first exposed? (commit timestamp, log timestamp)
    • Is the exposure still active? (repo public? log accessible?)
  2. Revoke immediately

    • Database password: ALTER ROLE app_user WITH PASSWORD 'new_password';
    • API key: Regenerate via provider console/API
    • Vault token: vault token revoke <token>
    • AWS access key: aws iam delete-access-key --access-key-id <key>
    • Cloud service account: Delete and recreate key
    • TLS certificate: Revoke via CA, generate new certificate
  3. Remove exposure

    • Public repo: Remove file, force-push to remove from history, request GitHub cache purge
    • Build logs: Delete log artifacts, rotate CI/CD secrets
    • Error page: Deploy fix to suppress secret in error output
    • Third-party: Contact vendor for log purge if applicable
  4. Deploy new credentials

    • Update secret store with rotated credential
    • Restart affected services to pick up new credential
    • Verify services are healthy with new credential

Phase 2: Assess (15-60 minutes)

  1. Audit blast radius

    • Query Vault/cloud SM audit logs for the compromised credential
    • Check for unauthorized usage during the exposure window
    • Review network logs for suspicious connections from unknown IPs
    • Check if the compromised credential grants access to other secrets (privilege escalation)
  2. Notify stakeholders

    • Security team (always)
    • Service owners for affected systems
    • Compliance team if regulated data was potentially accessed
    • Legal if customer data may have been compromised
    • Executive leadership for P0 incidents

Phase 3: Recover (1-24 hours)

  1. Rotate adjacent credentials

    • If the leaked credential could access other secrets, rotate those too
    • If a Vault token leaked, check what policies it had — rotate everything accessible
  2. Harden against recurrence

    • Add pre-commit hook to detect secrets (e.g., gitleaks, detect-secrets)
    • Review CI/CD pipeline for secret masking
    • Audit who has access to the source of the leak

Phase 4: Post-Mortem (24-72 hours)

  1. Document incident
    • Timeline of events
    • Root cause analysis
    • Impact assessment
    • Remediation actions taken
    • Preventive measures added

Response Communication Template

SECURITY INCIDENT — SECRET EXPOSURE
Severity: P0/P1
Time detected: YYYY-MM-DD HH:MM UTC
Secret type: [database password / API key / token / certificate]
Exposure vector: [public repo / build log / error output / other]
Status: [CONTAINED / INVESTIGATING / RESOLVED]

Immediate actions taken:
- [ ] Credential revoked at source
- [ ] Exposure removed
- [ ] New credential deployed
- [ ] Services verified healthy
- [ ] Audit log review in progress

Blast radius assessment: [PENDING / COMPLETE — no unauthorized access / COMPLETE — unauthorized access detected]

Next update: [time]
Incident commander: [name]

Vault Seal/Unseal Procedures

Understanding Seal Status

Vault uses a seal mechanism to protect the encryption key hierarchy. When sealed, Vault cannot decrypt any data or serve any requests.

Sealed State:
  Vault process running → YES
  API responding → YES (503 Sealed)
  Serving secrets → NO
  All active leases → FROZEN (not revoked)
  Audit logging → NO

Unsealed State:
  Vault process running → YES
  API responding → YES (200 OK)
  Serving secrets → YES
  Active leases → RESUMING
  Audit logging → YES

When to Seal Vault (Emergency Only)

Seal Vault when:

  • Active intrusion on Vault infrastructure is confirmed
  • Vault server compromise is suspected (unauthorized root access)
  • Encryption key material may have been extracted
  • Regulatory/legal hold requires immediate data access prevention

Do NOT seal for:

  • Routine maintenance (use graceful shutdown instead)
  • Single-node issues in HA cluster (let standby take over)
  • Suspected secret leak (revoke the secret, don't seal Vault)

Seal Procedure

# Seal a single node
vault operator seal

# Seal all nodes (HA cluster)
# Seal each node individually — leader last
vault operator seal -address=https://vault-standby-1:8200
vault operator seal -address=https://vault-standby-2:8200
vault operator seal -address=https://vault-leader:8200

Impact of sealing:

  • All active client connections dropped immediately
  • All token and lease timers paused
  • Applications lose secret access — prepare for cascading failures
  • Monitoring will fire alerts for sealed state

Unseal Procedure (Shamir Keys)

Requires a quorum of key holders (e.g., 3 of 5).

# Each key holder provides their unseal key
vault operator unseal <key-1>
vault operator unseal <key-2>
vault operator unseal <key-3>
# Vault unseals after reaching threshold

Operational checklist after unseal:

  1. Verify health: vault status shows Sealed: false
  2. Check audit devices: vault audit list — confirm all enabled
  3. Check auth methods: vault auth list
  4. Verify HA status: vault operator raft list-peers
  5. Check lease count: monitor vault.expire.num_leases
  6. Verify applications reconnecting (check application logs)

Unseal Procedure (Auto-Unseal)

If using cloud KMS auto-unseal, Vault unseals automatically on restart:

# Restart Vault service
systemctl restart vault

# Verify unseal (should happen within seconds)
vault status

If auto-unseal fails:

  • Check cloud KMS key permissions (IAM role may have been modified)
  • Check network connectivity to cloud KMS endpoint
  • Check KMS key status (not disabled, not scheduled for deletion)
  • Check Vault logs: journalctl -u vault -f

Mass Credential Rotation Procedure

When a broad compromise requires rotating many credentials simultaneously.

Pre-Rotation Checklist

  • [ ] Identify all credentials in scope
  • [ ] Map credential dependencies (which services use which credentials)
  • [ ] Determine rotation order (databases before applications)
  • [ ] Prepare rollback plan for each credential
  • [ ] Notify all service owners
  • [ ] Schedule maintenance window if zero-downtime not possible
  • [ ] Stage new credentials in secret store (but don't activate yet)

Rotation Order

  1. Infrastructure credentials — Database root passwords, cloud IAM admin keys
  2. Service credentials — Application database users, API keys
  3. Integration credentials — Third-party API keys, webhook secrets
  4. Human credentials — Force password reset, revoke SSO sessions

Rollback Plan

For each credential, document:

  • Previous value (store in sealed emergency envelope or HSM)
  • How to revert (specific command or API call)
  • Verification step (how to confirm old credential works)
  • Maximum time to rollback (SLA)

Vault Recovery Procedures

Lost Unseal Keys

If unseal keys are lost and auto-unseal is not configured:

  1. If Vault is currently unsealed: Enable auto-unseal immediately, then reseal/unseal with KMS
  2. If Vault is sealed: Data is irrecoverable without keys. Restore from Raft snapshot backup
  3. Prevention: Store unseal keys in separate, secure locations (HSMs, safety deposit boxes). Use auto-unseal for production.

Raft Cluster Recovery

Single node failure (cluster still has quorum):

# Remove failed peer
vault operator raft remove-peer <failed-node-id>

# Add replacement node
# (new node joins via retry_join in config)

Loss of quorum (majority of nodes failed):

# On a surviving node with recent data
vault operator raft join [email protected] https://surviving-node:8200

# If no node survives, restore from snapshot
vault operator raft snapshot restore /backups/latest.snap

Root Token Recovery

If root token is lost (it should be revoked after initial setup):

# Generate new root token (requires unseal key quorum)
vault operator generate-root -init
# Each key holder provides their key
vault operator generate-root -nonce=<nonce> <unseal-key>
# After quorum, decode the encoded token
vault operator generate-root -decode=<encoded-token> -otp=<otp>

Best practice: Generate a root token only when needed, complete the task, then revoke it:

vault token revoke <root-token>

Incident Severity Escalation Matrix

| Signal | Escalation | |--------|-----------| | Single secret exposed in internal log | P2 — Rotate secret, add log masking | | Secret in public repository (no evidence of use) | P1 — Immediate rotation, history scrub | | Secret in public repository (evidence of unauthorized use) | P0 — Full incident response, legal notification | | Vault node compromised | P0 — Seal cluster, rotate all accessible secrets | | Cloud KMS key compromised | P0 — Create new key, re-encrypt all secrets, rotate all credentials | | Audit log gap detected | P1 — Investigate cause, assume worst case for gap period | | Multiple failed auth attempts from unknown source | P2 — Block source, investigate, rotate targeted credentials |