AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

authoring-log-alerts

Author useful, low-noise log alerts on services in a PostHog project. Use when the user asks to set up alerts for their logs, suggest alerts they should add, or evaluate whether a service is worth monitoring. Covers service triage, baseline characterisation, threshold drafting, back-testing via simulate, and shipping with a notification destination.

Original source
posthog
License
MIT
Pinned revision
cac7c80a9e5b73484d18eda1bffb47297c938e0c

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read posthog--authoring-log-alerts at revision cac7c80a9e5b73484d18eda1bffb47297c938e0c, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 3

references/volume-floor-alerts.md

Volume-floor alerts (threshold_operator: below)

Use to alert when a service stops producing logs — a strong signal that the service is down, the logging pipeline is broken, or something upstream stopped sending traffic.

When to use

  • The user says "tell me if X stops logging."
  • A service has dependably constant volume across all hours (no nightly drop, no weekend drop) and the user cares about availability.

When NOT to use

  • The service has any quiet hours (overnight, weekends, batch-only). Below-threshold alerts will fire every quiet period.
  • The service is bursty by design — a 5-minute "no logs" window is normal.
  • You don't have at least 7 days of stable baseline. With less data, "below normal" is unknowable.

How to size the threshold

The baseline_stats.py script doesn't suggest below-thresholds directly — derive it manually from the script's output. Don't compute by hand; use Python:

python3 -c "import sys, json; d=json.load(sys.stdin); print(max(1, round(d['stats']['p50'] * (5/d['bucket_minutes']) * 0.25)))" < stats.json

The reasoning:

  1. Take p50 from the script (the typical bucket count).
  2. Rate-scale to the alert window (window_minutes / bucket_minutes).
  3. Multiply by 0.25 — fire when volume drops to 25% of the typical bucket. Buffer absorbs normal variance.
  4. Set threshold_operator: below.

Recommended N-of-M for floors

  • evaluation_periods: 3, datapoints_to_alarm: 3 (3-of-3) — require 3 consecutive quiet windows. A single blip won't fire.
  • window_minutes: 15 minimum — 5-minute floor alerts on noisy services are unreliable.
  • cooldown_minutes: 60 — a floor alert can flap near the threshold, and this keeps it to one notification per hour.

Simulate is essential here

Floor alerts are easy to misconfigure into "fires every night." Always run posthog:logs-alerts-simulate-create over -7d before shipping. If fire_count > 0 and the user has not had outages in the last 7 days, the threshold is too aggressive.