AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

configuring-experiment-analytics

Configures the analytics side of a PostHog experiment — exposure criteria (server-resolved default exposure event vs custom exposure events), primary and secondary metrics, the supported metric types (count, sum, ratio with `math` and `math_property`, retention with `retention_window_start` and `start_handling`), multivariate user handling ("Exclude from analysis" vs "Use first seen variant"), and how to read results once the experiment is live. Use when the user adds or edits a primary or secondary metric (e.g. "add a secondary metric tracking 'downloaded_file' per user"), sets up a ratio metric (e.g. "revenue from purchase_completed / pageviews"), sets up a retention metric (e.g. "$pageview → uploaded_file, 7-day window"), configures custom exposure (e.g. "only count users who hit /checkout"), changes multivariate handling, or asks "who is in the analysis?", "how do I measure impact?", "is this winning?", "what's the confidence level?", or "should I ship?".

Original source
posthog
License
MIT
Pinned revision
cac7c80a9e5b73484d18eda1bffb47297c938e0c

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

Ask your assistant

Use Agentic Memory skills to read posthog--configuring-experiment-analytics at revision cac7c80a9e5b73484d18eda1bffb47297c938e0c, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 4

references/metric-templates.md

Metric templates

Most metric requests match one of these shapes. Pick the template, fill in the project's events (confirmed with read-data-schema), and set its defaults. Check for a shared metric first (Step 1 in SKILL.md): a matching shared metric beats any template.

Every conversion window needs a unit: set conversion_window and conversion_window_unit together, or neither. A window without a unit is ignored and the metric counts events until the experiment ends.

| Template | Use when the user asks | Shape | Defaults to set | Say in the summary | | ------------------- | -------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Conversion | "more people do X" | funnel, one step (exposure is step 0) | goal: "increase". A window with a unit when X usually happens within a known delay (a signup within a day) | X must be captured for the exposed person, with the same identity | | Journey | "more people get from A through B to C" | funnel, 2 to 4 ordered steps | Same as conversion | Numbers won't match a product analytics funnel: exposure comes first and test accounts are removed | | Count per user | "people do X more often" | mean, math: "total" | upper_bound_percentile: 0.99 | Heavy users can dominate the mean; the upper bound caps them | | Revenue per user | "more revenue", "people spend more" | mean, math: "sum", math_property = the revenue property | upper_bound_percentile: 0.99 | A few large orders make revenue noisy. If revenue arrives on several events or properties, the metric must cover all of them. Server-side purchase events need the same identity as the exposure | | Average value | "X gets faster", "sessions get longer" | mean, math: "avg" with math_property = the value property | goal: "decrease" when lower is better (latency, load time), otherwise "increase" | Server-side events have no session | | Unique-user rate | "a larger share of users do X" relative to another group | ratio, math: "dau" (unique users) on both numerator and denominator | None | A few heavy users can move a total-over-total ratio; unique users avoids that | | Average order value | "bigger orders" | ratio, numerator math: "sum" of the revenue property, denominator math: "total" of the same event | numerator_outlier_handling: {"upper_bound_percentile": 0.99} (ratio metrics take no top-level upper_bound_percentile) | Read it together with conversion and revenue per user: fewer, larger orders can raise it while revenue falls | | Return rate | "people come back", "retention" | retention, start_handling: "first_seen", retention_window_start: 1, retention_window_end: 7, retention_window_unit: "day" | Completed window only (see below) | Users exposed near the end have had less time to return and pull the rate down | | Trial to paid | "more trials convert to paid" | funnel with a fixed window | A window with its unit, longer than the trial: it starts at exposure, not at the trial start, so it must cover the time to start the trial plus the trial. Completed window only (see below) | One window must cover both the trial start and the payment |

Completed window only

The experiment setting "Require completed conversion or retention window" (only_count_matured_users) counts only people whose whole window has passed. Creation turns it on when the team's default is on (team_defaults.only_count_matured_users in experiment-setup-context). The agent cannot set it: experiment-create and experiment-update don't expose it. For return rate and trial to paid, when the team default is off, recommend it in the summary and tell the user to turn it on in the experiment's Settings tab.

Which template is primary

The primary metric decides the ship call, so it should be the most frequent event that still expresses the hypothesis. If the requested event is rare on this page (the running time comes out at months), make it secondary and propose an earlier, more frequent event as primary.