references/empty-experiment.md · 24000
EDIUM]
Eligibility filtering should happen before you call the flag — otherwise unaffected users are pulled into the analysis and the picture gets noisy. This shows up as exposures being much higher than expected and metric rates unexpectedly low.
The same shape has other sources: a flag that is read on more surfaces than the one under test, and an experiment on new users whose flag existing users also evaluate.
In each, the experiment counts people the decision is not about (D12 in numbers-vs-sql.md).
Fix: structure the code as: eligibility check → flag check → render. Not: flag check → eligibility → render. Where the code cannot change, narrow who counts: a filter in the exposure criteria, which removes a person only when every exposure event of that person fails it (B12 lists what such a filter can read), or a custom exposure event sent at the point of the change.
B10 — "Variant always undefined / false" [MEDIUM]
Almost always one of:
- B1 (wrong evaluation method), when the code reads a payload:
getFeatureFlagPayload()returns the payload, which is empty for a flag without one - B2 (
identify()timing), on a flag whose release conditions read person properties that are set atidentify(): a read before it gets no variant posthog is not defined(SDK init order — initialize PostHog before any flag call)- The code reads the flag before the SDK has loaded the flags.
posthog-js then returns
undefinedand sends no$feature_flag_called, unless the browser holds flags from an earlier visit - The flag is genuinely off —
feature_flag.active === false(statusispaused), or rollout0%, or the user is outside release conditions - The flag buckets on the device id, and the call carries none (A10 in
bias-and-skew.md) - The code reads the multivariate flag as a boolean and gets
true, or replaces "no variant" with a default of its own
Fix: walk the user through their SDK setup. Verify in this order: (a) is PostHog initialized? (b) is the flag active and rolled out? (c) is the right variant key being requested?
When only a part of the audience gets no variant, the question is who: A10 in bias-and-skew.md.
B11 — $feature/<key> is missing on metric events [HIGH]
For a current experiment this is not a fault.
Only the exposure event decides a person's variant.
Metric events are matched to that person and need no flag property, for every metric type.
A user who adds $feature/<flag-key> to metric events changes nothing in the results.
A legacy experiment shows no results at all (E13 in mid-run-changes.md).
Where the property does matter:
- A custom exposure event needs it (B4).
- The user's own insight or SQL that filters or breaks down by
$feature/<flag-key>. It undercounts events from a server SDK, which adds the property only when the capture call is given the evaluated flags or asked to evaluate them[MEDIUM]. It also overcounts in the other direction: the property is on events of everyone whose SDK holds the flag's value, including people who were never exposed, so it is not a filter for participants.
Verify directly. A missing conversion has another cause: B2 (ids), B8 (order), D1 in numbers-vs-sql.md (scope).
Part of the public docs still asks for the property on metric events. The exposures page states the current rule.
B12 — An exposure criteria filter that cannot match [MEDIUM]
Symptom (in the user's vocabulary): "exposures dropped to zero after I added a filter", "the experiment shows far fewer users than the flag", "I copied the experiment and the new one is empty".
Behavior. A filter in the exposure criteria is evaluated on each exposure event. It reads the event's own properties, and person properties either as they were attached to that event or as the person holds them now: which of the two depends on a project setting that the tools do not show. A filter that names something the exposure event cannot carry removes every exposure, and nothing reports it.
Shapes:
- A property the exposure event does not have. A browser property (device type, URL) on an exposure that a server sends.
- A person property that is set after the exposure, such as a sign-up timestamp on an exposure that happens before sign-up. This shape occurs only where person properties are read as they were attached to the event.
- A misspelled property name or value.
- Criteria copied from another experiment that name the other experiment's flag or event.
- An exposure event the project has never received.
- A URL filter, or another event-property filter, that is meant to keep people out. It drops the events that do not match. A person with one matching exposure event is in, with everything they did.
Evidence to gather.
exposure_criteria.exposure_configfromexperiment-get: the event and itsproperties.- The raw-responses query once without and once with the filter's condition. A count that falls to zero, or nearly, names the filter. One condition at a time when there are several.
- For a person-property filter: the raw-responses query with the same condition on
person.properties.<key>. That query reads person properties the same way the experiment does, so the check holds whichever way the project reads them.
Co-occurs with: B7, and D9 in numbers-vs-sql.md.
Recommendation. Remove or correct the filter.
Filter on what the exposure event carries.
To restrict the experiment to a group of people, use the flag's release conditions.
An edit of the exposure criteria recomputes the whole run (E5 in mid-run-changes.md).
If none of the above: the code path may not be running
Two sub-cases:
Never had exposures (the experiment has shown 0 since launch). After B1–B12, check the obvious:
- Has the deploy with the flag-reading code shipped to production?
- Is real traffic flowing through that code path?
- Does the code read the experiment's flag key, and not another key?
- Is the date range correct (start_date in the future, etc.)?
Ask explicitly. The "empty experiment" shape often resolves to a feature flag still on a feature branch that hasn't merged, or a page that calls the flag not being live yet.
Exposures were healthy then stopped (the experiment ran for weeks/months, then the exposure count plateaued and never moved again). Before investigating, check the experiment's status from Step 1:
exposure_frozen— someone froze exposure. The plateau is the intended behavior: enrollment is closed, metrics still flow (E16 inmid-run-changes.md).paused— the flag is off.stopped— the results end at the end date.
Then check where the project keeps its flag calls: project-get returns flag_evaluations_mode.
At 2, $feature_flag_called no longer arrives in events for the projects that setting covers, and an experiment that counts that event gains no exposures, without a warning.
That is no fault of the SDK or of the application: say so, and point the user to PostHog support.
Then read the flag's change history, before any conclusion about the application:
- A change of the flag after launch can explain the plateau by itself.
A split edited to 0% for one variant, a variant pinned on a broad release condition, or a rollout cut to 0 stops one arm or all of them while
statusstill readsrunning(A5, A6 and A7b inbias-and-skew.md, E3 inmid-run-changes.md). - No change of the flag after launch means that an edit of the rollout, the split or the release conditions cannot explain it. What is left: the application stopped calling the flag, a release condition on a cohort or a property stopped matching new people, the audience has no new people, or the project's test-account filter changed, which the experiment's history does not show.
Verify directly:
exposures.timeseriesin the exposure data: the cumulative count per variant stops growing. One arm flat while the other grows points to the flag or to one code path. All arms flat points to the application, or to an audience without new people: the series counts each person once.last_seenper variant in the raw-responses query, over the last days: no recent events means the application stopped sending them, or that only people who already sent the event still reach the code (B3). Both variants silent is a code-path removal, or an audience without new people. One variant silent while the other still fires is a one-sided change, in the flag or in the code.
Common causes:
- The flag-reading call was removed in a refactor (most common).
- The page or component that hosts the flag-read was deprecated or rerouted (e.g. URL restructuring moved the eligible traffic onto a different page that doesn't read this flag).
- A different flag is now serving the same UX (intentional migration that wasn't paired with ending the original experiment).
Recommend:
- If the hypothesis is settled enough: end the experiment with the appropriate conclusion (won / lost / inconclusive). The metric data accumulated before the plateau is the experiment's documented outcome. Don't ship the variant unless the code path is being restored — an "end + ship" on a dormant flag flips the variant distribution to a UX that isn't being served anyway.
- If you want to keep running the hypothesis: restore the flag-reading call in the application code, then either continue (and treat the pre-/post-resumption windows separately) or reset + relaunch for a clean comparison window.
SDK-side fallback. If B1, B2, and B10 are all on the table and you can't pin one down, invoke the
posthog:diagnosing-sdk-health skill — an outdated SDK is one possible cause of the "no exposures
at all" shape (missing instrumentation, broken identify() ordering, deprecated flag methods).