SKILL.md · 24000
ttern visually
llma-evaluation-*tools useevaluation:readfor read tools andevaluation:writefor mutating tools, exceptllma-evaluation-config-getandllma-evaluation-judge-models, which usellm_analytics:read- Hog evaluators are reproducible — if you suspect a regression,
llma-evaluation-test-hogwith the suspect source against the failing generations is the fastest way to bisect whether the change is in the evaluator or in the producer of the generations - LLM-judge evaluators are non-deterministic across reruns; expect 1-5% noise even with
a fixed prompt and model. If you're chasing a small regression in fail rate, prefer
Hog or pin a deterministic provider/seed in the
model_configuration
Related skills
creating-online-evaluations— author a new evaluation from scratchexploring-ai-failures— ground the next evaluation in observed failure modes