AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

phoenix-evals

Build and run evaluators for AI/LLM applications using Phoenix.

Original source
github
License
MIT
Pinned revision
5269f7ea2d6c3a9bfda5ed878b6726cf509121fd

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

Ask your assistant

Use Agentic Memory skills to read github--phoenix-evals at revision 5269f7ea2d6c3a9bfda5ed878b6726cf509121fd, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 35

references/error-analysis-multi-turn.md

Error Analysis: Multi-Turn Conversations

Debugging complex multi-turn conversation traces.

The Approach

  1. End-to-end first - Did the conversation achieve the goal?
  2. Find first failure - Trace backwards to root cause
  3. Simplify - Try single-turn before multi-turn debug
  4. N-1 testing - Isolate turn-specific vs capability issues

Find First Upstream Failure

Turn 1: User asks about flights ✓
Turn 2: Assistant asks for dates ✓
Turn 3: User provides dates ✓
Turn 4: Assistant searches WRONG dates ← FIRST FAILURE
Turn 5: Shows wrong flights (consequence)
Turn 6: User frustrated (consequence)

Focus on Turn 4, not Turn 6.

Simplify First

Before debugging multi-turn, test single-turn:

# If single-turn also fails → problem is retrieval/knowledge
# If single-turn passes → problem is conversation context
response = chat("What's the return policy for electronics?")

N-1 Testing

Give turns 1 to N-1 as context, test turn N:

context = conversation[:n-1]
response = chat_with_context(context, user_message_n)
# Compare to actual turn N

This isolates whether error is from context or underlying capability.

Checklist

  1. Did conversation achieve goal? (E2E)
  2. Which turn first went wrong?
  3. Can you reproduce with single-turn?
  4. Is error from context or capability? (N-1 test)