AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

microsoft-foundry

Build, deploy, evaluate, optimize, fine-tune, and manage Microsoft Foundry agents, models, and resources end to end. USE FOR: foundry, azd ai agent, azd provision/deploy, hosted agent scaffold/develop/run/deploy/troubleshoot, prompt agent, managed harness agent, Copilot harness, voice agent, create/update/invoke/evaluate agent, add tool to agent, agent insights, batch/continuous eval, continuous monitoring, agent CI/CD, optimize/improve prompt, optimize agent instructions, Agent Optimizer, dataset curation from traces, deploy model, model fine-tuning (SFT/DPO/RFT), Foundry project, RBAC, role assignment, permissions, quota, capacity, region, deployment failure, AI Services, create Foundry resource, knowledge index, customize deployment, onboard, availability, training-data, grader, distillation, large file upload. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure-prepare).

Original source
microsoft
License
MIT
Pinned revision
3f5b461b2314e01c400564051e92469114c5b68f

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read microsoft--microsoft-foundry at revision 3f5b461b2314e01c400564051e92469114c5b68f, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 151

foundry-agent/trace/references/kql-templates.md

KQL Templates — GenAI Trace Query Reference

Ready-to-use KQL templates for querying GenAI OpenTelemetry traces in Application Insights.

Table of Contents: App Insights Table Mapping · Key GenAI OTel Attributes · Span Correlation · Hosted Agent Attributes · Response ID Formats · Common Query Templates · OTel Reference Links

App Insights Table Mapping

| App Insights Table | GenAI Data | |-------------------|------------| | dependencies | GenAI spans: LLM inference (chat), tool execution (execute_tool), agent invocation (invoke_agent) | | requests | Incoming HTTP requests to the agent endpoint. For hosted agents, also carries gen_ai.agent.name (Foundry name) and azure.ai.agentserver.* attributes — preferred entry point for agent-name filtering | | customEvents | GenAI evaluation results (gen_ai.evaluation.result) — scores, labels, explanations | | traces | Log events, including GenAI events (input/output messages) | | exceptions | Error details with stack traces |

Key GenAI OTel Attributes

Stored in customDimensions on dependencies spans:

| Attribute | Description | Example | |-----------|-------------|---------| | gen_ai.operation.name | Operation type | chat, invoke_agent, execute_tool, create_agent | | gen_ai.conversation.id | Conversation/session ID | conv_5j66UpCpwteGg4YSxUnt7lPY | | gen_ai.response.id | Response ID | chatcmpl-123 | | gen_ai.agent.name | Agent name | my-support-agent | | gen_ai.agent.id | Agent identifier | asst_abc123 | | gen_ai.request.model | Requested model | gpt-4o | | gen_ai.response.model | Actual model used | gpt-4o-2024-05-13 | | gen_ai.usage.input_tokens | Input token count | 450 | | gen_ai.usage.output_tokens | Output token count | 120 | | gen_ai.response.finish_reasons | Stop reasons | ["stop"], ["tool_calls"] | | error.type | Error classification | timeout, rate_limited, content_filter | | gen_ai.provider.name | Provider | azure.ai.openai, openai | | gen_ai.input.messages | Full input messages (JSON array) — on invoke_agent spans | [{"role":"user","parts":[{"type":"text","content":"..."}]}] | | gen_ai.output.messages | Full output messages (JSON array) — on invoke_agent spans | [{"role":"assistant","parts":[{"type":"text","content":"..."}]}] |

Stored in customDimensions on customEvents (name == gen_ai.evaluation.result):

| Attribute | Description | Example | |-----------|-------------|---------| | gen_ai.evaluation.name | Evaluator name | Relevance, IntentResolution | | gen_ai.evaluation.score.value | Numeric score | 4.0 | | gen_ai.evaluation.score.label | Human-readable label | pass, fail, relevant | | gen_ai.evaluation.explanation | Free-form explanation | "Response lacks detail..." | | gen_ai.response.id | Correlates to the evaluated span | chatcmpl-123 | | gen_ai.conversation.id | Correlates to conversation | conv_5j66... |

Correlation: Eval results do NOT link via id-parentId. Use gen_ai.conversation.id and/or gen_ai.response.id to join with dependencies spans.

Span Correlation

| Field | Purpose | |-------|---------| | operation_Id | Trace ID — groups all spans in one request | | id | Span ID — unique identifier for this span | | operation_ParentId | Parent span ID — use with id to build span trees |

Operation_Id Join (requests → dependencies)

Use requests as the hosted-agent entry point, then carry operation_Id forward as the trace key when joining into dependencies, traces, or customEvents:

let agentRequests = materialize(
    requests
| where timestamp > ago(7d)
| extend
    foundryAgentName = coalesce(
        tostring(customDimensions["gen_ai.agent.name"]),
        tostring(customDimensions["azure.ai.agentserver.agent_name"])
    ),
    agentId = tostring(customDimensions["gen_ai.agent.id"]),
    agentNameFromId = tostring(split(agentId, ":")[0]),
    agentVersion = iff(agentId contains ":", tostring(split(agentId, ":")[1]), ""),
    conversationId = coalesce(
        tostring(customDimensions["gen_ai.conversation.id"]),
        tostring(customDimensions["azure.ai.agentserver.conversation_id"]),
        operation_Id
    )
| where foundryAgentName == "<foundry-agent-name>"
    or agentNameFromId == "<foundry-agent-name>"
| project operation_Id, conversationId, agentVersion
);
dependencies
| where timestamp > ago(7d)
| where isnotempty(customDimensions["gen_ai.operation.name"])
| join kind=inner agentRequests on operation_Id
| extend
    operation = tostring(customDimensions["gen_ai.operation.name"]),
    model = tostring(customDimensions["gen_ai.request.model"])
| project timestamp, duration, success, operation, model, conversationId, agentVersion, operation_Id
| order by timestamp desc

Hosted Agent Attributes

Stored in customDimensions on both requests and traces tables (NOT on dependencies spans):

| Attribute | Description | Example | |-----------|-------------|---------| | azure.ai.agentserver.agent_name | Hosted agent name | hosted-agent-022-001 | | azure.ai.agentserver.agent_id | Internal agent ID | code-asst-xmwokux85uqc7fodxejaxa | | azure.ai.agentserver.conversation_id | Conversation ID | conv_d7ab624de92d... | | azure.ai.agentserver.response_id | Response ID (caresp format) | caresp_d7ab624de92d... |

Important: Use requests as the preferred entry point for agent-name filtering — it has both azure.ai.agentserver.agent_name and gen_ai.agent.name with the Foundry-level name. To reach downstream spans and related telemetry, carry operation_Id forward from the filtered request set and join other tables on that trace key.

💡 Version enrichment: Some hosted-agent requests telemetry emits gen_ai.agent.id in <foundry-agent-name>:<version> format. When that delimiter is present, split on : to recover agentVersion; if it is absent, keep filtering on the requests-scoped name fields and leave version blank.

⚠️ gen_ai.agent.name means different things on different tables:

  • On requests: the Foundry agent name (user-visible) → e.g., hosted-agent-022-001
  • On dependencies: the code-level class name → e.g., BingSearchAgent

Always start from requests when filtering by the Foundry agent name the user knows.

Response ID Formats

| Agent Type | Prefix | Example | |------------|--------|---------| | Hosted agent (AgentServer) | caresp_ | caresp_d7ab624de92da637008Rhr4U4E1y9FSE... | | Prompt agent (Foundry Responses API) | resp_ | resp_4e2f8b016b5a0dad00697bd3c4c1b881... | | Azure OpenAI chat completions | chatcmpl- | chatcmpl-abc123def456 |

When searching by response ID, use the appropriate prefix to narrow results. The gen_ai.response.id attribute appears on dependencies spans (for chat operations) and in customEvents (for evaluation results).

Common Query Templates

Overview — Conversations in last 24h

dependencies
| where timestamp > ago(24h)
| where isnotempty(customDimensions["gen_ai.operation.name"])
| summarize
    spanCount = count(),
    errorCount = countif(success == false),
    avgDuration = avg(duration),
    totalInputTokens = sum(toint(customDimensions["gen_ai.usage.input_tokens"])),
    totalOutputTokens = sum(toint(customDimensions["gen_ai.usage.output_tokens"]))
  by bin(timestamp, 1h)
| order by timestamp desc

Error Rate by Operation

dependencies
| where timestamp > ago(24h)
| where isnotempty(customDimensions["gen_ai.operation.name"])
| summarize
    total = count(),
    errors = countif(success == false),
    errorRate = round(100.0 * countif(success == false) / count(), 1)
  by operation = tostring(customDimensions["gen_ai.operation.name"])
| order by errorRate desc

Token Usage by Model

dependencies
| where timestamp > ago(24h)
| where customDimensions["gen_ai.operation.name"] == "chat"
| summarize
    calls = count(),
    totalInput = sum(toint(customDimensions["gen_ai.usage.input_tokens"])),
    totalOutput = sum(toint(customDimensions["gen_ai.usage.output_tokens"])),
    avgInput = avg(todouble(customDimensions["gen_ai.usage.input_tokens"])),
    avgOutput = avg(todouble(customDimensions["gen_ai.usage.output_tokens"]))
  by model = tostring(customDimensions["gen_ai.request.model"])
| order by totalInput desc

Tool Call Details

dependencies
| where operation_Id == "<operation_id>"
| where customDimensions["gen_ai.operation.name"] == "execute_tool"
| project timestamp, duration, success,
    toolName = tostring(customDimensions["gen_ai.tool.name"]),
    toolType = tostring(customDimensions["gen_ai.tool.type"]),
    toolCallId = tostring(customDimensions["gen_ai.tool.call.id"]),
    toolArgs = tostring(customDimensions["gen_ai.tool.call.arguments"]),
    toolResult = tostring(customDimensions["gen_ai.tool.call.result"])
| order by timestamp asc

Key tool attributes:

| Attribute | Description | Example | |-----------|-------------|---------| | gen_ai.tool.name | Tool function name | remote_functions.bing_grounding, python | | gen_ai.tool.type | Tool type | extension, function | | gen_ai.tool.call.id | Unique call ID | call_db64aa6a004a... | | gen_ai.tool.call.arguments | JSON arguments passed | {"query": "latest AI news"} | | gen_ai.tool.call.result | Tool output (may be truncated) | <<ImageDisplayed>> |

Evaluation Results by Conversation

customEvents
| where timestamp > ago(24h)
| where name == "gen_ai.evaluation.result"
| extend
    evalName = tostring(customDimensions["gen_ai.evaluation.name"]),
    score = todouble(customDimensions["gen_ai.evaluation.score.value"]),
    label = tostring(customDimensions["gen_ai.evaluation.score.label"]),
    conversationId = tostring(customDimensions["gen_ai.conversation.id"])
| summarize
    evalCount = count(),
    avgScore = avg(score),
    failCount = countif(label == "fail" or label == "not_relevant" or label == "incorrect"),
    evaluators = make_set(evalName)
  by conversationId
| order by failCount desc

For detailed eval queries by response ID or conversation ID, see Eval Correlation.

OTel Reference Links