AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

microsoft-foundry

Build, deploy, evaluate, optimize, fine-tune, and manage Microsoft Foundry agents, models, and resources end to end. USE FOR: foundry, azd ai agent, azd provision/deploy, hosted agent scaffold/develop/run/deploy/troubleshoot, prompt agent, managed harness agent, Copilot harness, voice agent, create/update/invoke/evaluate agent, add tool to agent, agent insights, batch/continuous eval, continuous monitoring, agent CI/CD, optimize/improve prompt, optimize agent instructions, Agent Optimizer, dataset curation from traces, deploy model, model fine-tuning (SFT/DPO/RFT), Foundry project, RBAC, role assignment, permissions, quota, capacity, region, deployment failure, AI Services, create Foundry resource, knowledge index, customize deployment, onboard, availability, training-data, grader, distillation, large file upload. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure-prepare).

Original source
microsoft
License
MIT
Pinned revision
3f5b461b2314e01c400564051e92469114c5b68f

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read microsoft--microsoft-foundry at revision 3f5b461b2314e01c400564051e92469114c5b68f, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 151

foundry-agent/invocations-ws/invocations-ws.md

Invocations WebSocket (invocations_ws) Protocol

Build, deploy, and connect to Foundry hosted agents that expose a duplex WebSocket endpoint instead of an HTTP request/response surface. Use this for real-time, bidirectional workloads — voice agents, live transcripts, custom streaming protocols, and signaling for out-of-band media transports.

ℹ️ Generally available. invocations_ws is GA — no preview feature flag is required. Every upgrade must carry the required api-version=v1 query parameter. For current region availability see Foundry Hosted Agents — region availability.

Migrating from preview: the old HostedAgents=V1Preview gate (sent via the foundry_features query parameter or the Foundry-Features header) has been removed — stop sending it. Project and agent are now path segments instead of query parameters.

Quick Reference

| Property | Value | |----------|-------| | Agent type | Hosted (Bring Your Own container) only | | Protocol id (azure.yaml) | invocations_ws | | Recommended version | 1.0.0 | | Container route | WS /invocations_ws (served by azure-ai-agentserver-invocations; the host binds the port and probes for you) | | Foundry-side URL | wss://{account}.services.ai.azure.com/api/projects/{project}/agents/{agentName}/endpoint/protocols/invocations_ws?api-version=v1&agent_session_id={sessionId} | | Auth | Authorization: Bearer <Entra token> for scope https://ai.azure.com/.default | | Wire format | Developer-defined (binary frames, JSON text frames, protobuf, raw PCM — anything) | | Session affinity | Per-connection, keyed by the agent_session_id query parameter (optional — auto-generated if omitted) | | Multi-turn / state | Agent-managed inside the container; platform does not store history |

When to Use This Skill

  • Build or operate a hosted real-time voice agent (audio in / audio out, control frames)
  • Bridge an out-of-band media transport (WebRTC, SFU, telephony) to a Foundry-hosted bot via WebSocket signaling
  • Stream events bidirectionally that don't fit responses (OpenAI-compatible) or invocations (single bytes-in/bytes-out HTTP)
  • Connect a browser or native client to an already-deployed invocations_ws agent

ℹ️ For HTTP-based invocation (single request/response, OpenAI responses API, or custom HTTP invocations), use the invoke skill instead.

Protocol Comparison

| Aspect | responses | invocations | invocations_ws | |--------|-------------|---------------|------------------| | Transport | HTTPS | HTTPS | WebSocket (wss://) | | Lifetime | Per request | Per request | Long-lived duplex | | Wire format | OpenAI-compatible JSON | Raw bytes (developer-defined) | Frames, developer-defined | | History | Platform via conversationId | Agent-managed | Agent-managed via agent_session_id | | Streaming | stream: true (SSE) | Agent-controlled | Native duplex | | Best for | Chat | Webhooks / classifiers / protocol bridges | Voice, signaling, real-time |

Workflow

Step 1: Author the Container

Use the azure-ai-agentserver-invocations host — the same package that serves HTTP /invocations — and register a WebSocket handler with @app.ws_handler. The host runs the server, binds the port, exposes /readiness, handles await websocket.accept(), runs Ping/Pong keep-alive (default 30s), maps uncaught handler exceptions to close code 1011, and emits the structured close event used by azd ai agent monitor. You can register @app.invocation_handler (HTTP POST /invocations) and @app.ws_handler (WebSocket GET /invocations_ws) on the same app.

from azure.ai.agentserver.invocations import InvocationAgentServerHost
from starlette.websockets import WebSocket

app = InvocationAgentServerHost()

@app.ws_handler                    # GET /invocations_ws (WebSocket upgrade)
async def ws(websocket: WebSocket) -> None:
    await run_bot(websocket)       # your duplex protocol lives here

app.run()

Inside the handler, read the session id from FOUNDRY_AGENT_SESSION_ID (env var set by the host), or fall back to the agent_session_id query parameter. The container does not see the Authorization header — APIM and the Agents service strip it after validation, so don't depend on it and don't accept an authorization query parameter.

⚠️ You define the wire format. The platform forwards frames as-is in both directions. There is no schema validation, no OpenAPI registration, no platform-managed history. Document your protocol for callers.

See Invocations WebSocket Protocol Guide for the framing model, the agent_session_id query parameter, control-vs-data frame patterns, and discovery guidance.

Step 2: Declare the Protocol in azure.yaml

In the agent's service block (host: azure.ai.agent):

services:
  my-ws-agent:
    host: azure.ai.agent
    kind: hosted
    name: my-ws-agent
    protocols:
      - protocol: invocations_ws
        version: 1.0.0
    container:
      resources:
        cpu: "1"          # voice/media: at least 1 vCPU / 2 GiB; up to 2 vCPU / 4 GiB
        memory: 2Gi
    environmentVariables:
      - name: SOME_SECRET
        value: ${SOME_SECRET}
      # Resolve every secret from the azd environment; do not bake values into the image.

The matching agent.manifest.yaml declares the same protocol: invocations_ws under template.protocols.

⚠️ The default azd scaffold uses 0.25 cpu / 0.5Gi, which is too small for most real-time workloads. Bump resources before deploying.

Step 3: Deploy via azd

Use the standard hosted-agent flow from the deploy skill:

mkdir ~/azd-deploys/my-ws-agent && cd ~/azd-deploys/my-ws-agent
azd ai agent init -m <path>/agent.manifest.yaml -p <project-resource-id> --no-prompt
# azd env set ... for every variable referenced in azure.yaml
azd deploy my-ws-agent

Once Running, the Foundry endpoint is reachable at the URL pattern in the Quick Reference table above.

Step 4: Connect a Client

Connect to the Foundry-side WebSocket directly:

  1. Mint an Entra token for the audience https://ai.azure.com:

    az account get-access-token --resource https://ai.azure.com --query accessToken -o tsv
    
  2. Build the upstream URL. Project and agent are path segments (mirroring the HTTP /invocations shape); api-version=v1 is a required query parameter. The agent_session_id query parameter is optional — if you omit it the platform generates one; supply your own (URL-safe; see Session Management for ID format) only when you need to resume an existing session:

    wss://{account}.services.ai.azure.com/api/projects/{project}/agents/{agentName}/endpoint/protocols/invocations_ws
      ?api-version=v1
      &agent_session_id={your-id}        # optional
    
  3. Open the WebSocket with header Authorization: Bearer <token>. Browser code typically needs a small server-side proxy because the browser WebSocket constructor cannot set headers.

  4. Speak your protocol. Send and receive whatever your container expects.

Step 5: Multi-turn / Session State

There is no platform-managed history. To correlate frames across reconnects or keep per-user state, reuse the same agent_session_id and key your state off it inside the container. See Session Management.

Step 6: Observe and Troubleshoot

Stream container logs while testing:

azd ai agent monitor my-ws-agent --follow
# scope to a single connection
azd ai agent monitor my-ws-agent --session-id <agent_session_id> --follow

The same agent_session_id can be used to stream container logs (see the troubleshoot skill for deeper diagnostics).

Error Handling

| Error | Cause | Resolution | |-------|-------|------------| | HTTP 401 / 403 on WS upgrade | Missing or stale Entra token | Re-run az account get-access-token --resource https://ai.azure.com; ensure the caller has Foundry data-plane RBAC | | HTTP 404 on upgrade | Wrong {project} / {agentName} path segment, missing api-version, or unsupported region | Verify with agent_get; ensure the project/agent path segments are correct and api-version=v1 is on the URL; confirm region per Hosted Agents region availability | | WS closes immediately after accept | Container handler raised inside the request | Check logs via azd ai agent monitor; typical causes are missing env vars or unreachable backend services | | Browser cannot connect directly | Browser WebSocket cannot set Authorization | Run a thin server-side proxy that injects the token before forwarding | | Frames received but no response | Wire-format mismatch | Confirm both ends use the same framing (binary vs text, codec, sample rate, schema). The platform does not validate or transcode frames | | Cold-start delay on first connect | Container initialising (VAD, model load, etc.) | Expected; subsequent connections to the same container are fast | | State lost across reconnect | Different agent_session_id used | Reuse the same agent_session_id query parameter to preserve agent-managed state |

Reference Samples

End-to-end working samples (server container + browser portal) live in the foundry-samples repo under:

samples/python/hosted-agents/bring-your-own/invocations_ws/

Each sub-folder shows a different media-path strategy (audio entirely over the WebSocket vs. WebSocket as signaling-only for an out-of-band media transport). Pick the one whose architecture matches your latency, NAT-traversal, and operational constraints.

Additional Resources