AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

microsoft-foundry

Build, deploy, evaluate, optimize, fine-tune, and manage Microsoft Foundry agents, models, and resources end to end. USE FOR: foundry, azd ai agent, azd provision/deploy, hosted agent scaffold/develop/run/deploy/troubleshoot, prompt agent, managed harness agent, Copilot harness, voice agent, create/update/invoke/evaluate agent, add tool to agent, agent insights, batch/continuous eval, continuous monitoring, agent CI/CD, optimize/improve prompt, optimize agent instructions, Agent Optimizer, dataset curation from traces, deploy model, model fine-tuning (SFT/DPO/RFT), Foundry project, RBAC, role assignment, permissions, quota, capacity, region, deployment failure, AI Services, create Foundry resource, knowledge index, customize deployment, onboard, availability, training-data, grader, distillation, large file upload. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure-prepare).

Original source
microsoft
License
MIT
Pinned revision
3f5b461b2314e01c400564051e92469114c5b68f

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read microsoft--microsoft-foundry at revision 3f5b461b2314e01c400564051e92469114c5b68f, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 151

quota/references/capacity-planning.md

Capacity Planning Guide

Comprehensive guide for planning Microsoft Foundry capacity, including cost analysis, model selection, and workload calculations.

Table of Contents: Cost Comparison: TPM vs PTU · Production Workload Examples · Model Selection and Deployment Type Guidance

Cost Comparison: TPM vs PTU

Official Pricing Sources:

TPM (Standard) Pricing:

  • Pay-per-token for input/output
  • No upfront commitment
  • Rates: See Azure OpenAI Pricing
    • GPT-4o: ~$0.0025-$0.01/1K tokens
    • GPT-4 Turbo: ~$0.01-$0.03/1K
    • GPT-3.5 Turbo: ~$0.0005-$0.0015/1K
  • Best for: Variable workloads, unpredictable traffic

PTU (Provisioned) Pricing:

  • Hourly billing: $/PTU/hr × PTUs × 730 hrs/month
  • Monthly commitment with Reservations discounts
  • Rates: See PTU Billing Guide
  • Use PTU calculator to determine requirements (Microsoft Foundry → Operate → Quota → Provisioned Throughput Unit tab)
  • Best for: High-volume (>1M tokens/day), predictable traffic, guaranteed throughput

Cost Decision Framework (Analytical Guidance):

Step 1: Calculate monthly TPM cost
  Monthly TPM cost = (Daily tokens × 30 days × $price per 1K tokens) / 1000

Step 2: Calculate monthly PTU cost
  Monthly PTU cost = Required PTUs × 730 hours/month × $PTU-hour rate
  (Get Required PTUs from Microsoft Foundry portal: Microsoft Foundry → Operate → Quota → Provisioned Throughput Unit tab)

Step 3: Compare
  Use PTU when: Monthly PTU cost < (Monthly TPM cost × 0.7)
  (Use 70% threshold to account for commitment risk)

Example Calculation (Analytical):

Scenario: 1M requests/day, average 1,000 tokens per request

  • Daily tokens: 1,000,000 × 1,000 = 1B tokens/day
  • TPM Cost (using GPT-4o at $0.005/1K avg): (1B × 30 × $0.005) / 1000 = ~$150,000/month
  • PTU Cost (estimated 100 PTU at ~$5/PTU-hour): 100 PTU × 730 hours × $5 = ~$365,000/month
  • Decision: Use TPM (significantly lower cost for this workload)

Important: Always use the official Azure Pricing Calculator and Microsoft Foundry portal PTU calculator (Microsoft Foundry → Operate → Quota → Provisioned Throughput Unit tab) for exact pricing by model, region, and workload. Prices vary by region and are subject to change.


Production Workload Examples

To estimate quota requirements, use real-world production scenarios with capacity calculations for gpt-4, version 0613 (from Azure Foundry Portal calculator):

| Workload Type | Calls/Min | Prompt Tokens | Response Tokens | Cache Hit % | Total Tokens/Min | PTU Required | TPM Equivalent | |---------------|-----------|---------------|-----------------|-------------|------------------|--------------|----------------| | RAG Chat | 10 | 3,500 | 300 | 20% | 38,000 | 100 | 38K TPM | | Basic Chat | 10 | 500 | 100 | 20% | 6,000 | 100 | 6K TPM | | Summarization | 10 | 5,000 | 300 | 20% | 53,000 | 100 | 53K TPM | | Classification | 10 | 3,800 | 10 | 20% | 38,100 | 100 | 38K TPM |

How to Estimate Your Production Quota Requirements:

To calculate your quota needs for production deployments, follow these steps:

  1. Determine your peak calls per minute: Monitor or estimate maximum concurrent requests
  2. Measure token usage: Average prompt size + response size
  3. Account for cache hits: Prompt caching can reduce effective token count by 20-50%
  4. Calculate total tokens/min: (Calls/min × (Prompt tokens + Response tokens)) × (1 - Cache %)
  5. Choose deployment type:
    • TPM (Standard): Allocate 1.5-2× your calculated tokens/min for headroom
    • PTU (Provisioned): Use Microsoft Foundry portal PTU calculator for exact PTU count (Microsoft Foundry → Operate → Quota → Provisioned Throughput Unit tab)

Example Calculation (RAG Chat Production):

  • Peak: 10 calls/min
  • Prompt: 3,500 tokens (context + question)
  • Response: 300 tokens (answer)
  • Cache: 20% hit rate (reduces prompt tokens by 20%)
  • Total TPM needed: (10 × (3,500 × 0.8 + 300)) = 31,000 TPM
  • With 50% headroom: 46,500 TPM → Round to 50K TPM deployment

PTU Recommendation: For the combined workload (40 calls/min, 135K tokens/min total), use 200 PTU (from calculator above).


Model Selection and Deployment Type Guidance

Official Documentation:

Model Characteristics (from official Azure OpenAI documentation):

| Model | Key Characteristics | Best For | |-------|---------------------|----------| | GPT-4o | Matches GPT-4 Turbo performance in English text/coding, superior in non-English and vision tasks. Cheaper and faster than GPT-4 Turbo. | Multimodal tasks, cost-effective general purpose, high-volume production workloads | | GPT-4 Turbo | Superior reasoning capabilities, larger context window (128K tokens) | Complex reasoning tasks, long-context analysis | | GPT-3.5 Turbo | Most cost-effective, optimized for chat and completions, fast response time | Simple tasks, customer service, high-volume low-cost scenarios | | GPT-4o mini | Fastest response time, low latency | Latency-sensitive applications requiring immediate responses | | text-embedding-3-large | Purpose-built for vector embeddings | RAG applications, semantic search, document similarity |

Deployment Type Selection (from official deployment types guide):

| Traffic Pattern | Recommended Deployment Type | Reason | |-----------------|---------------------------|---------| | Variable, bursty traffic | Standard or Global Standard (pay-per-token) | No commitment, pay only for usage | | Consistent high volume | Provisioned types (PTU) | Reserved capacity, predictable costs | | Large batch jobs (non-time-sensitive) | Global Batch or DataZone Batch | 50% cost savings vs Standard | | Low latency variance required | Provisioned types | Guaranteed throughput, no rate limits | | No regional restrictions | Global Standard or Global Provisioned | Access to best available capacity |

Capacity Planning Approach (from PTU onboarding guide):

To calculate and estimate your capacity requirements:

  1. Calculate your TPM requirements: Determine required tokens per minute based on your expected workload
  2. Use the built-in capacity planner: Available in Microsoft Foundry portal (Microsoft Foundry → Operate → Quota → Provisioned Throughput Unit tab)
  3. Input your metrics: Enter input TPM and output TPM based on your workload characteristics
  4. Get PTU recommendation: The calculator provides PTU allocation recommendation
  5. Compare costs: Evaluate Standard (TPM) vs Provisioned (PTU) using the official pricing calculator

Note: Microsoft does not publish specific "X requests/day = Y TPM" recommendations as capacity requirements vary significantly based on prompt size, response length, cache hit rates, and model choice. Use the built-in capacity planner with your actual workload characteristics.