SKILL.md · 48000
LL.md` inside the skill folder).
Source materials (in priority order):
- Official Microsoft Learn docs (via
microsoft-docsMCP) - SDK source code from the repository
- Existing reference files in the skill
Format:
# Acceptance Criteria: <skill-name>
**SDK**: `package-name`
**Repository**: https://github.com/Azure/azure-sdk-for-<language>
**Purpose**: Skill testing acceptance criteria
---
## 1. Correct Import Patterns
### 1.1 Client Imports
#### ✅ CORRECT: Main Client
\`\`\`python
from azure.ai.mymodule import MyClient
from azure.identity import DefaultAzureCredential
\`\`\`
#### ❌ INCORRECT: Wrong Module Path
\`\`\`python
from azure.ai.mymodule.models import MyClient # Wrong - Client is not in models
\`\`\`
## 2. Authentication Patterns
#### ✅ CORRECT: DefaultAzureCredential + context manager
\`\`\`python
credential = DefaultAzureCredential()
with MyClient(endpoint, credential) as client:
client.do_thing()
\`\`\`
#### ❌ INCORRECT: Hardcoded Credentials
\`\`\`python
client = MyClient(endpoint, api_key="hardcoded") # Security risk
\`\`\`
#### ❌ INCORRECT: Connection string / account key when Entra is supported
\`\`\`python
client = MyClient.from_connection_string(os.environ["CONNECTION_STRING"]) # Bypasses Entra audit/rotation
\`\`\`
#### ❌ INCORRECT: Bare client without context manager
\`\`\`python
client = MyClient(endpoint, credential) # Leaks HTTP transport on exception / interpreter exit
client.do_thing()
\`\`\`
Critical patterns to document:
- Import paths (these vary significantly between Azure SDKs)
- Authentication patterns
- Client initialization
- Async variants (
.aiomodules) - Common anti-patterns
6.2 Create Test Scenarios
Location: tests/scenarios/<skill-name>/scenarios.yaml
config:
model: gpt-4
max_tokens: 2000
temperature: 0.3
scenarios:
- name: basic_client_creation
prompt: |
Create a basic example using the Azure SDK.
Include proper authentication and client initialization.
expected_patterns:
- "DefaultAzureCredential"
- "MyClient"
- "with MyClient" # enforce context manager
forbidden_patterns:
- "api_key="
- "hardcoded"
- "from_connection_string" # prefer Entra over connection strings
tags:
- basic
- authentication
mock_response: |
import os
from azure.identity import DefaultAzureCredential
from azure.ai.mymodule import MyClient
credential = DefaultAzureCredential()
with MyClient(
endpoint=os.environ["AZURE_ENDPOINT"],
credential=credential,
) as client:
# ... rest of working example
pass
Scenario design principles:
- Each scenario tests ONE specific pattern or feature
expected_patterns— patterns that MUST appearforbidden_patterns— common mistakes that must NOT appearmock_response— complete, working code that passes all checkstags— for filtering (basic,async,streaming,tools)
6.3 Run Tests
cd tests
pnpm install
# Check skill is discovered
pnpm harness --list
# Run in mock mode (fast, deterministic)
pnpm harness <skill-name> --mock --verbose
# Run with Ralph Loop (iterative improvement)
pnpm harness <skill-name> --ralph --mock --max-iterations 5 --threshold 85
Success criteria:
- All scenarios pass (100% pass rate)
- No false positives (mock responses always pass)
- Patterns catch real mistakes
Step 7: Update Documentation
After creating the skill:
-
Update README.md — Add the skill to the appropriate language section in the Skill Catalog
- Update total skill count (line ~73:
> N skills in...) - Update Skill Explorer link count (line ~15:
Browse all N skills) - Update language count table (lines ~77-83)
- Update language section count (e.g.,
> N skills • suffix: -py) - Update category count (e.g.,
<summary><strong>Foundry & AI</strong> (N skills)</summary>) - Add skill row in alphabetical order within its category
- Update test coverage summary (line ~622:
**N skills with N test scenarios**) - Update test coverage table — update skill count, scenario count, and top skills for the language
- Update total skill count (line ~73:
-
Regenerate GitHub Pages data — Run the extraction script and rebuild the docs site from one scoped directory change
(cd docs-site && npx tsx scripts/extract-skills.ts && npm run build)This updates
docs-site/src/data/skills.jsonwhich feeds the Astro-based docs site, then rebuilds the site intodocs/, which is served by GitHub Pages. -
Verify AGENTS.md — Ensure the skill count is accurate
Step 8: Regenerate Existing Skills from Latest SDK Sources
Use this workflow when an existing skill has stale examples, outdated API signatures, or changed package guidance.
- Identify canonical source files first
For Azure SDK language skills, use official upstream source docs and examples as the source of truth:
- Go:
https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/<service>/<module>/README.md - Go examples:
https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/<service>/<module>/ - Rust:
https://github.com/Azure/azure-sdk-for-rust/tree/main/sdk/<service>/<crate>/README.md - Rust examples:
https://github.com/Azure/azure-sdk-for-rust/tree/main/sdk/<service>/<crate>/examples/ - .NET/Java/Python/TS/Go: use current Microsoft Learn package docs + official SDK repos
- Refresh skill content surgically
- Update code snippets to match current constructor/method signatures
- Keep crate/package names aligned with official publisher guidance
- Preserve skill structure/frontmatter unless intentionally changing behavior
- Update "Best Practices" and "Reference Links" when upstream recommendations change
- For Rust, if code uses
azure_coretypes/imports directly, ensureazure_coreis present inCargo.toml; if only service-crate re-exports are used, directazure_coredependency is optional
API Surface Parity Gate (required for every regenerated skill)
Use the language-specific authoritative source as the contract for every snippet in the regenerated skill:
- Python, .NET, Java, TypeScript, Go: Treat the current Microsoft Learn API reference as the contract.
- Rust: Treat the official SDK repository (
https://github.com/Azure/azure-sdk-for-rust) and crates.io documentation as the contract; Rust packages do not have Learn API-reference pages.
Before finalizing any regenerated skill:
- Identify each SDK type/method shown in snippets (clients, operation groups, model constructors, enum members, long-running methods like
begin_*). - Verify each symbol and signature against the authoritative source for that language/package (see above).
- If the authoritative source shows a different shape (for example nested
properties=...models, renamed methods,begin_*LRO methods), update the snippet to match. - Re-check imports so model/client modules match the authoritative source exactly.
- Do not keep compatibility shortcuts that contradict authoritative examples in primary snippets.
Scenario Coverage Gate (required for every regenerated skill, all languages)
Regeneration is not complete when snippets compile — it is complete when the skill demonstrates real usage breadth.
Before finalizing any regenerated skill:
- Identify hero scenarios from the current authoritative docs/samples for that SDK (Microsoft Learn where available; otherwise the upstream SDK repo and package documentation).
- Ensure each hero scenario is represented in the skill with copy-pastable snippets (or an explicit link to a bundled reference file when too large).
- Add/refresh test scenarios so hero flows are validated by harness patterns.
- Add at least one important non-hero scenario (for example: update/patch, delete/cleanup, export/import, advanced auth mode, paging/filtering, retries/error handling, or LRO monitoring) when supported by the SDK. For Python SDKs that support both sync and async clients, present both forms with equal priority; do not treat either as universally preferred.
- For Azure SDK skills, structure
references/as:references/capabilities.mdas a concise index that records each hero scenario and where it is covered (SKILL.mdor a bundled reference), plus links to deeper non-hero references, with no historical/migration narration.references/non-hero-scenarios.mdfor concrete non-hero examples that are intentionally kept out of the mainSKILL.md.- Additional
references/*.mdfiles for specialized deep-dives (operation groups, tools, evaluator matrices, etc.).
- If the SDK has broad operation-group coverage (common in management SDKs), include an operation-group table and explicitly call out which groups are covered in snippets vs. referenced only.
- Never claim "full API surface" unless the skill genuinely demonstrates all major operation groups; otherwise state that the skill is optimized for hero workflows plus selected secondary scenarios.
Regeneration Workflow Step 3: Validate Regenerated Skill Behavior
(cd tests && pnpm harness <skill-name> --mock --verbose)
If the skill has a Vally scenario, run that eval as well (locally or in CI) before finalizing.
Rust regeneration gate (required for Rust skills):
When regenerating any Rust skill, verify the generated ## Best Practices section contains these exact first two rules:
Use cargo add to manage dependencies, never edit Cargo.toml directlyAdd azure_core only when importing azure_core types directly
Use a content check before finalizing:
rg -n "Use `cargo add` to manage dependencies, never edit `Cargo.toml` directly|Add `azure_core` only when importing `azure_core` types directly" .github/plugins/azure-sdk-rust/skills/**/SKILL.md
The regeneration is not complete unless both lines are present in each affected Rust skill.
Regeneration Workflow Step 4: Regenerate Docs Artifacts After Refresh
(cd docs-site && npx tsx scripts/extract-skills.ts && npm run build)
Regeneration Workflow Step 5: Record What Changed
In the PR/commit notes, include:
- Which upstream docs/examples were used
- Which snippets/signatures were corrected
- Which tests/evals were run and their outcomes
Python plugin batch recipe: azure-sdk-python
Use this when the request is "regenerate all Python skills under azure-sdk-python."
- Scope the exact targets first
# Canonical source of truth for Python plugin skills
ls .github/plugins/azure-sdk-python/skills/*/SKILL.md
- Treat
.github/plugins/azure-sdk-python/skills/as canonical. - Keep
.github/skills/<name>links in sync after edits (symlink check/fix step below).
- For each skill, refresh from authoritative sources
- Always use
microsoft-docsMCP first for current Microsoft Learn API guidance. - Verify the installed package version with
pip show <package>, then inspect the installed package or official API reference to verify every symbol and signature used in snippets. - For Azure SDK skills, prefer package overview + official SDK repo examples.
- For non-Azure Python skills in this plugin (for example
fastapi-router-py,pydantic-models-py), keep language-specific best-practice variants and skip Azure-specific auth callouts when lifecycle/auth is not applicable.
- Apply Python enforcement rules consistently
- Keep the standard section order for Azure SDK Python skills.
- Ensure
## Authentication & Lifecyclestarts with the required callout block (verbatim) when applicable. - Ensure every client example uses
with/async withlifecycle patterns. - Ensure
## Best Practicesstarts with the two required user-facing rules (or the documented variant for async-only/provider-pattern skills). - Ensure each regenerated Azure SDK Python skill has
references/capabilities.md(index) andreferences/non-hero-scenarios.md(concrete non-hero examples). - Keep existing references/assets/scripts unless stale or incorrect.
- Validate all regenerated Python skills
# Fast frontmatter/structure validation for every Python skill
python .github/skills/skill-creator/scripts/quick_validate.py .github/plugins/azure-sdk-python/skills/<skill-name>
# Run Python skill harness in mock mode (all *-py scenarios)
(cd tests && pwsh ./run-harness-by-language.ps1 -Language py -Mock)
- Sync skill links and docs artifacts
# Ensure .github/skills links point at plugin canonical skills
python .github/scripts/sync_skill_links.py --plugin azure-sdk-python --check
python .github/scripts/sync_skill_links.py --plugin azure-sdk-python --apply
# Refresh docs site data after content changes
(cd docs-site && npx tsx scripts/extract-skills.ts && npm run build)
- Completion criteria for batch regeneration
- Every targeted
.github/plugins/azure-sdk-python/skills/*/SKILL.mdis updated or explicitly confirmed current. - Harness mock run for
-pyskills passes without regressions. - Skill links are in sync for
azure-sdk-python. - PR notes include upstream docs used, signature corrections, and validation outcomes.
Progressive Disclosure Patterns
Pattern 1: High-Level Guide with References
# SDK Name
## Quick Start
[Minimal example]
## Advanced Features
- **Streaming**: See [references/streaming.md](references/streaming.md)
- **Tools**: See [references/tools.md](references/tools.md)
Pattern 2: Language Variants
azure-service-skill/
├── SKILL.md (overview + language selection)
└── references/
├── python.md
├── dotnet.md
├── go.md
├── java.md
└── typescript.md
Pattern 3: Feature Organization
azure-ai-agents/
├── SKILL.md (core workflow)
└── references/
├── tools.md
├── streaming.md
├── async-patterns.md
└── error-handling.md
Design Pattern References
| Reference | Contents |
| ---------------------------------- | ------------------------------------ |
| references/workflows.md | Sequential and conditional workflows |
| references/output-patterns.md | Templates and examples |
| references/azure-sdk-patterns.md | Language-specific Azure SDK patterns |
Anti-Patterns
| Don't | Why |
| ------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------- |
| Create skill without SDK context | Users must provide package name/docs URL |
| Put "when to use" in body | Body loads AFTER triggering |
| Hardcode credentials | Security risk |
| Skip authentication section | Agents will improvise poorly |
| Use outdated SDK patterns | APIs change; search docs first |
| Include README.md | Agents don't need meta-docs |
| Deeply nest references | Keep one level deep |
| Skip acceptance criteria | Skills without tests can't be validated |
| Skip symlink categorization | Skills won't be discoverable by category |
| Use wrong import paths | Azure SDKs have specific module structures |
| Omit sync/async + context-manager bullets from Best Practices in Python skills | End users won't follow rules that aren't written down; examples alone aren't enough |
| Mix sync and async in the same Python example | Demonstrates the anti-pattern the skill is supposed to prevent |
| Ship regenerated skills with zero test scenarios | Hero workflows and regressions cannot be validated |
| Claim full API coverage from a single happy-path sample | Hides operation-group and non-hero gaps users need for production |
| Omit references/*.md coverage for non-hero capabilities | Forces advanced capabilities out of context and leaves API breadth undocumented |
Checklist
Before completing a skill:
Prerequisites:
- [ ] User provided SDK package name or documentation URL
- [ ] Verified SDK patterns via
microsoft-docsMCP - [ ] Verified every snippet's API surface against the current official language-specific API reference for that SDK (Microsoft Learn where available, otherwise the upstream SDK repo — see canonical sources above)
Skill Creation:
- [ ] Description includes what AND when (trigger phrases)
- [ ] SKILL.md under 500 lines
- [ ] Authentication follows language rules (
DefaultAzureCredentialfor Python/.NET/Java/TS/Go local dev;DeveloperToolsCredentiallocal dev +ManagedIdentityCredentialproduction for Rust) - [ ] Includes cleanup/delete in examples
- [ ] References organized by feature (
capabilities.mdindex + dedicated deep-dive files) - [ ] Hero scenarios from the current authoritative docs/samples for that SDK are explicitly covered in snippets and tests
- [ ] At least one high-value non-hero scenario is included when the SDK supports a distinct non-hero scenario (otherwise note that no distinct non-hero scenario applies)
- [ ] For Azure SDK skills,
references/capabilities.mdindexes hero/non-hero coverage and links to dedicated non-hero docs - [ ] For Azure SDK skills,
references/non-hero-scenarios.mdcontains concrete non-hero examples distinct from hero snippets - [ ] For broad SDKs (especially management SDKs), operation-group coverage is explicit (covered in snippets vs. reference-only)
- [ ] (Python skills only) Best Practices section contains the two user-facing rules (sync-or-async consistency + context managers for clients and async credentials), using the variant matched to the skill type
- [ ] For Rust skills:
## Best Practicesstarts with cargo dependency rule +azure_coredirect-import rule
Categorization:
- [ ] Skill created in
.github/skills/<skill-name>/ - [ ] Symlink created in
skills/<language>/<category>/<short-name> - [ ] Symlink points to
../../../.github/skills/<skill-name>
Testing:
- [ ]
tests/scenarios/<skill-name>/acceptance-criteria.mdcreated with correct/incorrect patterns - [ ]
tests/scenarios/<skill-name>/scenarios.yamlcreated - [ ] At least one hero scenario and one non-hero scenario are test-covered (when the SDK supports both)
- [ ] All scenarios pass (
pnpm harness <skill> --mock) - [ ] Import paths documented precisely
Documentation:
- [ ] README.md skill catalog updated
- [ ] Instructs to search
microsoft-docsMCP for current APIs