AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

finetuning

Fine-tune models on Microsoft Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset preparation, training job submission, deployment, and evaluation. USE FOR: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, training job, large file upload, calibrate grader, deploy fine-tuned model, evaluate fine-tuned model. DO NOT USE FOR: general model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).

Original source
microsoft
License
MIT
Pinned revision
3f5b461b2314e01c400564051e92469114c5b68f

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read microsoft--finetuning at revision 3f5b461b2314e01c400564051e92469114c5b68f, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 18

references/agentic-rft.md

Agentic RFT — Tool Calling

Train reasoning models (o4-mini) for agentic scenarios where the model invokes external tools during chain-of-thought reasoning.

⚠️ Access required: Agentic RFT with tool calling and GPT-5 RFT are behind feature flags. You must request access through the Microsoft Foundry portal or your Microsoft account team. o4-mini RFT without tools is generally available.

Tool Definition Format

tools = [
    {
        "name": "search",
        "server_url": "https://your-function-app.azurewebsites.net/api/tools",
        "headers": {
            "Authorization": "Bearer <your-key>"
        }
    },
    {
        "name": "get_by_id",
        "server_url": "https://your-function-app.azurewebsites.net/api/tools",
        "headers": {
            "Authorization": "Bearer <your-key>"
        }
    }
]

Submitting an Agentic RFT Job

job = client.fine_tuning.jobs.create(
    model="o4-mini-2025-04-16",
    training_file=train.id,
    validation_file=valid.id,
    method={
        "type": "reinforcement",
        "reinforcement": {
            "grader": grader,
            "tools": tools,
            "max_episode_steps": 10,
            "hyperparameters": {
                "eval_interval": 5,
                "eval_samples": 10,
                "compute_multiplier": 1.5,
                "reasoning_effort": "medium"
            }
        }
    }
)

Tool Response Format

Your tool endpoint must return:

{
    "type": "function_call_output",
    "call_id": "call_12345xyz",
    "output": "The result of the tool call...",
    "id": "fc_12345xyz"
}

Tool Endpoint Requirements

| Constraint | Limit | |-----------|-------| | Recommended throughput | 50 QPS | | Max input payload | 1 MB | | Max return payload | 1 MB (413 error if exceeded) | | Timeout | 10 minutes | | Parallel calls | Supported — handle race conditions | | Retry on 5xx | 3 attempts, then rollout discarded | | On 4xx | Error serialized and shown to model |

Infrastructure: Use Always On, sufficient compute (S2+), multiple instances. Under-provisioned endpoints can cause jobs to hang during post-training eval.

RFT Hyperparameters

| Parameter | Description | Recommended Start | |-----------|-------------|-------------------| | reasoning_effort | "low", "medium", "high" | "medium" | | compute_multiplier | Scales rollouts per step | 1.5 | | learning_rate_multiplier | Scales the learning rate | 1.0 | | n_epochs | Data passes | 2–3 | | eval_interval | Eval every N steps | 5 | | eval_samples | Validation examples per eval | 10 | | max_episode_steps | Max tool calls + reasoning steps per rollout | 5–10 |

Notes: Higher LR increases output verbosity without improving accuracy. Compute multiplier 1.5 balances rollout quality and training time. Platform may early-stop before all epochs.

When to Use Agentic RFT

  • Model needs to decide when to call tools (not just follow instructions)
  • Task involves multi-step reasoning with external data lookups
  • Model needs to learn tool selection — choosing the right tool for the job
  • Standard RFT (without tools) can't capture the agentic behavior