AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

finetuning

Fine-tune models on Microsoft Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset preparation, training job submission, deployment, and evaluation. USE FOR: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, training job, large file upload, calibrate grader, deploy fine-tuned model, evaluate fine-tuned model. DO NOT USE FOR: general model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).

Original source
microsoft
License
MIT
Pinned revision
3f5b461b2314e01c400564051e92469114c5b68f

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read microsoft--finetuning at revision 3f5b461b2314e01c400564051e92469114c5b68f, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 18

references/hyperparameters.md

Hyperparameter Guide

SFT / DPO Core Parameters

| Parameter | What it controls | Default | Typical range | |-----------|-----------------|---------|---------------| | Epochs | Passes through data | 2 | 1–5 | | Learning rate multiplier | Weight change aggressiveness | 1.0 | 0.1–2.0 | | Batch size | Examples per gradient step | Model-dependent | 4–32 |

Dataset Size vs Epochs

| Dataset size | Recommended epochs | |-------------|-------------------| | < 100 examples | 3–5 | | 100–500 examples | 2–3 | | 500–2,000 examples | 1–2 | | > 2,000 examples | 1 |

Learning Rate Guidelines

  • Higher LR (1.5–2.0): Large/diverse datasets, task very different from pre-training
  • Lower LR (0.1–0.5): Small datasets (<200), refining not overwriting base behavior
  • For 1,000+ examples, LR 0.2–0.5 often beats default 1.0

DPO-Specific Parameters

  • beta (default 0.1): Alignment strength. Lower = more conservative.
  • l2_multiplier (default 0.1): Regularization to prevent drift from base model.

HP Sweep Strategy

| Run | Epochs | LR | Why | |-----|--------|----|-----| | 1 | 2 | 1.0 | Baseline | | 2 | 2 | 0.5 | Conservative | | 3 | 2 | 1.5 | Aggressive | | 4 | 3 | 1.0 | More training | | 5 | 1 | 1.0 | Minimal intervention |

Checkpoint Trick

When overfitting (val loss rises after epoch 2): deploy the epoch-2 checkpoint directly instead of retraining. Azure saves checkpoints at each epoch boundary.

checkpoints = client.fine_tuning.jobs.checkpoints.list(job_id)
for cp in checkpoints.data:
    print(f"Step {cp.step_number}: val_loss={cp.metrics.valid_loss}")

Model-Specific Recommendations

| Model | Recommended Start | Notes | |-------|------------------|-------| | gpt-4.1-mini | 2ep, lr=0.5–1.0 | Very capable base; small nudges work | | gpt-4.1-nano | 2–3ep, lr=1.0–1.5 | Smaller capacity, needs more epochs | | gpt-oss-20b | 2ep, lr=0.2–0.5 | Lower LR critical; deployment may need capacity=100 | | o4-mini (RFT) | Grader quality > HPs | Focus on grader, not HP sweep |

OSS Model Parameters

All OSS models require trainingType: "globalStandard" in the API request.

| Model | Recommended Start | Best Found | Notes | |-------|------------------|------------|-------| | Ministral-3B | 5ep, lr=1.0 | 10ep, lr=0.5 | Small model, slow convergence | | gpt-oss-20b | 2ep, lr=0.3 | 2ep, lr=0.3 | lr=1.0 overfits quickly | | Llama-3.3-70B | 3ep, lr=0.3 | 5ep, lr=0.5 | lr=2.0 causes catastrophic degradation | | Qwen-3-32B | 3ep, lr=0.3 | 3ep, lr=0.3 | Most fragile — more data can hurt |

Key patterns: OSS models need 2–5× more epochs than nano. Lower LR (0.3–0.5) is safer. More data doesn't always help.

RFT Hyperparameters

| Parameter | Description | Recommended Start | |-----------|-------------|-------------------| | reasoning_effort | "low", "medium", "high" | "medium" | | compute_multiplier | Scales rollouts per step | 1.5 | | learning_rate_multiplier | Scales LR | 1.0 | | n_epochs | Data passes | 2–3 | | eval_interval | Eval every N steps | 5 | | eval_samples | Validation examples per eval | 10 | | max_episode_steps | Max tool calls + reasoning steps | 5–10 |