AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

finetuning

Fine-tune models on Microsoft Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset preparation, training job submission, deployment, and evaluation. USE FOR: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, training job, large file upload, calibrate grader, deploy fine-tuned model, evaluate fine-tuned model. DO NOT USE FOR: general model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).

Original source
microsoft
License
MIT
Pinned revision
3f5b461b2314e01c400564051e92469114c5b68f

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read microsoft--finetuning at revision 3f5b461b2314e01c400564051e92469114c5b68f, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 18

references/training-curves.md

Training Curve Analysis

SFT Metrics

| Column | What it means | |--------|---------------| | train_loss | Loss on training batch (should decrease) | | train_mean_token_accuracy | Token-level accuracy on training data | | valid_loss | Loss on validation set (primary metric) | | valid_mean_token_accuracy | Token-level accuracy on validation data | | full_valid_loss | Full-pass validation loss (more accurate, less frequent) | | full_valid_mean_token_accuracy | Full-pass token accuracy |

Overfitting Detection

Overfitting ratio at each checkpoint: valid_loss / train_loss

| Ratio | Interpretation | |-------|---------------| | < 1.2 | Healthy — generalizes well | | 1.2–1.5 | Mild overfitting — acceptable for small datasets | | 1.5–2.0 | Moderate — consider reducing epochs | | > 2.0 | Severe — deploy an earlier checkpoint |

val_losses = [cp.metrics.valid_loss for cp in checkpoints if cp.metrics.valid_loss]
best_val = min(val_losses)
final_val = val_losses[-1]
if final_val > best_val * 1.2:
    print(f"⚠️ OVERFIT: Best={best_val:.4f}, final={final_val:.4f}")

Best Checkpoint Selection (SFT)

checkpoints = client.fine_tuning.jobs.checkpoints.list(job_id)
best_cp = min(checkpoints.data, key=lambda cp: cp.metrics.valid_loss or float('inf'))
print(f"Best: step {best_cp.step_number}, valid_loss={best_cp.metrics.valid_loss:.4f}, "
      f"model={best_cp.fine_tuned_model_checkpoint}")

Diagnosis Table

| Observation | Diagnosis | Action | |-------------|-----------|--------| | Train loss barely decreases | LR too low or noisy data | Increase LR or clean data | | Train loss crashes to ~0 | LR too high or easy data | Decrease LR or add harder examples | | Valid loss rises after epoch 2 | Overfitting | Deploy epoch-2 checkpoint | | Valid loss plateaus after epoch 1 | Learned quickly | Try epoch=1 or lower LR | | Valid loss oscillates | Small batch or inconsistent data | Increase batch size or audit data | | Both losses stay high | Task too hard | Larger model or simplify task | | Large train-valid gap from start | Insufficient/mismatched data | Add diverse training data |

RFT Metrics

| Column | What it means | |--------|---------------| | train_mean_reward | Average reward across rollouts (primary — should increase) | | full_valid_mean_reward | Validation reward (overfitting check) | | completion_tokens_mean | Average response length per rollout | | reasoning_tokens_mean | Average reasoning tokens (o-series models) | | mean_unresponsive_rewards | Rollouts with no scoreable output | | train_sample_parse_error_count | Grader couldn't parse output | | train_other_error_count | Grader logic bugs — should be 0 |

RFT Reward Curve Patterns

  • Reward flat at ~0: Grader broken or threshold too strict
  • Reward always negative: pass_threshold too high
  • Reward immediately high + flat: Threshold too lenient
  • Train-valid reward gap > 0.10: Possible reward hacking

Token Growth

  • Moderate (tokens double): Normal — model becoming more thorough
  • Excessive (3x+): Grader may incentivize verbosity — check scoring dimensions
  • When comparing checkpoints, equal accuracy at fewer tokens is strictly better

Parse Errors vs Logic Errors

  • sample_parse_error_count: Often high in agentic RFT (mid-reasoning captures). Training still works if reward is climbing.
  • other_error_count: Bugs in grader logic. Fix before continuing.

RFT Checkpoint Selection

checkpoints = client.fine_tuning.jobs.checkpoints.list(job_id)
for cp in checkpoints:
    m = cp.metrics
    tr = f"{m.train_mean_reward:.3f}" if m.train_mean_reward is not None else "n/a"
    vr = f"{m.full_valid_mean_reward:.3f}" if m.full_valid_mean_reward is not None else "n/a"
    ct = f"{m.completion_tokens_mean:.0f}" if m.completion_tokens_mean is not None else "n/a"
    print(f"Step {cp.step_number}: train_reward={tr}, valid_reward={vr}, tokens={ct}")

Don't rely solely on valid_reward for RFT — deploy 2–3 candidates (peak reward, final, mid-training) and evaluate with your real task harness including tool execution.