references/training-curves.md
Training Curve Analysis
SFT Metrics
| Column | What it means |
|--------|---------------|
| train_loss | Loss on training batch (should decrease) |
| train_mean_token_accuracy | Token-level accuracy on training data |
| valid_loss | Loss on validation set (primary metric) |
| valid_mean_token_accuracy | Token-level accuracy on validation data |
| full_valid_loss | Full-pass validation loss (more accurate, less frequent) |
| full_valid_mean_token_accuracy | Full-pass token accuracy |
Overfitting Detection
Overfitting ratio at each checkpoint: valid_loss / train_loss
| Ratio | Interpretation | |-------|---------------| | < 1.2 | Healthy — generalizes well | | 1.2–1.5 | Mild overfitting — acceptable for small datasets | | 1.5–2.0 | Moderate — consider reducing epochs | | > 2.0 | Severe — deploy an earlier checkpoint |
val_losses = [cp.metrics.valid_loss for cp in checkpoints if cp.metrics.valid_loss]
best_val = min(val_losses)
final_val = val_losses[-1]
if final_val > best_val * 1.2:
print(f"⚠️ OVERFIT: Best={best_val:.4f}, final={final_val:.4f}")
Best Checkpoint Selection (SFT)
checkpoints = client.fine_tuning.jobs.checkpoints.list(job_id)
best_cp = min(checkpoints.data, key=lambda cp: cp.metrics.valid_loss or float('inf'))
print(f"Best: step {best_cp.step_number}, valid_loss={best_cp.metrics.valid_loss:.4f}, "
f"model={best_cp.fine_tuned_model_checkpoint}")
Diagnosis Table
| Observation | Diagnosis | Action | |-------------|-----------|--------| | Train loss barely decreases | LR too low or noisy data | Increase LR or clean data | | Train loss crashes to ~0 | LR too high or easy data | Decrease LR or add harder examples | | Valid loss rises after epoch 2 | Overfitting | Deploy epoch-2 checkpoint | | Valid loss plateaus after epoch 1 | Learned quickly | Try epoch=1 or lower LR | | Valid loss oscillates | Small batch or inconsistent data | Increase batch size or audit data | | Both losses stay high | Task too hard | Larger model or simplify task | | Large train-valid gap from start | Insufficient/mismatched data | Add diverse training data |
RFT Metrics
| Column | What it means |
|--------|---------------|
| train_mean_reward | Average reward across rollouts (primary — should increase) |
| full_valid_mean_reward | Validation reward (overfitting check) |
| completion_tokens_mean | Average response length per rollout |
| reasoning_tokens_mean | Average reasoning tokens (o-series models) |
| mean_unresponsive_rewards | Rollouts with no scoreable output |
| train_sample_parse_error_count | Grader couldn't parse output |
| train_other_error_count | Grader logic bugs — should be 0 |
RFT Reward Curve Patterns
- Reward flat at ~0: Grader broken or threshold too strict
- Reward always negative: pass_threshold too high
- Reward immediately high + flat: Threshold too lenient
- Train-valid reward gap > 0.10: Possible reward hacking
Token Growth
- Moderate (tokens double): Normal — model becoming more thorough
- Excessive (3x+): Grader may incentivize verbosity — check scoring dimensions
- When comparing checkpoints, equal accuracy at fewer tokens is strictly better
Parse Errors vs Logic Errors
sample_parse_error_count: Often high in agentic RFT (mid-reasoning captures). Training still works if reward is climbing.other_error_count: Bugs in grader logic. Fix before continuing.
RFT Checkpoint Selection
checkpoints = client.fine_tuning.jobs.checkpoints.list(job_id)
for cp in checkpoints:
m = cp.metrics
tr = f"{m.train_mean_reward:.3f}" if m.train_mean_reward is not None else "n/a"
vr = f"{m.full_valid_mean_reward:.3f}" if m.full_valid_mean_reward is not None else "n/a"
ct = f"{m.completion_tokens_mean:.0f}" if m.completion_tokens_mean is not None else "n/a"
print(f"Step {cp.step_number}: train_reward={tr}, valid_reward={vr}, tokens={ct}")
Don't rely solely on valid_reward for RFT — deploy 2–3 candidates (peak reward, final, mid-training) and evaluate with your real task harness including tool execution.