AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

deep-learning-book

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

Original source
alirezarezvani
License
MIT
Pinned revision
e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read alirezarezvani--deep-learning-book at revision e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 30

cheatsheet.md

Cheatsheet

Decision rules and thresholds. One line each; the chapter has the reasoning.

Loss selection (ch03, ch06)

| Target | Distribution | Output unit | Loss | |---|---|---|---| | Real, unbounded | Gaussian | linear | MSE | | Binary | Bernoulli | sigmoid | binary cross-entropy | | One of K | Categorical | softmax | cross-entropy | | Real, heavy-tailed | Laplace | linear | MAE | | Multimodal real | Mixture | mixture density | NLL of the mixture |

Never pair sigmoid output with MSE — the gradient vanishes where the model is most wrong.

Underfit vs overfit (ch05, ch11)

| Train error | Val error | Verdict | First action | |---|---|---|---| | High | High | Underfit / optimization | Check LR, init, capacity — not more data | | Low | High | Overfit | More data → augmentation → regularization | | Low | Low | Working | Stop; check the metric is the right one | | ~0 on 20 examples: fails | — | Bug | Fix the pipeline before anything else |

Past the interpolation threshold, prefer more data / longer training / more regularization over shrinking the model (double descent, ch05).

Optimizer defaults (ch08)

| Situation | Reach for | |---|---| | Any starting point | AdamW, LR ~1e-3 (small nets) / ~1e-4 (large), warmup + cosine decay | | Recurrence in the graph | Add gradient clipping, always | | Tiny batches | Group/layer norm, not batch norm | | Loss flat, grad norm large | Ill-conditioning — momentum, adaptive, or normalization | | Loss NaN | Numerics first: log(0), div by ~0, exp overflow, exploding grad |

Regularization ladder (ch07)

More real data → label-preserving augmentation → early stopping → decoupled weight decay → label smoothing → dropout → smaller model. One at a time; measure the gap after each.

Architecture prior (ch09, ch10)

| Data | Prior that fits | Note | |---|---|---| | Grid, translation-invariant statistics | Convolution | Cheapest when data is limited | | Sequence, long-range dependence | Attention/transformer | The 2016 RNN advice is superseded | | Sequence, very long context | State-space / linear recurrence | Post-2016; ch10's gradient analysis applies | | Arbitrary tabular columns | Neither | Locality prior is false |

Generative model taxonomy (ch20)

| Family | Handles Z by | Characteristic failure | |---|---|---| | Autoregressive | Chain rule — no Z | Slow sequential sampling | | VAE | Bounding likelihood (ELBO) | Blurry samples | | GAN | Avoiding likelihood | Mode collapse, unstable training | | Diffusion / score-based | Learning ∇ log p | Many sampling steps (mitigable) | | Boltzmann machines | Sampling the negative phase | Historical; do not start here |

Numerical hygiene (ch04)

Log-space for probability products · stabilized/fused softmax and cross-entropy · clip gradients with recurrence · bf16 over fp16 when range matters · check condition number before blaming LR.

Study-order rule (ch01–ch20)

Ch 5 and Ch 11 are the highest-value chapters for a practitioner and can be read early. Ch 2–4 are prerequisites for Ch 8 specifically. Ch 13→14→19→20 is the only strict chain in Part III. Ch 18 is worth reading even if you never train an undirected model — score matching became diffusion.