AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

deep-learning-book

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

Original source
alirezarezvani
License
MIT
Pinned revision
e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read alirezarezvani--deep-learning-book at revision e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 30

patterns.md

Patterns

Recurring techniques from the book, stated as instruments with their trade-offs. Each names the chapter that develops it.

Derive the loss from the output distribution (ch03, ch06)

Choose what p(y|x) is, then take its negative log. Gaussian → MSE, Bernoulli → binary cross-entropy, categorical → cross-entropy, Laplace → MAE, mixture → mixture density loss. Trade-off: none — this is strictly better than picking a loss by habit. Failure mode it prevents: MSE on bounded, count-valued or heavily skewed targets.

Work in log-space (ch04)

Sums of logs instead of products of probabilities; stabilized softmax and log_softmax rather than hand-rolled exp/normalize. Trade-off: slightly more care at implementation time. Prevents: silent underflow, NaN losses, and inf logits.

Train-error-first triage (ch11, ch05)

Read training error before deciding anything. High training error → capacity or optimization is the bottleneck; more data will not help. Low training error with high validation error → data or regularization. Trade-off: requires honest, non-leaky splits. Prevents: the most expensive common mistake, which is collecting data to fix underfitting.

Overfit a tiny subset as a smoke test (ch11)

Take 10–50 examples and drive training loss to ~0. If you cannot, you have a bug — not a hard problem. Trade-off: minutes. Prevents: weeks of tuning around a broken data pipeline, a wrong loss reduction, or a detached gradient.

Random search over grid search (ch11)

With more than about two hyperparameters, random search finds better configurations for the same budget because it does not spend trials re-testing unimportant dimensions. Trade-off: results are less tidy to tabulate. Prevents: exponential waste in dimensions that do not matter.

Regularize in cost order (ch07)

More real data → augmentation with label-preserving transforms → early stopping → weight decay → dropout → architecture change. Add one at a time and measure the train/val gap after each. Trade-off: slower than stacking everything. Prevents: an unattributable result you cannot tune.

Parameter sharing over parameter penalties (ch07, ch09)

When a real invariance exists, encode it structurally (convolution, weight tying) rather than penalizing a free parameter into behaving. Trade-off: the prior is hard — wrong invariance means a wrong model, with no way for data to override it. Prevents: paying for capacity you then have to regularize away.

Diagnose by gradient behaviour (ch08, ch04)

Gradient norm exploding → clip. Norm large, loss flat → ill-conditioning; use momentum, an adaptive optimizer, or normalization. Norm near zero with high loss → saturation or dead units; check initialization and activations. Loss NaN → numerics before modelling. Trade-off: requires instrumentation. Prevents: changing the architecture to fix an optimizer problem.

Tune learning rate and schedule before optimizer family (ch08)

The schedule usually dominates the choice among SGD/Adam variants. Trade-off: none. Prevents: optimizer-shopping while the real problem is a step size two orders of magnitude off.

Name what prevents the identity map (ch14)

Every autoencoder needs an answer: bottleneck, sparsity penalty, input corruption, or Jacobian contraction. Trade-off: each constraint defines a different notion of useful. Prevents: an overcomplete autoencoder that learns a copy and reports a low loss.

Escape the partition function deliberately (ch18, ch16)

Four routes: sample the negative phase (CD/PCD), sidestep Z algebraically (pseudolikelihood, score matching), learn around it (NCE), or estimate it for evaluation only (AIS). Trade-off: each optimizes a different objective, so they are not interchangeable. Prevents: discovering mid-project that your undirected model cannot be trained or compared.

Pick your generative failure mode (ch20, ch03)

Likelihood-based objectives cover modes and blur. Adversarial objectives sharpen and drop modes. This follows from KL direction, so it is a design choice, not bad luck. Trade-off: you must decide which error your application tolerates. Prevents: treating blurriness as a bug to be tuned away.

Evaluate representations by transfer, not reconstruction (ch15, ch14)

Linear probes and downstream task performance. Trade-off: needs a downstream task. Prevents: optimizing reconstruction error into a code that memorized the input.

Report Monte Carlo estimates with error bars (ch17)

Standard error and a mixing diagnostic, always. Trade-off: more reporting. Prevents: a confidently unimodal answer from a chain that never left its starting mode.