AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

deep-learning-book

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

Original source
alirezarezvani
License
MIT
Pinned revision
e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read alirezarezvani--deep-learning-book at revision e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 30

glossary.md

Glossary

Terms as this skill uses them, with the chapter that develops each. Definitions are written for lookup, not for teaching — read the chapter file for the reasoning.

  • Adam / AdamW (ch08, ch07) — adaptive optimizer combining momentum with per-parameter scaling. AdamW decouples weight decay from the gradient, which Adam+L2 does not do correctly.
  • Amortized inference (ch19, ch14) — training a network to predict a posterior's parameters instead of optimizing per example.
  • Ancestral sampling (ch16) — sampling a directed model by drawing each variable after its parents.
  • Attention (ch10) — a learned weighted read over a set of positions; introduced here as a fix for the seq2seq bottleneck, later the whole architecture.
  • Backpropagation (ch06) — reverse-mode automatic differentiation over the computation graph.
  • Batch normalization (ch08) — reparameterization normalizing layer inputs by batch statistics.
  • Bias–variance decomposition (ch05) — expected error as bias² + variance + irreducible noise.
  • Capacity, effective vs representational (ch05) — functions reachable by the optimizer vs functions expressible by the architecture.
  • Condition number (ch02, ch04) — ratio of largest to smallest singular value; how amplified small perturbations become.
  • Contrastive divergence (CD-k) (ch18) — biased approximation of the negative phase using a short chain started at the data.
  • Cross-entropy (ch03) — H(p,q) = H(p) + D_KL(p‖q); minimizing it over q minimizes KL.
  • Curse of dimensionality (ch05, ch01) — configurations grow exponentially with dimension, defeating local-interpolation methods.
  • Denoising autoencoder (DAE) (ch14) — reconstructs clean input from a corrupted copy; its learned map estimates the score.
  • Disentangled representation (ch15) — separate generative factors on separate directions.
  • Distributed representation (ch15) — n features describing exponentially many regions, versus one-hot's n.
  • d-separation (ch16) — reading conditional independence off a directed graph.
  • Double descent (ch05) — test error falling again past the interpolation threshold; post-dates the book and qualifies its U-curve.
  • ELBO (ch19) — evidence lower bound; log p(x) minus KL(q ‖ posterior).
  • Empirical risk minimization (ch08) — minimizing training loss as a surrogate for unmeasurable true risk.
  • Energy-based model (ch16) — p(x) ∝ exp(−E(x)).
  • Equivariance (ch09) — a shift in input produces the corresponding shift in output; convolution's defining property.
  • Exposure bias (ch10) — train/inference mismatch from teacher forcing.
  • Gradient clipping (ch08, ch10) — rescaling gradients above a norm threshold to survive cliffs.
  • Ill-conditioning (ch04, ch08) — wide Hessian eigenvalue spread; loss stalls while gradient norm stays large.
  • Importance sampling (ch17) — sampling from q and reweighting by p/q.
  • KL divergence (ch03) — asymmetric divergence between distributions; direction decides mode-covering vs mode-seeking.
  • Manifold hypothesis (ch05, ch14) — real data concentrates near a low-dimensional manifold.
  • Maximum likelihood estimation (MLE) (ch05) — parameters maximizing data probability; the justification for standard losses.
  • Mean field (ch19) — factorized variational family; buys tractability with an independence assumption.
  • Mixing time / burn-in (ch17) — steps an MCMC chain needs before its samples are usable.
  • Mode collapse (ch20) — a generator covering only part of the data distribution.
  • Momentum / Nesterov momentum (ch08) — velocity accumulation across gradient steps.
  • No Free Lunch theorem (ch05) — averaged over all distributions, all learners tie; priors carry all leverage.
  • Noise-contrastive estimation (NCE) (ch18) — density estimation recast as data-versus-noise classification.
  • Partition function (Z) (ch16, ch18) — the normalizer of an undirected model; the central intractability of Part III.
  • Persistent contrastive divergence (PCD) (ch18) — CD with a chain persisted across updates.
  • Pooling (ch09) — local aggregation giving approximate invariance to small translations.
  • Positive / negative phase (ch18) — the two terms of an undirected model's likelihood gradient.
  • Pseudolikelihood (ch18) — product of conditionals, in which Z cancels.
  • Receptive field (ch09) — the input region influencing one output unit.
  • Regularization (ch07, ch05) — any change intended to reduce generalization error but not training error.
  • Reparameterization trick (ch20) — expressing a sample as a differentiable function of noise, making the ELBO backprop-able.
  • Representation learning (ch15, ch01) — learning features rather than hand-designing them.
  • Saddle point (ch08) — critical point that is a minimum in some directions and a maximum in others; the dominant critical point in high dimensions.
  • Score matching (ch18) — matching ∇_x log p instead of p, so Z vanishes; the basis of diffusion models.
  • SGD, minibatch (ch08) — gradient descent on noisy gradients from data subsets.
  • Singular value decomposition (SVD) (ch02) — A = U D Vᵀ for any matrix.
  • Sparse coding (ch13) — heavy-tailed latent prior with inference by optimization.
  • Teacher forcing (ch10) — training an autoregressive model on ground-truth history.
  • Universal approximation theorem (ch06) — one wide hidden layer suffices in principle; silent on width, learnability and sample cost.
  • Vanishing / exploding gradients (ch10, ch08) — repeated Jacobian multiplication shrinking or growing gradients through depth or time.
  • Variational autoencoder (VAE) (ch20, ch19) — amortized variational inference plus the reparameterization trick.
  • Weight decay (ch07) — pulling parameters toward zero; equivalent to L2 for SGD, not for Adam.