AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

deep-learning-book

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

Original source
alirezarezvani
License
MIT
Pinned revision
e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read alirezarezvani--deep-learning-book at revision e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 30

chapters/ch12-applications.md

Chapter 12: Applications

Source chapter (free, official): https://www.deeplearningbook.org/contents/applications.html

Core Idea

How deep learning was actually deployed circa 2016: large-scale implementation (GPUs, distributed training, model compression), then computer vision, speech recognition, and NLP. This is the most time-dated chapter in the book — read the systems half, discount the state-of-the-art half.

Frameworks Introduced

  • Large-scale implementation: GPU data parallelism, model parallelism, asynchronous SGD (Hogwild-style), parameter servers.
  • Model compression: distillation, quantization, pruning — for inference cost.
  • Dynamic structure / conditional computation: cascades and gating so that not every input pays the full cost.
  • Preprocessing in vision: contrast normalization, whitening, dataset augmentation.
  • Speech: the acoustic-model pipeline and its shift from GMM-HMM to deep networks.
  • NLP: n-gram models, word embeddings and the curse of dimensionality over vocabularies, hierarchical softmax and sampling-based approximations for large output vocabularies, neural machine translation.
  • Recommender systems and the exploration/exploitation problem.
  • Knowledge representation and relational reasoning.

Mental Models

  • Separate capability claims (perishable) from systems constraints (durable). Memory bandwidth, batch efficiency, and inference cost still shape architecture choices.
  • Read the large-vocabulary softmax section as an instance of the general pattern: when the normalizing sum is expensive, approximate it — the same problem Ch 18 attacks head-on.
  • Treat conditional computation as the ancestor of Mixture-of-Experts routing.

Anti-patterns

  • Citing this chapter's benchmark numbers or SOTA claims — they are a decade old.
  • Copying its NLP pipeline: subword tokenization (BPE/SentencePiece) and pretrained transformers replaced nearly all of it.

What changed after 2016

Almost everything at the application layer. Speech moved to end-to-end CTC/attention models and then to large self-supervised encoders. NLP moved to pretrained transformers (BERT 2018, GPT family), with subword tokenization and full-softmax over ~30k–200k subwords making hierarchical softmax largely unnecessary. Vision moved to self-supervised pretraining and ViTs. Distillation, quantization and pruning grew into a mature inference-optimization discipline. Conditional computation matured into sparse MoE. Confidence: high.

Key Takeaways

  1. Read this chapter for the systems reasoning and the approximation techniques, not for what is state of the art.
  2. When your output vocabulary or normalizing constant is huge, recognize it as the recurring partition-function problem.
  3. Plan inference cost as an architecture constraint from the start.

Connects To

  • Ch 18: the partition-function problem in its general form.
  • Ch 9 / Ch 10: the architectures being applied here.
  • references/book_to_2026_delta.md: what replaced each application pipeline.