AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

deep-learning-book

Use when studying or applying Goodfellow, Bengio and Courville's Deep Learning, or asking if its advice still holds. Indexes all 20 chapters with a 2016-to-2026 delta, plans reading paths with reading_path_planner.py and diagnoses training failures with training_diagnostics.py.

Original source
alirezarezvani
License
MIT
Pinned revision
e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read alirezarezvani--deep-learning-book at revision e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 30

chapters/ch01-introduction.md

Chapter 1: Introduction

Source chapter (free, official): https://www.deeplearningbook.org/contents/intro.html

Core Idea

Deep learning is representation learning stacked: instead of hand-designing the features a model consumes, you let the model learn a hierarchy of representations, each layer expressed in terms of the one below it. Depth is the mechanism that makes composition cheap.

Key Concepts

  • Representation learning: learning the features, not just the mapping from features to output.
  • Depth as composition: a deep model reuses sub-computations, so a function needing exponentially many units at depth 1 may need polynomially many at depth k.
  • The curse of dimensionality: as input dimension grows, the number of distinguishable configurations grows exponentially, so local-interpolation methods run out of neighbours.
  • Prior of compositionality: deep learning bets that the data was generated by a composition of simple factors. This is a prior, not a theorem — where the bet is wrong, depth does not help.
  • AI winters and rebrands: cybernetics → connectionism → deep learning; the ideas outlived three funding cycles because the constraint was data and compute, not concept.

Mental Models

  • Think of each layer as re-coordinatizing the input so the next layer's job gets easier — not as "extracting meaning."
  • Prefer depth over width when the target function plausibly factors; prefer width when it does not. Depth is a structural prior, and priors have to be earned.
  • Treat "the model learns features automatically" as shifting where the human judgment goes (architecture, objective, data curation), not eliminating it.

Anti-patterns

  • Reading this chapter as history and skipping it: the three growth curves it draws (dataset size, model size, benchmark accuracy) are the argument for why anything else works.
  • Assuming depth is always better: without the compositional structure, extra depth buys optimization difficulty and nothing else.

What changed after 2016

The chapter's central claim — capability tracks dataset and model scale — held, and was made quantitative by neural scaling laws (Hestness et al. 2017; Kaplan et al. 2020; Hoffmann et al. 2022 "Chinchilla"), which the 2016 text predates. Its examples of "what deep learning can do" are now badly dated; its explanation of why depth helps is not. Confidence: high — the scaling-law literature is public and replicated.

Key Takeaways

  1. Before adding depth, state the compositional structure you believe the data has.
  2. Use the curse of dimensionality as the test for whether a nearest-neighbour-style baseline is even viable on your input.
  3. Read the growth curves as a resource argument: most 2016-era "deep learning doesn't work here" results were data or compute verdicts, not architecture verdicts.

Connects To

  • Ch 5: the formal learning-theory machinery behind the claims made informally here.
  • Ch 15: representation learning, promised here and delivered there.
  • references/book_to_2026_delta.md: the scaling-law layer this chapter now needs.