{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  },
  "lecture_migration": {
   "project_id": "c0b185e7-d7e3-48b6-a605-8f1acdfe243c",
   "notebook_id": "bfca6645b58549e096e6fd8774de288f",
   "destination": "L2",
   "cloud_run": false
  },
  "title": "LAB 04 · 卷积怎样变成图像分类器"
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "f08d9207e34a4694b2fb4b1008f4314a",
     "type": "text-cell-h1",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-1"
   },
   "source": "LAB 04 · 卷积怎样变成图像分类器",
   "id": "f08d9207e34a4694b2fb4b1008f4314a"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "7a556d94381c43dbb38ce01bfacb1644",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-2"
   },
   "source": [
    "[返回第二讲 · 步长与边界的按需实现](https://codingai-lec02.pages.dev/#/stride-changes-sampling/0) · [第二讲导读与学习材料](https://codingai-lec02.pages.dev/course/guide.html) · [问卷作业 ↗](https://tally.so/r/VLroVv)（右键在新标签页打开）。\n",
    "\n",
    "**按问题查实现：本册不要求从头运行。** 主讲中的每个算子，在这里保留完整索引、梯度与源码依据；课堂只选当前卡住的一项。\n",
    "\n",
    "| 想核对什么 | 找到的标题 | 最小运行顺序 |\n",
    "| --- | --- | --- |\n",
    "| 窗口起点与输出大小 | “Stride 与 padding 改变了哪些条件” | 先运行 `import inspect / torch` 的准备块，再运行该节 `nn.Conv2d` 代码 |\n",
    "| 输入通道求和、输出通道堆叠 | “一个输出通道会读取全部输入通道” | 准备块 → “一个输出数”定义 `corr2d` → 本节代码 |\n",
    "| 核的共享梯度 | “共享参数的梯度…” | 准备块 → 本节 `gx,gk,gy` 代码；若看 `unfold` 再接下一节 |\n",
    "| 不装PyTorch也能检验一个更新 | 文末“独立核对：先预测，再运行” | 该标准库代码块可以单独运行 |\n",
    "\n",
    "**可选短练：** 在 stride 那个代码块中，只将 `stride=2` 改成1，保持输入32×32、核3×3、padding=1与通道数不变。先预测输出尺寸和参数数目，再运行核对：空间输出会变，权重数量不变。不要同时改 padding。\n",
    "\n",
    "连续解释读 [第二讲 CLS03](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/c0b185e7-d7e3-48b6-a605-8f1acdfe243c/notebook/15edefc493064d3382fb418f52d8fabb?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=15edefc493064d3382fb418f52d8fabb&utm_term=get_notebook)；完整短练用 [第二讲 LAB01](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/c0b185e7-d7e3-48b6-a605-8f1acdfe243c/notebook/0b7ff2d0e61347cd917939e62deb701c?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=0b7ff2d0e61347cd917939e62deb701c&utm_term=get_notebook)，真实图像训练结果用 [第二讲 LAB02](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/c0b185e7-d7e3-48b6-a605-8f1acdfe243c/notebook/03c7a68e5b484caea0cdfc24a5e229a0?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=03c7a68e5b484caea0cdfc24a5e229a0&utm_term=get_notebook)。\n",
    "\n",
    "后半册 A–F 是推导与源码查阅区。文本里的源码节选未必包含完整上下文，不作为可单独运行的程序；可运行块的依赖按上表选择。"
   ],
   "id": "7a556d94381c43dbb38ce01bfacb1644"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "0819de95cc694c71bc722dc97b9fb719",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-3"
   },
   "source": "## 共同起点：怎样把识别写成学习问题",
   "id": "0819de95cc694c71bc722dc97b9fb719"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "402ba35966434dcc90e7c18df5e99f2e",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-4"
   },
   "source": [
    "**表示与学习信号的起点。** 本册默认你能区分输入、表示、读出与评价。连续例子在 [CLS01](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/c0b185e7-d7e3-48b6-a605-8f1acdfe243c/notebook/a09cce95fed04f51850a46e7afd37c2e?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=a09cce95fed04f51850a46e7afd37c2e&utm_term=get_notebook)；这里查实现时始终追问：当前算子保留了哪些输入区别，哪些参数能被最终误差改变？\n",
    "\n",
    "[原背景来源：CS231n Lecture 2](https://cs231n.stanford.edu/slides/2026/lecture_2.pdf)"
   ],
   "id": "402ba35966434dcc90e7c18df5e99f2e"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "ac5fd44cede14a97869c80599cb48774",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-5"
   },
   "source": "**kNN：最简单的学习系统也要先定义“相似”。** 最近邻保存训练图像与标签，为新图像寻找距离最近的例子；kNN 把范围扩大到 k 个邻居，再投票。增大 k 通常让局部决策更平滑，但也可能把不同类别混在一起。\n\n这个起点暴露了表示的问题：把图像平移几个像素，逐像素的 L1/L2 距离可能变大，但类别并未改变；两幅背景相似的不同类别图像又可能距离很近。k 不能弥补所有距离定义的缺陷。可以先问“需要保留什么相似性”，再问“如何更快搜索”。在已学到的合适表示上，近邻仍有用途，不应把它当成被淘汰的算法。[CS231n 分类讲义](https://cs231n.github.io/classification/)",
   "id": "ac5fd44cede14a97869c80599cb48774"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "ac9842768b4b44189eb9a4054e435ed0",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-6"
   },
   "source": "**线性分类器的三种视角。** 对展平的图像向量 $x\\in\\mathbb R^D$，令 $s=Wx+b\\in\\mathbb R^C$。代数上，每一行权重把所有输入加权成一个类别分数；视觉上，一行权重可以重新排成一个类别模板；几何上，类别 $a,b$ 的分界满足 $(w_a-w_b)^Tx+(b_a-b_b)=0$。\n\n这三个视角描述同一个函数。预测时使用已学参数，不再逐一比较全部训练图像；代价是模型只能在当前坐标里给出这类边界。同一类别有多种姿态和布局时，单一模板容易把不同模式叠在一起。展平本身是可逆索引变换，并未删除像素；真正的限制来自模型怎样组织和共享权重。[CS231n 线性分类讲义](https://cs231n.github.io/linear-classify/)",
   "id": "ac9842768b4b44189eb9a4054e435ed0"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "485a2f11e0974c39b5bd7d77a390947e",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-7"
   },
   "source": "## 从类别分数到可学习的目标",
   "id": "485a2f11e0974c39b5bd7d77a390947e"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "dd1eb80b525141e78665bdb5f933c690",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-8"
   },
   "source": [
    "**目标接口查阅。** Softmax 和交叉熵的连续推导及精确数值在 [CLS01](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/c0b185e7-d7e3-48b6-a605-8f1acdfe243c/notebook/a09cce95fed04f51850a46e7afd37c2e?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=a09cce95fed04f51850a46e7afd37c2e&utm_term=get_notebook)，缩放与熵反例在 [LAB06](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/c0b185e7-d7e3-48b6-a605-8f1acdfe243c/notebook/8ed9e8ca4b444a07921ce1df42c6db01?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=8ed9e8ca4b444a07921ce1df42c6db01&utm_term=get_notebook)。本册在“分类的损失怎样训练前面的卷积”保留 $\\partial\\ell/\\partial s=p-\\mathrm{onehot}(y)$ 与 autograd 核对；共享梯度中的上游 δ 正是由这条链继续传回。\n",
    "\n",
    "[原来源](https://cs231n.github.io/linear-classify/)"
   ],
   "id": "dd1eb80b525141e78665bdb5f933c690"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "ffec8bd94b5b4391b6c6461ebb9bc4a5",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-9"
   },
   "source": "**正则化：训练误差之外，我们还偏好什么解？** 有限例子不能唯一确定在所有新输入上应怎样预测。目标可写成 $J(\\theta)=\\frac1N\\sum_i\\ell_i(\\theta)+\\frac\\lambda2\\|W\\|_F^2$；L2 项对指定权重的大幅增长施加代价。正则化不是故意让模型学不好，而是在多个拟合方案间加入偏好。\n\n数据增强提供另一种约束：当我们平移、裁切或改变颜色而保留标签时，就在声明这些变化应当不影响类别。这个声明可能不适合所有任务。因而要同时看训练表现和独立验证表现，并让增强操作服从任务语义；本课程 LAB 02 正是用结构 × 平移增强的对照检验这一点。[CS231n Lecture 3](https://cs231n.stanford.edu/slides/2026/lecture_3.pdf)",
   "id": "ffec8bd94b5b4391b6c6461ebb9bc4a5"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "a8308a4e65f94397a3f8ccfd74323f40",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-10"
   },
   "source": "**SGD：用当前小批量提供一次便宜的方向估计。** 对每次采样的 minibatch $B_t$ 计算平均损失的梯度，加上正则项梯度，得到 $g_t$；再更新 $\\theta_{t+1}=\\theta_t-\\eta_tg_t$。梯度回答参数怎样微小变化会影响当前目标，学习率 $\\eta_t$ 决定这次走多远。\n\n全量梯度每步需要看完整数据；小批量降低单步成本，并在通常的均匀采样条件下估计训练目标的平均梯度。代价是方向有噪声，单步损失未必下降。不要把“梯度有噪声”直接等同于坏事，也不要据一次下降断言泛化改善。后面的卷积模型仍使用这套目标与更新机制，只是参数化不同。[CS231n 优化讲义](https://cs231n.github.io/optimization-1/)",
   "id": "a8308a4e65f94397a3f8ccfd74323f40"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "3d8cc77f4c9a4e149ad471bcec162462",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-11"
   },
   "source": "## 训练策略改变路径，不替代表示选择",
   "id": "3d8cc77f4c9a4e149ad471bcec162462"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "7a8be06c56d74037b40b87b862b063c7",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-12"
   },
   "source": "**Momentum：让连续多步的一致方向累积起来。** 一种常见约定是 $v_t=\\beta v_{t-1}+g_t$，然后 $\\theta_{t+1}=\\theta_t-\\eta_tv_t$。在狭长的损失谷里，普通 SGD 可能反复横跳；如果横向梯度方向交替而纵向方向持续，动量能抵消一部分横向摆动并积累纵向推进。\n\n这是一种关于优化几何的机制说明，不是“任何问题都更快”的保证。也有实现把 $(1-\\beta)$ 乘在新梯度上，采用严格指数平均；两种公式的数值尺度与学习率约定不能随意混用。主讲只需要理解历史方向如何影响下一步，具体状态变量可交给实现检查。[CS231n Lecture 3](https://cs231n.stanford.edu/slides/2026/lecture_3.pdf)",
   "id": "7a8be06c56d74037b40b87b862b063c7"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "6ef3cfc17f0c443ab7b08fcabf0a88bb",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-13"
   },
   "source": "**AdaGrad / Adam：不同坐标能否采用不同尺度？** 如果一些参数的梯度长期很大、另一些很稀疏，统一步长可能不合适。AdaGrad 累加各坐标的平方梯度，用它缩放更新；长期累积也可能使有效步长持续缩小。Adam 用一阶与二阶矩的指数平均记住方向和尺度，并对早期估计作偏差校正。\n\n可以用三个问题区分训练策略：动量在记忆哪个方向？自适应方法怎样缩放不同坐标？全局学习率现在允许多大的移动？它们改善的是搜索路径与数值条件，不能让原本线性的模型突然表达非线性边界。无需把完整 Adam 实现背成主讲知识点。[CS231n 训练讲义](https://cs231n.github.io/neural-networks-3/)",
   "id": "6ef3cfc17f0c443ab7b08fcabf0a88bb"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "980068b9ebcd4d2d804f830f9c094b59",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-14"
   },
   "source": "**学习率与数据划分决定怎样判断训练。** 较大的步长有利于早期快速移动，但也可能越过局部有效区域；训练后期减小步长可让更新更细。阶梯衰减、余弦安排和 warmup 都是控制这一过程的方法，应根据训练曲线和验证结果判断，而不是把一种日程当成普遍最优。\n\n训练集用于拟合参数；验证集用于选择模型、超参数和 checkpoint；测试集用于评价选定方案。若反复按测试结果改设计，测试集就参与了选择。比较两个模型还需说明数据、优化预算、算量和随机种子；相同参数量不等于相同计算量，单次结果也不能证明普遍排序。[CS231n Lecture 2](https://cs231n.stanford.edu/slides/2026/lecture_2.pdf)、[训练讲义](https://cs231n.github.io/neural-networks-3/)",
   "id": "980068b9ebcd4d2d804f830f9c094b59"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "dddd6facf1624e45bd66661430f700d1",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-15"
   },
   "source": "## MLP 与反向传播：让表示也参与学习",
   "id": "dddd6facf1624e45bd66661430f700d1"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "00feb9ef121a46c3893e8b826ab359b9",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-16"
   },
   "source": "**先改变坐标，线性判别就可能变容易。** 在 $s=W\\phi(x)+b$ 中，$\\phi$ 可以是手工构造的图像特征，也可以被学习。两层 MLP 把它写成 $h=\\sigma(W_1x+b_1)$，再由 $s=W_2h+b_2$ 读出类别。第一层产生共享的中间响应，第二层将它们组合为不同类别证据。\n\n隐藏单元无需一开始就被命名为“眼睛”“轮子”；它们首先是为了目标而调整的数值表示。把“学一个分类边界”扩展为“同时学适合分类的坐标”，是从线性模型进入表示学习的关键。传统视觉怎样设计 $\\phi$，可以接着读 LAB 05 开头的历史部分。[CS231n 神经网络讲义](https://cs231n.github.io/neural-networks-1/)",
   "id": "00feb9ef121a46c3893e8b826ab359b9"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "04865f5f9eb44dcfab7ca44ef7f3e189",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-17"
   },
   "source": "**非线性让叠层不再只是重新写一个矩阵。** 如果中间没有激活函数，$W_2(W_1x+b_1)+b_2=(W_2W_1)x+(W_2b_1+b_2)$，仍是一个仿射映射。ReLU 等非线性使不同输入进入不同激活区域，从而改变可以表示的函数形式。\n\n深度让中间表示能够复用、组合并逐步改变，但“更深”并不自动等于“更好”。函数能否表示、梯度能否有效训练、数据是否足够，是三个不同问题。后面的 VGG、BN 和 ResNet 将分别在这些限制上提出设计，而不是简单增加层数。[CS231n Lecture 4](https://cs231n.stanford.edu/slides/2026/lecture_4.pdf)",
   "id": "04865f5f9eb44dcfab7ca44ef7f3e189"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "d39b2a373f814a8c9d2e69d6ed33158d",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-18"
   },
   "source": "**反向传播把最终目标连接到没有单独标签的隐藏层。** 前向计算记录输入、隐藏响应、分数和损失的依赖关系；反向按相反顺序应用链式法则，把上游梯度与本步的局部导数组合，得到每个参数对同一个最终损失的梯度。一个变量被多个分支使用时，各条路径贡献相加。\n\nBP 计算梯度，SGD / Adam 根据梯度更新参数，这两步应分开理解。自动求导不是为隐藏层猜一个新标签，也不是对每个参数反复试探。卷积核在多个位置被调用，反向会汇总这些调用对同一套参数的贡献；后文的共享梯度实验展示的就是这套机制。现在已具备完整训练闭环，接下来才讨论图像空间结构怎样约束模型。[CS231n BP 讲义](https://cs231n.github.io/optimization-2/)",
   "id": "d39b2a373f814a8c9d2e69d6ed33158d"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "7f914f54007c441c8c4c9f212eac8fe3",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-19"
   },
   "source": "## 空间关系如何变成模型约束",
   "id": "7f914f54007c441c8c4c9f212eac8fe3"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "00c958136d0e4179b6aad2ba6a96583a",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-20"
   },
   "source": "展平图像是一种可逆的索引变换，并没有自动丢掉像素信息。问题是普通全连接层允许每个输入位置与每个输出位置使用独立权重，没有显式编码“相邻像素组成局部结构”“相同局部结构可能出现在不同位置”。\n\n先考虑单通道输入和带二维位置索引的输出。一般线性层可以写成\n$$Y_{i,j}=b_{i,j}+\\sum_{k,l}W_{i,j,k,l}X_{k,l}.$$\n令 $k=i+a,l=j+b$，得到\n$$Y_{i,j}=b_{i,j}+\\sum_{a,b}V_{i,j,a,b}X_{i+a,j+b}.$$\n这一步只是重索引，参数没有减少。若要求同一种相对位置关系在各处遵循相同规则，令 $V_{i,j,a,b}=K_{a,b}$，偏置也不依赖位置；再约束远处偏移的权重为零，就得到局部共享算子。两个约束分别是**位置共享**与**局部支撑**，并非对任何任务都无代价成立。\n\n自然图像中的局部纹理常能跨位置复用；绝对解剖方位、传感器位置或地理坐标也可能具有预测信息。是否引入坐标或放松共享，取决于任务。[D2L 7.1](https://d2l.ai/chapter_convolutional-neural-networks/why-conv.html)",
   "id": "00c958136d0e4179b6aad2ba6a96583a"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "b8244e5fde8649a59825f850ac7a1245",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-21"
   },
   "source": "![局部连接与同位置规则复用的概念图](https://codingai-lec02.pages.dev/art/locality-and-sharing.webp)\n\n*概念图（ImageGen）：同一套局部规则在不同空间位置复用，固定的是连接与共享方式，系数仍可学习。 此图用于解释结构，不是模型实测输出。*",
   "id": "b8244e5fde8649a59825f850ac7a1245"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "ad5d862692db4300bb08efd7490f5a32",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-22"
   },
   "source": "import inspect\nimport torch\nfrom torch import nn\nfrom torch.nn import functional as F\ntorch.manual_seed(20260921)\ntorch.set_num_threads(2)\nprint(\"PyTorch\", torch.__version__)",
   "id": "ad5d862692db4300bb08efd7490f5a32",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "072a909878d040f3a3063ac1b255ef6e",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-23"
   },
   "source": "## 一个输出数怎样产生",
   "id": "072a909878d040f3a3063ac1b255ef6e"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "7641a83b67b442a5beb193d428c6328c",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-24"
   },
   "source": "令 $X=\\begin{bmatrix}0&1&2\\\\3&4&5\\\\6&7&8\\end{bmatrix}$，$K=\\begin{bmatrix}0&1\\\\2&3\\end{bmatrix}$。左上窗口得到 $0\\times0+1\\times1+3\\times2+4\\times3=19$。向右移一格，输入窗口改变，核不变；完整输出为 $\\begin{bmatrix}19&25\\\\37&43\\end{bmatrix}$。\n\n下面直接按定义实现，而不是先调用黑盒。每个输出是局部输入与同一核的内积。`torch.stack`将标量组成张量，同时保留计算图。[D2L 7.2](https://d2l.ai/chapter_convolutional-neural-networks/conv-layer.html)",
   "id": "7641a83b67b442a5beb193d428c6328c"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "ae9dc598ac4544cea244f35719c4ebfe",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-25"
   },
   "source": "def corr2d(x, k):\n    kh, kw = k.shape\n    return torch.stack([\n        torch.stack([(x[i:i+kh, j:j+kw] * k).sum()\n                     for j in range(x.shape[1]-kw+1)])\n        for i in range(x.shape[0]-kh+1)])\nx = torch.arange(9., dtype=torch.float64).reshape(3,3)\nk = torch.arange(4., dtype=torch.float64).reshape(2,2)\nmanual = corr2d(x,k)\nnative = F.conv2d(x[None,None], k[None,None])[0,0]\nprint(manual)\ntorch.testing.assert_close(manual,native)",
   "id": "ae9dc598ac4544cea244f35719c4ebfe",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "827e31da1eff453db50ee47ffa1cbeec",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-26"
   },
   "source": "## 移动的是窗口，不是参数",
   "id": "827e31da1eff453db50ee47ffa1cbeec"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "42c73feac66a49e2b73c7cbf12ae602d",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-27"
   },
   "source": "设 $T_\\Delta$ 表示平移。等变要求 $f(T_\\Delta X)=T_\\Delta f(X)$：特征图随对象移动；不变要求分类输出 $g(T_\\Delta X)=g(X)$。卷积首先对应前一个关系，如何聚合成类别输出是另一个设计问题。\n\n在无限网格或周期边界、stride=1条件下，平移不改变每次读取的相对偏移，因此可以直接在求和式中换元证明等变。有限图像的补零、裁切及stride会改变边界或采样格点，需分别检查。stride=2通常与2的整倍数平移相容，并不产生“半个输出像素”。\n\n共享还改变了学习：一个核在很多位置使用，这些位置的训练信号汇入同一个参数。但重叠patch高度相关，不能把它们称为很多独立样本。",
   "id": "42c73feac66a49e2b73c7cbf12ae602d"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "7144100d20c94ae4aaf16b466215d2d3",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-28"
   },
   "source": "## 一个输出通道会读取全部输入通道",
   "id": "7144100d20c94ae4aaf16b466215d2d3"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "5704bb59bb834befbd696f156b98408c",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-29"
   },
   "source": "标准多通道卷积的权重排列为 $K\\in\\mathbb R^{C_{out}\\times C_{in}\\times k_h\\times k_w}$。一个输出通道读取所有输入通道，先分别做空间乘加，再在输入通道上求和：\n$$Y_{n,o,i,j}=b_o+\\sum_{c,a,b}K_{o,c,a,b}X_{n,c,i+a,j+b}.$$\n这里暂取valid、stride=1、groups=1。输入通道上的**sum**将多种贡献合成一个响应；输出通道上的**stack**创建多个响应。二者不能互换。深层通道是共同优化的表示，不应要求每个通道都有唯一、固定的人类名称。[D2L 7.4](https://d2l.ai/chapter_convolutional-neural-networks/channels.html)",
   "id": "5704bb59bb834befbd696f156b98408c"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "634334f566854ab3b7055ecb2c00973b",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-30"
   },
   "source": "![输入通道与多个局部响应的概念图](https://codingai-lec02.pages.dev/art/channels-and-features.webp)\n\n*概念图（ImageGen）：空间位置与特征通道是不同维度；一个输出通道一般汇聚全部输入通道。精确求和见相邻公式。 此图用于解释结构，不是模型实测输出。*",
   "id": "634334f566854ab3b7055ecb2c00973b"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "77b2724131eb4f5283caf2a7f79e5e0c",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-31"
   },
   "source": "def multi_in(x,k):\n    assert x.shape[0] == k.shape[0]\n    return torch.stack([corr2d(xc,kc) for xc,kc in zip(x,k)]).sum(0)\ndef multi_out(x,k):\n    return torch.stack([multi_in(x,ko) for ko in k])\nxm=torch.randn(2,5,6,dtype=torch.float64)\nkm=torch.randn(3,2,3,2,dtype=torch.float64)\nym=multi_out(xm,km)\nprint(\"Cin=2, Cout=3:\",ym.shape)\ntorch.testing.assert_close(ym,F.conv2d(xm[None],km)[0])",
   "id": "77b2724131eb4f5283caf2a7f79e5e0c",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "fdbab7539027437aa041fec3bf3de145",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-32"
   },
   "source": "## Shape、参数、计算量分别在问什么",
   "id": "fdbab7539027437aa041fec3bf3de145"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "c921b9a813c04f1ab6ca76cf9a04840f",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-33"
   },
   "source": "输入 $32\\times32\\times3$，使用16个 $3\\times3$ filter，stride=1、padding=1且带bias：输出是 $32\\times32\\times16$；权重432个、bias16个，总参数448。单样本卷积MAC为 $32\\times32\\times16\\times27=442368$，这里不计bias与激活。\n\n空间尺寸扩大到64×64，参数不变而MAC约四倍。通道数同时翻倍，权重和MAC大致四倍；激活内存取决于batch、输出位置、通道和dtype。参数存储、前向计算与训练需保存的中间激活是不同负担。\n\n一次乘法及累加记为1 MAC；若两种算术分别计数，则约2 FLOPs。真实运行时间还受内存、算子融合与硬件影响。PyTorch输入默认NCHW，图示常写HWC，换表示时先核对维度。",
   "id": "c921b9a813c04f1ab6ca76cf9a04840f"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "8b1198a8fcee4fe0bf587734edc090a0",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-34"
   },
   "source": "## Stride 与 padding 改变了哪些条件",
   "id": "8b1198a8fcee4fe0bf587734edc090a0"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "dec8f47d0ae04634993e0263fc801b20",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-35"
   },
   "source": "按PyTorch每侧padding $p$、dilation $d$，单个方向的输出是\n$$H_{out}=\\left\\lfloor\\frac{H+2p-d(k-1)-1}{s}\\right\\rfloor+1.$$\n分子表示有效输入中可以容纳核的剩余长度，除以stride再向下取整表示完整可放置的移动次数。D2L部分公式用总padding，不可直接把该符号代入API的每侧padding。\n\npadding是一种边界假设。零、反射、周期边界提供不同的外围信息；stride则选择采样相位。改变二者不只是改变shape，还改变哪些数据进入计算。",
   "id": "dec8f47d0ae04634993e0263fc801b20"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "d23390a588944c42b83128c2dc693eaf",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-36"
   },
   "source": "conv=nn.Conv2d(3,16,3,stride=2,padding=1)\nprint(\"input/output:\",(1,3,32,32),tuple(conv(torch.zeros(1,3,32,32)).shape))\nprint(\"weight:\",tuple(conv.weight.shape),\"parameters:\",sum(p.numel() for p in conv.parameters()))\n# 实际读取当前安装版本的Python层；不是底层数值kernel。\nprint(inspect.getsource(nn.Conv2d.forward))\nprint(inspect.getsource(nn.Conv2d._conv_forward))",
   "id": "d23390a588944c42b83128c2dc693eaf",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "1f0a1ea6e5804815b4b2f468ad244052",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-37"
   },
   "source": "本地已执行的真实源码节选（PyTorch 2.14.0+cpu）；上方 `inspect` 可以在你的运行环境重新读取对应版本。这里展示 Python 调用与模式分支，不是底层 CUDA/CPU 数值实现。\n\n```python\ndef forward(self, input: Tensor) -> Tensor:\n        return self._conv_forward(input, self.weight, self.bias)\n\n    def _conv_forward(self, input: Tensor, weight: Tensor, bias: Tensor | None):\n        if self.padding_mode != \"zeros\":\n            return F.conv2d(\n                F.pad(\n                    input, self._reversed_padding_repeated_twice, mode=self.padding_mode\n                ),\n                weight,\n                bias,\n                self.stride,\n                _pair(0),\n                self.dilation,\n                self.groups,\n            )\n\n        return F.conv2d(\n            input, weight, bias, self.stride, self.padding, self.dilation, self.groups\n        )\n```",
   "id": "1f0a1ea6e5804815b4b2f468ad244052"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "568b15127e524a07add728ca537b2fde",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-38"
   },
   "source": "## 核的数值是谁决定的",
   "id": "568b15127e524a07add728ca537b2fde"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "336c3b8915e34bc192103d45549defdc",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-39"
   },
   "source": "D2L先用已知边缘核生成目标响应，再从随机参数恢复该算子。这使“学到了什么”有可核对答案。图像分类不同：监督只告诉最终类别，各层核通过总损失共同调整，没有逐层规定边缘、眼睛或物体。\n\n`nn.Conv2d`保存weight/bias等状态；`F.conv2d`接收输入与权重完成运算；`backward()`计算梯度；优化器根据梯度更新参数。把普通Tensor放进Python对象不等于注册可训练参数，`nn.Parameter`与Module注册机制让参数能被`parameters()`和`state_dict()`找到。\n\n**LAB01**保留手算与算子学习，并展开多通道与梯度对照。下面用最小模块观察参数身份。",
   "id": "336c3b8915e34bc192103d45549defdc"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "e4f536148010440ea0da6dfa151e8f5b",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-40"
   },
   "source": "class CorrLayer(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.weight=nn.Parameter(torch.randn(2,2,dtype=torch.float64))\n        self.bias=nn.Parameter(torch.zeros((),dtype=torch.float64))\n    def forward(self,x):\n        return corr2d(x,self.weight)+self.bias\nlayer=CorrLayer()\nprint(\"parameters:\",[(n,tuple(p.shape)) for n,p in layer.named_parameters()])\nloss=layer(x).square().mean()\nloss.backward()\nprint(\"kernel gradient:\",layer.weight.grad)",
   "id": "e4f536148010440ea0da6dfa151e8f5b",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "22b635986dd4423285b60f5d59860d57",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-41"
   },
   "source": "## 共享参数的梯度：把每次使用的贡献相加",
   "id": "22b635986dd4423285b60f5d59860d57"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "651666fada1948648345a3ed89db314e",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-42"
   },
   "source": "记 $\\delta_{n,o,i,j}=\\partial L/\\partial Y_{n,o,i,j}$。对核的一个元素，链式法则给出\n$$\\frac{\\partial L}{\\partial K_{o,c,a,b}}=\\sum_{n,i,j}\\delta_{n,o,i,j}X_{n,c,i+a,j+b}.$$\n它在每个batch、每个位置参与了前向，所以这些路径的导数必须累加。loss采用sum还是mean会通过delta改变梯度尺度。下面给定任意上游delta，以内积构造标量loss，逐位置累计梯度，再与autograd比较。这直接验证公式，不借分类准确率推断梯度。",
   "id": "651666fada1948648345a3ed89db314e"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "7e9478e020ad4ae39ed878c9353fd4be",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-43"
   },
   "source": "gx=torch.randn(2,2,5,6,dtype=torch.float64)\ngk=torch.randn(3,2,3,2,dtype=torch.float64,requires_grad=True)\ngy=F.conv2d(gx,gk)\ndelta=torch.randn_like(gy)\n(gy*delta).sum().backward()\nmanual_grad=torch.zeros_like(gk)\nfor n in range(2):\n    for o in range(3):\n        for i in range(gy.shape[2]):\n            for j in range(gy.shape[3]):\n                manual_grad[o]+=delta[n,o,i,j]*gx[n,:,i:i+3,j:j+2]\nprint(\"maximum gradient difference:\",(manual_grad-gk.grad).abs().max().item())\ntorch.testing.assert_close(manual_grad,gk.grad)",
   "id": "7e9478e020ad4ae39ed878c9353fd4be",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "10cf012d42d7411a9a761628a4c241df",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-44"
   },
   "source": "## 可选深读：滑窗矩阵与算子实现",
   "id": "10cf012d42d7411a9a761628a4c241df"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "7352309fba8b4fdf8280d30c9b780051",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-45"
   },
   "source": "把每个patch排列为矩阵的一列，记 $P\\in\\mathbb R^{C_{in}k_hk_w\\times H_{out}W_{out}}$，则前向可以写成 $Y_{flat}=K_{flat}P$。核梯度为 $\\delta_{flat}P^\\top$，再对batch求和。这个表示把卷积接回线性代数。\n\n`unfold`适合解释和验证，但显式patch展开会重复存储重叠像素。实际后端可选择其他算法；读到Python wrapper调用`F.conv2d`，还没有读到底层CUDA/CPU计算实现。",
   "id": "7352309fba8b4fdf8280d30c9b780051"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "988d8124d4d3487d95e4c82290ebdc3e",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-46"
   },
   "source": "patches=F.unfold(gx,kernel_size=(3,2))\nby_matmul=torch.einsum(\"ok,nkl->nol\",gk.reshape(3,-1),patches)\ntorch.testing.assert_close(by_matmul.reshape_as(gy),gy)\nprint(\"input elements:\",gx.numel(),\"unfold elements:\",patches.numel())",
   "id": "988d8124d4d3487d95e4c82290ebdc3e",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "29c9df903d544a9eb45372735dd89fa6",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-47"
   },
   "source": "## 加入非线性，堆叠才有新的表达能力",
   "id": "29c9df903d544a9eb45372735dd89fa6"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "c522e8395e16454494ca6cf011fcfa49",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-48"
   },
   "source": "仿射映射复合仍是仿射映射。在中间加入ReLU后，输入决定哪些单元通过，整体成为分段线性函数；深度增加的是可组合的非线性计算，不只是重复乘矩阵。\n\n感受野要同时跟踪范围 $r$ 与相邻输出对应的输入间距 $j$。从 $r_0=j_0=1$ 开始：\n$$r_l=r_{l-1}+(k_l-1)d_lj_{l-1},\\qquad j_l=s_lj_{l-1}.$$\n三层3×3、stride=1得到3→5→7；若中间插入stride=2的pool，后续每一步已跨越更大的输入间距。理论感受野是可能影响范围，当前输入和权重下的梯度强度还受ReLU/max选择影响。",
   "id": "c522e8395e16454494ca6cf011fcfa49"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "7be37899b4db4b09a8085530ee2b26e4",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-49"
   },
   "source": "r,j=1,1\nfor name,kernel,stride in [(\"conv1\",3,1),(\"conv2\",3,1),(\"pool\",2,2),(\"conv3\",3,1)]:\n    r=r+(kernel-1)*j\n    j=j*stride\n    print(name,\"receptive field\",r,\"jump\",j)\nassert (r,j)==(10,2)",
   "id": "7be37899b4db4b09a8085530ee2b26e4",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "f0d76f7c9c91409f80d04b59ed011d8d",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-50"
   },
   "source": "## 从特征图到类别分数",
   "id": "f0d76f7c9c91409f80d04b59ed011d8d"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "eb9aa73ad4074699837d3519a325bb5d",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-51"
   },
   "source": "池化将局部响应聚合，并常伴随下采样。Max选择最大元素，反向把梯度路由到获胜位置；Average平均分配局部贡献。它们减少空间信息，但不保证对所有平移不变：窗口边界变化可能改变最大值，抽样相位变化也可能改变输出。\n\n卷积网络常逐步降低空间分辨率、增加通道，把细粒度位置换成更丰富的局部表示。分类头需要把该表示映射到类别；Flatten+Linear保留位置专属权重，而GAP在空间平均后再分类引入另一种约束。1×1/GAP 的矩阵核对在 LAB03，NiN 的参考实现保留于 LAB05。",
   "id": "eb9aa73ad4074699837d3519a325bb5d"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "f79469dd10064648903ed9d75ea677d7",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-52"
   },
   "source": "## 分类的损失怎样训练前面的卷积",
   "id": "f79469dd10064648903ed9d75ea677d7"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "8cfa27f798044997891c4bdb84364113",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-53"
   },
   "source": "网络输出logits $z\\in\\mathbb R^C$，交叉熵可以写成\n$$L=\\log\\sum_j e^{z_j}-z_y,\\qquad \\frac{\\partial L}{\\partial z_j}=p_j-\\mathbf1[j=y].$$\n第一项使错误类分数参与竞争，第二项提高正确类相对分数。数值实现通常用log-sum-exp稳定计算：先减去最大logit，可避免直接指数溢出。PyTorch `cross_entropy`接收logits并合并相关计算，无须先softmax。\n\n这个梯度经过分类head、非线性和卷积的链式法则，回到早期kernel；卷积不是另一套学习算法，而是同一反向传播中的受约束模块。",
   "id": "8cfa27f798044997891c4bdb84364113"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "7e645356f33d4b6bb94edaed1c45c4e4",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-54"
   },
   "source": "z=torch.tensor([[2.,-1.,.5]],dtype=torch.float64,requires_grad=True)\ny=torch.tensor([2])\nF.cross_entropy(z,y).backward()\nexpected=z.detach().softmax(1)-F.one_hot(y,3)\nprint(\"logit gradient:\",z.grad)\ntorch.testing.assert_close(z.grad,expected)",
   "id": "7e645356f33d4b6bb94edaed1c45c4e4",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "e3954eee48cf4f2c863a52023ff3165c",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-55"
   },
   "source": "## 在 LAB 02 中区分三个问题",
   "id": "e3954eee48cf4f2c863a52023ff3165c"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "96e1afd65e5444ac9f7b3399ff4f6821",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-56"
   },
   "source": [
    "[第二讲 LAB02 · 小 CNN 与图像平移](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/c0b185e7-d7e3-48b6-a605-8f1acdfe243c/notebook/03c7a68e5b484caea0cdfc24a5e229a0?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=03c7a68e5b484caea0cdfc24a5e229a0&utm_term=get_notebook)保留 MLP/CNN × 有／无平移增强的四组 MNIST 结果。当前四组使用同一12,000张训练子集、各6 epochs、相同验证选择规则，参数和MAC未配平。\n",
    "\n",
    "读记录时分别回答：结构能否执行？训练目标是否下降？未见输入条件下表现怎样？同一模型的原图／右移配对比较隔离了输入变换；它不自动证明结构间的公平性。第一讲MLP属于另一训练预算，不与本册数字拼成同一对照。"
   ],
   "id": "96e1afd65e5444ac9f7b3399ff4f6821"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "083fa06aadce4bb9899aa389e8f26186",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-57"
   },
   "source": "## 自查：把推理说完整",
   "id": "083fa06aadce4bb9899aa389e8f26186"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "8dfa226160a047bc8c3508da69ec7a92",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-58"
   },
   "source": "1. 若允许每个空间位置有自己的kernel，哪一个等变约束被放松？参数量怎样增长？\n2. 一个kernel元素的梯度为何跨batch和位置求和，而不跨输出通道求和？\n3. 把所有训练输入改成常数图，学习到目标响应是否足以唯一确定kernel？\n4. 连续两层线性卷积是否能表达任意同感受野的大核？在有限宽度、边界和分解约束下需要检查什么？\n5. 同一CNN对右移两像素更稳，如何设计下一项对照来区分架构与增强的贡献？\n\n核对思路：分别检查参数约束、求导索引、design rank、函数分解和实验干预；答案应落到公式或可运行例子。",
   "id": "8dfa226160a047bc8c3508da69ec7a92"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "bc74cc57d7de42ccb64e7c87ed1b6fbd",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-59"
   },
   "source": "## 延伸阅读与下一步",
   "id": "bc74cc57d7de42ccb64e7c87ed1b6fbd"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "67f9d039a2ef45869da006d916185533",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-60"
   },
   "source": "本本改编自D2L第7章及本课程2025Fall/2026Spring卷积材料，补入共享梯度、框架源码与实验辨识讨论。D2L文字与示例代码来源及许可分别为[正文CC BY-SA 4.0](https://github.com/d2l-ai/d2l-en/blob/master/LICENSE)与[示例代码许可](https://github.com/d2l-ai/d2l-en/blob/master/LICENSE-SAMPLECODE)。本讲改编的D2L教学说明以CC BY-SA 4.0提供；代码遵循各源文件许可并保留归属。\n\n[CS231n卷积讲义](https://cs231n.stanford.edu/slides/2026/lecture_5.pdf) · [PyTorch Conv2d](https://docs.pytorch.org/docs/2.14/generated/torch.nn.Conv2d.html) · [感受野推导](https://distill.pub/2019/computing-receptive-fields/) · [抗混叠CNN](https://proceedings.mlr.press/v97/zhang19a.html)\n\n**下一本：LAB 05** · **返回导读**",
   "id": "67f9d039a2ef45869da006d916185533"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "33bfa88f8cd342c3839d5d27e3ec7eab",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-61"
   },
   "source": "## 按问题查阅：补充推导与机制核对",
   "id": "33bfa88f8cd342c3839d5d27e3ec7eab"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "d245eb7c1ac84a318744feb1bce3861a",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-62"
   },
   "source": [
    "**查阅方式。** A 解释局部与共享约束，B/C 核对通道和边界，D 看核／输入梯度，E/F 看实现存储与感受野。按问题选一节即可；算例有自己的形状与假设，不必把所有段落串成一次训练。\n",
    "\n",
    "需要回到现场图解时，用开头的 Slides 返回链接；全课材料分工见[第二讲导读](https://codingai-lec02.pages.dev/course/guide.html)。"
   ],
   "id": "d245eb7c1ac84a318744feb1bce3861a"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "04e72aa852224b67be7c173d4c6d6256",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-63"
   },
   "source": "## A. 从自由连接到共享算子",
   "id": "04e72aa852224b67be7c173d4c6d6256"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "2c026324d0b246d3917e3060ecc6b8ab",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-64"
   },
   "source": "先考虑一个最自由的空间线性层：输出位置 (i,j) 与输入位置 (k,l) 的连接可拥有独立权重 Wᵢⱼₖₗ，偏置也可随输出位置变化。把 k=i+a,l=j+b 改写为相对偏移后，记为 Vᵢⱼₐᵦ。这只是索引重命名，尚未减少自由度。\n\n进一步要求同一局部规则可在不同位置复用，使 Vᵢⱼₐᵦ=Kₐᵦ，并使用空间共享偏置 β；再限制有限偏移范围，得到局部共享计算。相似局部模式可以出现在各处，激活数值却不必相同，因为每个窗口的输入不同。前向复用同一参数，反向就要汇总它每次使用的贡献。\n\n在无限网格或相容的边界定义、stride=1 下，平移可与共享卷积交换：先移输入再卷积，与先卷积再移响应一致。这是等变，不是类别输出不变。有限边界和下采样会改变成立条件；stride=s 通常只对与输出网格相容的位移保留相应关系。\n\n**先判断**：只把绝对坐标换成相对偏移，为什么还不能说得到了卷积？\n\n**核对提示**：参数仍可能依赖输出位置 (i,j)。还要规定位置共享及局部支撑；若偏置随位置变，也会破坏所要求的平移关系。\n\n**推导时可用的关系式**\n\n$$H_{ij}=U_{ij}+\\sum_{k,l}W_{ij,kl}X_{kl}$$\n\n这是全连接仿射层的空间索引写法，还没有卷积假设。\n\n$$V_{ijab}:=W_{ij,i+a,j+b}$$\n\n一一对应地换索引，参数数量不变。\n\n$$H_{ij}=U_{ij}+\\sum_{a,b}V_{ijab}X_{i+a,j+b}$$\n\n把输出位置与输入相对偏移显式分开。\n\n$$f(T_\\Delta X)=T_\\Delta f(X)$$\n\n等变保留位置关系；不变则要求f(TΔX)=f(X)。\n\n$$H_{ij}=\\beta+\\sum_{a,b}K_{ab}X_{i+a,j+b}$$\n\n位置索引从参数中消失，空间共享由此产生。\n\n$$(C_KT_\\Delta X)_{ij}=\\sum_{a,b}K_{ab}X_{i+a-\\Delta_i,j+b-\\Delta_j}=(T_\\Delta C_KX)_{ij}$$\n\n先忽略bias或使用空间共享bias，并采用相容边界。\n\n**来源与继续阅读**\n\n- [D2L 7.1 · From Fully Connected Layers to Convolutions](https://d2l.ai/chapter_convolutional-neural-networks/why-conv.html)\n- [CS231n 2026 · CNNs](https://cs231n.stanford.edu/slides/2026/lecture_5.pdf)\n- [D2L · Forward and Backward Propagation](https://d2l.ai/chapter_multilayer-perceptrons/backprop.html)\n- [D2L 7.2 · Convolutions for Images](https://d2l.ai/chapter_convolutional-neural-networks/conv-layer.html)\n- [Zhang · Making CNNs Shift-Invariant Again (2019)](https://proceedings.mlr.press/v97/zhang19a.html)",
   "id": "2c026324d0b246d3917e3060ecc6b8ab"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "a1c6bf20c1f64e8c9f1c750ce76ab3b5",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-65"
   },
   "source": "## B. 多通道接口与 1×1 配方",
   "id": "a1c6bf20c1f64e8c9f1c750ce76ab3b5"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "70387b5f35564a4692659af646f9fc1c",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-66"
   },
   "source": "一个窗口与核逐项相乘并求和，产生一个输出；扫描合法窗口产生响应图。标准卷积的每个输出通道读取全部输入通道，因此权重有 [C_out,C_in,K_h,K_w] 四个轴。多套 filter 保留不同测量，通道不预先对应一套固定的人类概念。\n\n对 RGB=(0.8,0.2,0.1)，三个手设 1×1 配方 (⅓,⅓,⅓)、(1,−1,0)、(0,0,1)，分别输出约 0.367、0.6、0.1。它们没有训练且无偏置；通道平均也不等于标准感知亮度。\n\n输入 32×32×3，使用 16 个 3×3 filter、stride=1、padding=1、带偏置：参数 16×3×9+16=448，输出 32×32×16，每张图卷积 MAC 为 32×32×16×27=442368（不计偏置加法）。图变大时参数不变，计算量增加；如果后接固定 Flatten 分类头，头部输入维度还会改变。\n\n减小空间尺寸与增加通道是两个不同的设计旋钮。空间下采样舍弃一部分位置细节，增加通道提供更多响应种类，并不自动找回被丢掉的位置。\n\n**先判断**：一个输出通道是不是只对应一个输入通道？\n\n**核对提示**：标准 groups=1 卷积不是。它把全部输入通道的局部乘加汇总为一个输出通道；只有特定分组连接才限制通道之间的组合。\n\n**来源与继续阅读**\n\n- [D2L · Convolutions for Images](https://d2l.ai/chapter_convolutional-neural-networks/conv-layer.html)\n- [CS231n 2026 · CNNs](https://cs231n.stanford.edu/slides/2026/lecture_5.pdf)\n- [D2L · Input and Output Channels](https://d2l.ai/chapter_convolutional-neural-networks/channels.html)\n- [D2L · VGG](https://d2l.ai/chapter_convolutional-modern/vgg.html)\n- [CS231n 2026 · CNN Architectures](https://cs231n.stanford.edu/slides/2026/lecture_6.pdf)",
   "id": "70387b5f35564a4692659af646f9fc1c"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "c572e8e4b64f42c9b85d5a5bd94b6f1f",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-67"
   },
   "source": "## C. 边界与合法窗口",
   "id": "c572e8e4b64f42c9b85d5a5bd94b6f1f"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "e02b1a7b3f3f47a5ab0a5c3e0c88fc26",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-68"
   },
   "source": "Padding 是边界模型，不是新增测量。对原信号 [2,5,7] 和差分核 [−1,0,1]，最左端窗口在补零、复制、反射（不重复边界点）下分别为 [0,2,5]、[2,2,5]、[5,2,5]，响应依次 5、3、0。张量形状相同，函数也可能不同。\n\nDilation 把核元素间距拉开，参数数目不变，有效核宽变成 d(K−1)+1。3×3 核、d=2 的跨度是 5×5，但并没有密集读取其中 25 个位置。\n\n单轴输出长度是 floor((H+P_left+P_right−d(K−1)−1)/s)+1。先列出合法窗口起点，再用公式核对：H=6,K=3,s=2,无填充时起点为 0、2，输出为 2；不能因为 6/2=3 就写 3。高宽应各算一次，非对称 padding 也应分别记录。\n\n**先判断**：保持输出尺寸是否意味着没有信息损失、也没有额外假设？\n\n**核对提示**：都不意味着。边界延拓加入了假设；后续算子与通道组合仍可能合并信息。形状账单只回答尺寸，不能替代信息判断。\n\n**推导时可用的关系式**\n\n$$H_{out}=\\left\\lfloor\\frac{H+P_{left}+P_{right}-d(K-1)-1}{s}+1\\right\\rfloor$$\n\n非对称padding的通用一维写法。\n\n**来源与继续阅读**\n\n- [D2L 7.3 · Padding and Stride](https://d2l.ai/chapter_convolutional-neural-networks/padding-and-strides.html)\n- [Zhang · Making CNNs Shift-Invariant Again (2019)](https://proceedings.mlr.press/v97/zhang19a.html)\n- [PyTorch · Conv2d API](https://docs.pytorch.org/docs/stable/generated/torch.nn.Conv2d.html)\n- [Araujo et al. · Computing Receptive Fields (2019)](https://distill.pub/2019/computing-receptive-fields/)",
   "id": "e02b1a7b3f3f47a5ab0a5c3e0c88fc26"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "27eac9133cc0440f86792880e03f4b21",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-69"
   },
   "source": "## D. 核梯度、输入梯度与池化反馈",
   "id": "27eac9133cc0440f86792880e03f4b21"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "77c540baf52b40eabdbcbfe79aeb7ceb",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-70"
   },
   "source": "记上游反馈 δ=∂L/∂Y。某个核元素在每个窗口中的局部导数是对应输入值，因此核梯度是 δ×输入在所有样本与输出位置上的和。核索引中的输出通道 o 是固定的，不应再对 o 求和；输入梯度则要找所有使用了该输入元素的输出，并累积对应 δ×权重。\n\n沿用主讲的 X=[1,2,3],K=[1,−1]，两输出均为 −1，教学目标设为 0，L=½(y₁²+y₂²)=1。两窗口梯度贡献 [−1,−2] 与 [−2,−3] 汇为 [−3,−5]；η=0.05 后 K′=[1.15,−0.75]，y′=[−0.35,0.05]，L′=0.0625。两个位置的贡献先算完，才统一更新。这里的目标零是便于核对的构造；分类 CNN 的反馈来自最终类别损失。\n\n若 η=0.1，则 K′=[1.3,−0.5],y′=[0.3,1.1],L′=0.65。第二个误差变大来自有限步长越过该位置的零误差点，不能拿它说明两路初始梯度彼此反对。\n\n池化也改变梯度路由。窗口 [[1,4],[2,3]] 的最大值为 4，平均值为 2.5；上游梯度为 8 时，max 把 8 交给唯一最大位置，average 给每个位置 2。若最大值并列，要结合具体实现的规则，不能沿用唯一最大值的手算。\n\n**先判断**：窗口移动时是否先改一次核，再让下一个窗口读取新核？\n\n**核对提示**：不是本例的共享前向。所有位置使用同一份当前核完成前向与求导，然后更新一次。边扫描边改核定义了另一种计算过程。\n\n**推导时可用的关系式**\n\n$$\\frac{\\partial\\mathcal L}{\\partial K_{ocab}}=\\sum_{n,i,j}\\delta_{noij}X_{nc,i+a,j+b}$$\n\n此式展示stride=1的索引；求和来自同一参数的多次使用。\n\n$$\\frac{\\partial L}{\\partial K_{ocab}}=\\sum_{n,i,j}\\delta_{noij}X_{nc,si-p+da,sj-p+db}$$\n\n为紧凑起见高宽采用同一s、p、d；实际可分别取值。\n\n**来源与继续阅读**\n\n- [D2L · Pooling](https://d2l.ai/chapter_convolutional-neural-networks/pooling.html)\n- [Stanford CS231n · Backpropagation](https://cs231n.github.io/optimization-2/)\n- [D2L 7.2 · Convolutions for Images](https://d2l.ai/chapter_convolutional-neural-networks/conv-layer.html)\n- [D2L 5.3 · Forward Propagation, Backward Propagation](https://d2l.ai/chapter_multilayer-perceptrons/backprop.html)\n- [PyTorch · Autograd mechanics and evaluation mode](https://docs.pytorch.org/docs/stable/notes/autograd.html)",
   "id": "77c540baf52b40eabdbcbfe79aeb7ceb"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "04fdc0fc1db444379a95e0d48f3c9c3d",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-71"
   },
   "source": "## E. Python 调用、unfold 与分组卷积",
   "id": "04fdc0fc1db444379a95e0d48f3c9c3d"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "5c1409e3c4a443f393ce65cfbe88ec9a",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-72"
   },
   "source": "nn.Conv2d 管理可训练权重与偏置；forward 把它们和 stride、padding、dilation、groups 传给函数式算子。下面保存的是课程固定版本的源码节选，用来辨认职责，不代表你当前环境或底层 CPU/GPU 内核的完整实现。独立运行 inspect 查看当前版本，比把一段节选当作永久 API 更可靠。\n\nunfold 把每个窗口排为一列，P 的行数为 C_inK_hK_w，列数为 H_outW_out。把 filter 排为矩阵后，K_flat P 与标准 groups=1 卷积的乘加等价，最后还原空间轴。显式展开的临时存储可能很大；数学等价不意味着这是最快实现。\n\ngroups=g 把通道连接分组，权重形状是 [C_out,C_in/g,K_h,K_w]，参数数目（带 bias）为 C_out(C_in/g)K_hK_w+C_out。输入和输出通道须满足分组整除要求。Depthwise 让每个输入通道独立做空间处理，通常再由 pointwise 操作混合通道；节省计算的同时限制了本层能组合的信息。\n\n**先判断**：为什么 groups 增加后，不能只看输出 shape 就认为表示能力没变？\n\n**核对提示**：shape 可保持不变，但允许的跨通道连接变少。同一输出坐标能使用哪些输入已经改变，这是函数族的变化。\n\n**推导时可用的关系式**\n\n$$P=C_{out}\\frac{C_{in}}gK_hK_w+C_{out}$$\n\nbias=True；C_in与C_out满足groups的整除要求。\n\n**PyTorch Conv2d：删节，仅展示调用链与zeros分支**\n\n身份：PyTorch源码删节；完整出处与固定版本见链接。这是保留的说明节选，所需上下文未必齐全，不能按完整程序单独运行。\n\n```python\ndef forward(self, input):\n    return self._conv_forward(input, self.weight, self.bias)\n\n# _conv_forward 内 padding_mode == \"zeros\" 的分支\nreturn F.conv2d(input, weight, bias, self.stride,\n                self.padding, self.dilation, self.groups)\n```\n\n[固定出处](https://github.com/pytorch/pytorch/blob/08187d9e0fba026dc8217405802ab5381dc88d90/torch/nn/modules/conv.py)\n\n**课程实现：groups=1、bias略**\n\n身份：课程教学实现。这是保留的说明节选，所需上下文未必齐全，不能按完整程序单独运行。\n\n```python\npatches = F.unfold(x, kernel_size=3, padding=1)\nw = weight.flatten(1)\ny_flat = torch.einsum(\"oc,ncl->nol\", w, patches)\ny = y_flat.reshape(x.shape[0], weight.shape[0], H, W)\n```\n\n**来源与继续阅读**\n\n- [PyTorch 2.14.0 · Conv2d source (08187d9e)](https://github.com/pytorch/pytorch/blob/08187d9e0fba026dc8217405802ab5381dc88d90/torch/nn/modules/conv.py)\n- [PyTorch · Conv2d API](https://docs.pytorch.org/docs/stable/generated/torch.nn.Conv2d.html)\n- [D2L 7.2 · Convolutions for Images](https://d2l.ai/chapter_convolutional-neural-networks/conv-layer.html)\n- [D2L 7.4 · Multiple Input and Multiple Output Channels](https://d2l.ai/chapter_convolutional-neural-networks/channels.html)\n- [D2L 8.3 · Network in Network](https://d2l.ai/chapter_convolutional-modern/nin.html)",
   "id": "5c1409e3c4a443f393ce65cfbe88ec9a"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "156084d6cb7b4db9ad585c7e0b30390d",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-73"
   },
   "source": "## F. 感受野递推的结构账单",
   "id": "156084d6cb7b4db9ad585c7e0b30390d"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "d45f8484fc9b45ea98f99a7a4b98368b",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-74"
   },
   "source": "单轴单路径从 r=1,jump=1 开始。每层有效核宽 k_eff=d(k−1)+1，更新 r←r+(k_eff−1)jump，再令 jump←jump×stride。顺序不能颠倒：这一层的跨度使用进入本层前的采样间隔。\n\n课程核对序列 (k,s,d)=[(3,1,1),(2,2,1),(3,1,1)] 依次得到 (r,jump)=(3,1),(4,2),(8,2)。这是结构允许的理论跨度，尚未告诉你某个输入位置实际影响多大；有效影响与局部敏感性的边界见 [第二讲 CLS 05 · 观察与迁移](https://codingai-lec02.pages.dev/course/c05.html) 的参考部分。遇到多分支、非对称边界或坐标中心，应扩展账单，不能机械套用单路径递推。\n\n**先判断**：同样是两个 3×3 卷积，中间插入一次 stride=2 后，后层增加的输入跨度为什么不同？\n\n**核对提示**：进入后层前的 jump 变大，相邻后层采样点在原图中隔得更远。因此必须同时追踪范围和采样间隔。\n\n**课程工具：单轴、单路径**\n\n身份：课程教学实现。这是保留的说明节选，所需上下文未必齐全，不能按完整程序单独运行。\n\n```python\nr, jump = 1, 1\nfor kernel, stride, dilation in layers:\n    effective = dilation * (kernel - 1) + 1\n    r += (effective - 1) * jump\n    jump *= stride\n    print(\"receptive field:\", r, \"jump:\", jump)\n```\n\n**来源与继续阅读**\n\n- [Araujo et al. · Computing Receptive Fields (2019)](https://distill.pub/2019/computing-receptive-fields/)\n- [Luo et al. · Understanding the Effective Receptive Field (2016)](https://arxiv.org/abs/1701.04128)\n- [D2L 8.2 · Networks Using Blocks](https://d2l.ai/chapter_convolutional-modern/vgg.html)",
   "id": "d45f8484fc9b45ea98f99a7a4b98368b"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "0e01b3e72f1d4c50bec7a65f03c398e4",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-75"
   },
   "source": "## 从主讲继续：推导边界与研究来源",
   "id": "0e01b3e72f1d4c50bec7a65f03c398e4"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "203878a903244acf8679ac4a8baadf9d",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-76"
   },
   "source": "课堂模型固定为灰度 1×28×28 → Conv(1→8,3×3,p=1)/ReLU → Pool2 → Conv(8→16,3×3,p=1)/ReLU → Pool2 → Flatten784 → Linear32/ReLU → Linear10。两个池化后的形状依次为 8×14×14 与 16×7×7。Flatten 只换索引。主讲预览与回访使用同一模型，下面的代码片段只是核对工具，不把它悄悄替换成 GAP 网络。主讲平移 2×2 表格来自既有单种子协议，计算量未对齐；先比较同一模型有无增强，再讨论跨系统差异，不将它解释为 CNN 的普遍优势。\n\n**延伸自 一个filter读取全部输入通道**\n\n依据D2L示例改写；corr2d为课堂实现 · D2L逻辑的教学改写\n```python\ndef corr_multi_in(X, K):\n    return sum(corr2d(x, k)\n               for x, k in zip(X, K))\n```\nhttps://d2l.ai/chapter_convolutional-neural-networks/channels.html\n\n**延伸自 两个窗口的贡献，汇总到同一对核权重**\n\n步长追问（备份）：同一初始核与总梯度，η=0.1得到K′=[1.3,−0.5]、y′=[0.3,1.1]、L′=0.65。初始两路贡献[−1,−2]、[−2,−3]并不相互反对；第二个位置在这一步跨过目标零并走得过远。沿更新方向，y₀′=−1+13η、y₁′=−1+21η，因此分别在η=1/13、1/21时经过零。这个追问研究步长和汇总目标，不用于首次解释共享梯度。\n\n**延伸自 回到同一台CNN，解释预测与学习**\n\n课程教学实现 · 课程工具：沿Sequential打印形状\n```python\nwas_training = model.training\nmodel.eval()\nx = torch.zeros(2, 1, 28, 28)\nwith torch.no_grad():\n    for layer in model:\n        x = layer(x)\n        print(type(layer).__name__, tuple(x.shape))\nmodel.train(was_training)\n```\n\n\n上述 corr_multi_in 片段依赖已定义的 corr2d；Sequential 工具依赖已有 model 与 torch。它们是教学实现节选，完整可执行起点见 LAB01 与 LAB02，不能把片段的展示等同于本次已经运行完整模型。步长追问中，两路输出沿更新方向分别为 −1+13η 和 −1+21η，因此在 η=1/13 与 η=1/21 时分别经过零。\n\n**主讲研究来源**\n\n- [D2L · LeNet](https://d2l.ai/chapter_convolutional-neural-networks/lenet.html)\n- [CS231n 2026 · CNNs](https://cs231n.stanford.edu/slides/2026/lecture_5.pdf)\n- [D2L · From Fully Connected Layers to Convolutions](https://d2l.ai/chapter_convolutional-neural-networks/why-conv.html)\n- [D2L · Convolutions for Images](https://d2l.ai/chapter_convolutional-neural-networks/conv-layer.html)\n- [D2L · Multiple Input and Multiple Output Channels](https://d2l.ai/chapter_convolutional-neural-networks/channels.html)\n- [Stanford CS231n · Backpropagation](https://cs231n.github.io/optimization-2/)\n- [D2L · Forward and Backward Propagation](https://d2l.ai/chapter_multilayer-perceptrons/backprop.html)\n- [D2L · Padding and Stride](https://d2l.ai/chapter_convolutional-neural-networks/padding-and-strides.html)\n- [Zhang · Making CNNs Shift-Invariant Again (2019)](https://proceedings.mlr.press/v97/zhang19a.html)\n- [Araujo et al. · Computing Receptive Fields (2019)](https://distill.pub/2019/computing-receptive-fields/)\n- [D2L · VGG](https://d2l.ai/chapter_convolutional-modern/vgg.html)\n- [Luo et al. · Understanding the Effective Receptive Field (2016)](https://arxiv.org/abs/1701.04128)\n- [D2L · Pooling](https://d2l.ai/chapter_convolutional-neural-networks/pooling.html)\n- [Long et al. · Fully Convolutional Networks (2015)](https://arxiv.org/abs/1411.4038)\n- [Zeiler & Fergus · Visualizing and Understanding Convolutional Networks (2014)](https://arxiv.org/abs/1311.2901)",
   "id": "203878a903244acf8679ac4a8baadf9d"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "e0b65a465300436c9268c58b6e4d3df8",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-77"
   },
   "source": "## 独立核对：先预测，再运行",
   "id": "e0b65a465300436c9268c58b6e4d3df8"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "ef2bfde427df4c2dbe3a566ca88b7ac1",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-78"
   },
   "source": [
    "下面是完整、独立的小规模 Python 核对，只使用标准库；无需下载、安装、训练或访问外部服务。它保留“教学构造”的身份。先写预测，运行后解释哪一项支持判断，再只改一个条件复查。下方没有预填远端执行输出。\n",
    "\n",
    "本块固定 `x=[1,2,3]`、`k=[1,−1]`，分别比较学习率0.05与0.1。先手算两项核梯度，再运行；看 `outputs` 和重新计算的 `loss`，不要只看参数有没有动。想改变输入时，先重写受影响的数值断言前提，不要直接删断言。"
   ],
   "id": "ef2bfde427df4c2dbe3a566ca88b7ac1"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "6fc321b6262e4710b5c5864dcaec7082",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-79"
   },
   "source": "# 自包含手算核对：不依赖另一 notebook 的内存。\nx=[1.,2.,3.]; k=[1.,-1.]\ndef forward(weights):\n    return [sum(x[i+j]*weights[j] for j in range(2)) for i in range(2)]\ndef loss(weights): return .5*sum(y*y for y in forward(weights))\ny=forward(k)\ng=[sum(y[i]*x[i+j] for i in range(2)) for j in range(2)]\neps=1e-6\nfor j in range(2):\n    plus=k.copy();minus=k.copy();plus[j]+=eps;minus[j]-=eps\n    finite=(loss(plus)-loss(minus))/(2*eps)\n    assert abs(finite-g[j])<1e-8\nfor eta in [.05,.1]:\n    updated=[w-eta*dg for w,dg in zip(k,g)]\n    print('eta',eta,'kernel',updated,'outputs',forward(updated),'loss',loss(updated))\nassert abs(loss([1.15,-.75])-.0625)<1e-12\nr,jump=1,1\nfor kernel,stride,dilation in [(3,1,1),(2,2,1),(3,1,1)]:\n    r+=(dilation*(kernel-1))*jump;jump*=stride\n    print('理论跨度',r,'采样间隔',jump)\nassert (r,jump)==(8,2)\n",
   "id": "6fc321b6262e4710b5c5864dcaec7082",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "2c7ebc23da174142804ca51277d40a6d",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-80"
   },
   "source": "**继续一步**：修改一个数值或条件，先写变化方向，再运行核对。不要删除断言来让代码“通过”；若主动改变了断言的前提，应同时写出新预期。机制小算例通过，只说明这一小段计算与预期一致，不等于完整图像实验或新架构训练成功。",
   "id": "2c7ebc23da174142804ca51277d40a6d"
  }
 ]
}