{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  },
  "lecture_migration": {
   "project_id": "c0b185e7-d7e3-48b6-a605-8f1acdfe243c",
   "notebook_id": "ef41c1445ff347e4b27cb2bb891e817c",
   "destination": "L2",
   "cloud_run": false
  },
  "title": "LAB 09 · 视觉 Attention 完整层"
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "eebad92d9e1546e7990d247217afbb54",
     "type": "text-cell-h1",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-1"
   },
   "source": "LAB 09 · 视觉 Attention 完整层",
   "id": "eebad92d9e1546e7990d247217afbb54"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "479c8fe615c14a38863207a9c03b7c72",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-2"
   },
   "source": [
    "[返回第二讲 · 完整 Attention 选读](https://codingai-lec02.pages.dev/course/reference.html#lecture05) · [网页阅读版](https://codingai-lec02.pages.dev/course/lab11.html) · [第二讲导读](https://codingai-lec02.pages.dev/course/guide.html) · [问卷作业 ↗](https://tally.so/r/VLroVv)（右键在新标签页打开，保留当前材料）\n",
    "\n",
    "**改一块图像，为什么另一块的输出也会改变？** 本册是完整 Attention 的后续选读，不是本次课堂或第三讲的前置任务。\n",
    "\n",
    "用 4×4 灰度图、四个 2×2 patch、4 维表示与双 head 走完一个 pre-norm 层：`X → LN → attention → 加 X → LN → 逐 token MLP → 第二条残差`。全部权重固定，没有训练和类别标签；保存张量支持核对计算，不证明学到了语义关注。\n",
    "\n",
    "| 读法 | 从哪里开始 | 具体动作 |\n",
    "|---|---|---|\n",
    "| 先读保存结果 | 第 1、3、4 节的图 | 追踪被修改 patch 对其他 token 的影响 |\n",
    "| 核对一行计算 | 第 3 节 | 选 head=0、query=0，手算四个 V 的加权和 |\n",
    "| 独立重算 | 从第一个代码块顺序运行 | 只改一个像素或位置条件，再比较中间张量 |\n",
    "\n",
    "本册只需 NumPy 与 Matplotlib，不依赖其他 Notebook 的运行内存。语言的因果 mask 可在 [Lecture 01 · LAB05](https://deepnote.com/project/1eb6dbb3-8e29-4ff8-8a04-fb41b07e5f6e/notebook/4f43fb0786804aba8f4b07c737652c02) 对照阅读；这里四个图块互相可见，没有 causal mask。\n",
    "\n",
    "[参考图解：图像 patch](https://codingai-lec02.pages.dev/course/reference/frontier-patches-to-tokens.html) · [参考图解：读回与完整层](https://codingai-lec02.pages.dev/course/reference/frontier-attention-worked-example.html)"
   ],
   "id": "479c8fe615c14a38863207a9c03b7c72"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "dd59248ca5a6418d8e4c90cba5a709df",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-3"
   },
   "source": "## 1. 先作预测，再建立可复算的输入",
   "id": "dd59248ca5a6418d8e4c90cba5a709df"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "8a94ee860fe04e8c8521a9a2d481e6fe",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-4"
   },
   "source": "只改右下 patch 的一个像素：你预计是一个输出 token 变，还是全部 token 都可能变？如果 4 个 token 完全相同、又没有位置差异，注意力矩阵还会偏向某个位置吗？\n\n运行要求：Python 3.10+、NumPy、Matplotlib。若缺依赖，在 notebook 外安装 `python -m pip install numpy matplotlib`。重算只使用本文件代码；输出写入当前目录下 `outputs/LAB-B`，不会覆盖其他 Lab。",
   "id": "8a94ee860fe04e8c8521a9a2d481e6fe"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "39ef3d78b64848f685cdbdec0726ee1c",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-5"
   },
   "source": "from pathlib import Path\nimport json, sys, platform, hashlib\nimport numpy as np\nimport matplotlib.pyplot as plt\nnp.set_printoptions(precision=4, suppress=True)\nOUTPUT = Path('outputs/LAB-B')\nOUTPUT.mkdir(parents=True, exist_ok=True)\nSEED = 20260918\nDTYPE = np.float64\nEPS = 1e-5\nplt.rcParams.update({'font.size': 11, 'figure.dpi': 120, 'axes.spines.top': False, 'axes.spines.right': False})\nimage = np.array([[0., 0.2, 0.8, 1.0], [0.1, 0.3, 0.7, 0.9],\n                  [0.9, 0.7, 0.2, 0.1], [1., 0.8, 0.3, 0.]], dtype=DTYPE)\ndef patchify(im):\n    # Row-major order: top-left, top-right, bottom-left, bottom-right.\n    return im.reshape(2, 2, 2, 2).transpose(0, 2, 1, 3).reshape(4, 4)\npatches = patchify(image)\nW_patch = np.array([[1., .1, 0., -.2], [.2, .8, -.1, 0.],\n                    [0., -.2, 1., .2], [-.1, 0., .1, .9]])\nposition = np.array([[.2, 0., 0., 0.], [0., .2, 0., 0.],\n                     [0., 0., .2, 0.], [0., 0., 0., .2]])\nembedded = patches @ W_patch\nX = embedded + position\nprint('image', image.shape, 'patches', patches.shape, 'embedding', embedded.shape, 'tokens X', X.shape)\nprint('patch vectors:\\n', patches)\nprint('X = patches @ W_patch + position:\\n', X)\nfig, ax = plt.subplots(1, 2, figsize=(9, 3.8))\na = ax[0].imshow(image, cmap='gray', vmin=0, vmax=1)\nax[0].axvline(1.5, color='#e6b450', lw=2); ax[0].axhline(1.5, color='#e6b450', lw=2)\nfor i, (x, y) in enumerate([(0.5, .5), (2.5, .5), (.5, 2.5), (2.5, 2.5)]):\n    ax[0].text(x,y,f'P{i}',ha='center',color='#f2ab32',fontweight='bold')\nax[0].set_title('Constructed image | 4 patches')\nim = ax[1].imshow(X, cmap='viridis'); fig.colorbar(im, ax=ax[1], shrink=.8)\nax[1].set(title='Projected pixels + position', xlabel='Channel', ylabel='Patch token')\nfig.tight_layout(); fig.savefig(OUTPUT/'01-patches.png'); plt.show()\n",
   "id": "39ef3d78b64848f685cdbdec0726ee1c",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "e47a130be36c45ea95b3e9740152d5f0",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-6"
   },
   "source": "**保存输出 · 2026-09-18 本地逐单元计算，非 Deepnote 云端运行**\n\n```text\nimage (4, 4) patches (4, 4) embedding (4, 4) tokens X (4, 4)\npatch vectors:\n [[0.  0.2 0.1 0.3]\n [0.8 1.  0.7 0.9]\n [0.9 0.7 1.  0.8]\n [0.2 0.1 0.3 0. ]]\nX = patches @ W_patch + position:\n [[0.21 0.14 0.11 0.29]\n [0.91 0.94 0.69 0.79]\n [0.96 0.45 1.21 0.74]\n [0.22 0.04 0.29 0.22]]\n```\n\n![本单元实际计算保存图](https://codingai-lec02.pages.dev/course/images/LAB11-B/figure-01.png)",
   "id": "e47a130be36c45ea95b3e9740152d5f0"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "e4dc6917b21946ad99014b6e3018672e",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-7"
   },
   "source": "位置向量只是按 token 加进去的固定数；它不是从图片推断的坐标。此处 4 个 patch 都保留，没有 class token 或最终分类头。`W_patch` 和后面所有权重都是固定教学权重。",
   "id": "e4dc6917b21946ad99014b6e3018672e"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "56cded974bd8475ab2a3587bebbafe9b",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-8"
   },
   "source": "## 2. 完整层的约定",
   "id": "56cded974bd8475ab2a3587bebbafe9b"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "fe1a2dd0e8d44ca38fbf1ae6987dac73",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-9"
   },
   "source": "每个 token 独立做 LayerNorm：按它自己的 4 个通道求均值、方差；`gamma=1, beta=0, epsilon=1e-5`。每个 head 的维度 `d_k=d_v=2`，因此分数除以 `sqrt(2)`，再**沿被读取的 key 轴**做 softmax。\n\n`Q,K,V` 都来自第一个 LayerNorm 的输出。每个 head 读回的 V 拼接后经过 `W_O`，才与原始 X 相加。第二个 LN 的结果经过同一套逐 token MLP（4→6→4、GELU），最后与第一条残差的结果相加。dropout 关闭；本例没有 bias，LN affine 取恒等。\n\n先核对维度，再看数字。否则很容易把“各位置交换信息”与“每个位置变换通道”混为一谈。",
   "id": "fe1a2dd0e8d44ca38fbf1ae6987dac73"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "84bfd61e82544c7eac798e5a0d0901fc",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-10"
   },
   "source": "D, HEADS, DH, DFF = 4, 2, 2, 6\nrng = np.random.default_rng(SEED)\nweights = {name: rng.normal(0, .35, shape).astype(DTYPE) for name, shape in [\n    ('W_Q', (D,D)), ('W_K', (D,D)), ('W_V', (D,D)), ('W_O', (D,D)),\n    ('W_1', (D,DFF)), ('W_2', (DFF,D))]}\ndef ln(x):\n    return (x-x.mean(axis=-1,keepdims=True))/np.sqrt(x.var(axis=-1,keepdims=True)+EPS)\ndef softmax(x):\n    e = np.exp(x-x.max(axis=-1,keepdims=True))\n    return e/e.sum(axis=-1,keepdims=True)\ndef gelu(x):\n    # The tanh approximation is declared explicitly; no library default ambiguity.\n    return .5*x*(1+np.tanh(np.sqrt(2/np.pi)*(x+.044715*x**3)))\ndef encoder(x):\n    t = {'input': x.copy(), 'ln1': ln(x)}\n    for name in ('Q','K','V'):\n        t[name] = t['ln1'] @ weights['W_'+name]\n        t[name+'_heads'] = t[name].reshape(len(x),HEADS,DH).transpose(1,0,2)\n    t['scores'] = t['Q_heads'] @ t['K_heads'].transpose(0,2,1) / np.sqrt(DH)\n    t['attention'] = softmax(t['scores'])\n    t['read_values'] = t['attention'] @ t['V_heads']\n    t['concat'] = t['read_values'].transpose(1,0,2).reshape(len(x),D)\n    t['projected_attention'] = t['concat'] @ weights['W_O']\n    t['residual1'] = x + t['projected_attention']\n    t['ln2'] = ln(t['residual1'])\n    t['mlp_hidden_pre'] = t['ln2'] @ weights['W_1']\n    t['mlp_hidden'] = gelu(t['mlp_hidden_pre'])\n    t['mlp_output'] = t['mlp_hidden'] @ weights['W_2']\n    t['output'] = t['residual1'] + t['mlp_output']\n    return t\ntrace = encoder(X)\nfor name, value in trace.items(): print(f'{name:22s} {str(value.shape):14s}')\nprint('Head 0 attention weights (row=query, column=key):\\n', trace['attention'][0])\nprint('Head 0 read-back values:\\n', trace['read_values'][0])\nprint('Final output:\\n', trace['output'])\n",
   "id": "84bfd61e82544c7eac798e5a0d0901fc",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "61a4c00168964301b4102e3cbd13d7bb",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-11"
   },
   "source": "**保存输出 · 2026-09-18 本地逐单元计算，非 Deepnote 云端运行**\n\n```text\ninput                  (4, 4)        \nln1                    (4, 4)        \nQ                      (4, 4)        \nQ_heads                (2, 4, 2)     \nK                      (4, 4)        \nK_heads                (2, 4, 2)     \nV                      (4, 4)        \nV_heads                (2, 4, 2)     \nscores                 (2, 4, 4)     \nattention              (2, 4, 4)     \nread_values            (2, 4, 2)     \nconcat                 (4, 4)        \nprojected_attention    (4, 4)        \nresidual1              (4, 4)        \nln2                    (4, 4)        \nmlp_hidden_pre         (4, 6)        \nmlp_hidden             (4, 6)        \nmlp_output             (4, 4)        \noutput                 (4, 4)        \nHead 0 attention weights (row=query, column=key):\n [[0.1624 0.1705 0.3532 0.314 ]\n [0.3184 0.3073 0.179  0.1953]\n [0.0366 0.044  0.554  0.3655]\n [0.0378 0.0452 0.5511 0.3659]]\nHead 0 read-back values:\n [[-0.3022 -0.0763]\n [-0.0288 -0.1743]\n [-0.5419  0.0128]\n [-0.5396  0.0118]]\nFinal output:\n [[-0.5644  0.4532  0.0506 -0.1394]\n [ 0.6503  0.9269  0.6401  0.332 ]\n [-0.0476  1.3125  1.3218  0.9238]\n [-0.7901  0.6609  0.2428  0.0266]]\n```",
   "id": "61a4c00168964301b4102e3cbd13d7bb"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "6a1365fe35ad48c399b23c4291c766b4",
     "type": "text-cell-h3",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-12"
   },
   "source": "### 中间数值表：先查看，再独立核对一行",
   "id": "6a1365fe35ad48c399b23c4291c766b4"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "9219b1754ac34858bd50ddf04726a6ed",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-13"
   },
   "source": "下面列出投影、缩放后的分数和完整层两条分支。全表也写入 `results.json` 与 `tensors.npz`，因此可以在不截屏、不复制舍入数字的情况下核算。手算时选择一个 query、一个 head 即可；不要把四位小数的显示误差当成模型差异。",
   "id": "9219b1754ac34858bd50ddf04726a6ed"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "ea92ff516363465aa3110663a036d170",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-14"
   },
   "source": "for name in ['ln1','Q','K','V','scores','concat','projected_attention','residual1','ln2','mlp_hidden_pre','mlp_hidden','mlp_output','output']:\n    print(name, trace[name].shape)\n    print(trace[name])\n",
   "id": "ea92ff516363465aa3110663a036d170",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "eeb028c651044afdb18428ee21ec5c1e",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-15"
   },
   "source": "**保存输出 · 2026-09-18 本地逐单元计算，非 Deepnote 云端运行**\n\n```text\nln1 (4, 4)\n[[ 0.3238 -0.6836 -1.1153  1.475 ]\n [ 0.7778  1.0789 -1.4301 -0.4265]\n [ 0.4287 -1.3932  1.3218 -0.3572]\n [ 0.2969 -1.6465  1.0527  0.2969]]\nQ (4, 4)\n[[-0.2322  0.3838 -0.0612 -0.4661]\n [ 0.1665 -0.302  -0.4704  0.13  ]\n [-0.9418  0.9437  0.6336  0.2455]\n [-0.9088  0.9923  0.6176  0.0188]]\nK (4, 4)\n[[ 1.4217 -0.5779  0.3854  1.3568]\n [ 1.1773 -0.5467 -0.4739  0.3812]\n [-1.6563  0.4223  0.7644 -0.8131]\n [-1.1733  0.281   0.8805 -0.3077]]\nV (4, 4)\n[[ 0.0894 -0.4812  0.5537 -0.7136]\n [ 0.5686 -0.0902 -0.7087 -0.0763]\n [-0.6207  0.1009  1.0437  0.0813]\n [-0.6192 -0.0587  1.2253 -0.1574]]\nscores (2, 4, 4)\n[[[-0.3903 -0.3417  0.3866  0.2689]\n  [ 0.2908  0.2553 -0.2851 -0.1981]\n  [-1.3324 -1.1489  1.3848  0.9688]\n  [-1.3192 -1.1402  1.3607  0.9512]]\n\n [[-0.4638 -0.1051  0.2349  0.0633]\n  [-0.0034  0.1927 -0.329  -0.3212]\n  [ 0.4082 -0.1462  0.2013  0.3411]\n  [ 0.1864 -0.2019  0.323   0.3804]]]\nconcat (4, 4)\n[[-0.3022 -0.0763  0.6052 -0.1509]\n [-0.0288 -0.1743  0.3646 -0.2352]\n [-0.5419  0.0128  0.6444 -0.2524]\n [-0.5396  0.0118  0.6827 -0.2138]]\nprojected_attention (4, 4)\n[[-0.3659  0.2188 -0.0172  0.1302]\n [-0.156   0.0641 -0.0012 -0.0502]\n [-0.4727  0.3029  0.0235  0.3042]\n [-0.4889  0.3132  0.011   0.2995]]\nresidual1 (4, 4)\n[[-0.1559  0.3588  0.0928  0.4202]\n [ 0.754   1.0041  0.6888  0.7398]\n [ 0.4873  0.7529  1.2335  1.0442]\n [-0.2689  0.3532  0.301   0.5195]]\nln2 (4, 4)\n[[-1.461   0.7845 -0.376   1.0525]\n [-0.3492  1.6971 -0.8825 -0.4654]\n [-1.3815 -0.4459  1.2471  0.5803]\n [-1.6668  0.4275  0.2518  0.9874]]\nmlp_hidden_pre (4, 6)\n[[ 0.939  -0.0917 -0.3395  1.4627  0.3768  0.1692]\n [-1.4072 -0.6563 -1.5472  0.7997  0.2691  0.0457]\n [ 1.6753  0.6078  0.3688  1.1938 -0.3456  0.0579]\n [ 1.3559  0.1787 -0.1395  1.6026  0.1249  0.1478]]\nmlp_hidden (4, 6)\n[[ 0.7756 -0.0425 -0.1246  1.3575  0.2437  0.0959]\n [-0.1124 -0.1679 -0.0944  0.6302  0.1631  0.0237]\n [ 1.5966  0.4426  0.2374  1.0547 -0.1261  0.0303]\n [ 1.2369  0.102  -0.062   1.5151  0.0687  0.0826]]\nmlp_output (4, 4)\n[[-0.4085  0.0944 -0.0423 -0.5597]\n [-0.1036 -0.0772 -0.0487 -0.4078]\n [-0.5349  0.5596  0.0884 -0.1204]\n [-0.5212  0.3077 -0.0582 -0.493 ]]\noutput (4, 4)\n[[-0.5644  0.4532  0.0506 -0.1394]\n [ 0.6503  0.9269  0.6401  0.332 ]\n [-0.0476  1.3125  1.3218  0.9238]\n [-0.7901  0.6609  0.2428  0.0266]]\n```",
   "id": "eeb028c651044afdb18428ee21ec5c1e"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "d852d37de7974f73a2ae21110073e852",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-16"
   },
   "source": "## 3. 从一行权重核对到两条残差",
   "id": "d852d37de7974f73a2ae21110073e852"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "c490419b16dd40d7bf665b8c42846e3d",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-17"
   },
   "source": "选择 `head=0, query=0`：用该行的 4 个权重手工加权 4 个 V。下面同时做一个显式循环版本与向量化版本的独立核对，并检查两条残差。权重和为 1，**不意味着输出是输入像素的加权平均**：加权对象是学习投影 V，之后还有输出投影。",
   "id": "c490419b16dd40d7bf665b8c42846e3d"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "428a35de2dbe427e9b6b6620e088aed5",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-18"
   },
   "source": "h, q = 0, 0\nmanual_read = sum(trace['attention'][h,q,k]*trace['V_heads'][h,k] for k in range(4))\nassert np.allclose(manual_read, trace['read_values'][h,q], atol=1e-12)\nassert np.allclose(trace['attention'].sum(axis=-1), 1)\nassert np.allclose(trace['residual1']-X, trace['projected_attention'])\nassert np.allclose(trace['output']-trace['residual1'], trace['mlp_output'])\nassert all(np.isfinite(v).all() for v in trace.values())\nprint('Manual readback:', manual_read)\nprint('Vectorized readback:', trace['read_values'][h,q])\nprint('Softmax row sums:', trace['attention'].sum(axis=-1))\nfig, axes = plt.subplots(1, 3, figsize=(12, 3.4))\nfor h in range(HEADS):\n    im=axes[h].imshow(trace['attention'][h], vmin=0, vmax=1, cmap='Blues')\n    axes[h].set(title=f'Head {h}: all-to-all read', xlabel='Read from key', ylabel='Query')\n    for i in range(4):\n        for j in range(4): axes[h].text(j,i,f'{trace[\"attention\"][h,i,j]:.2f}',ha='center',va='center')\nfig.colorbar(im, ax=list(axes[:2]), fraction=.025)\naxes[2].plot(np.linalg.norm(trace['projected_attention'],axis=1),'o-',label='Attention branch')\naxes[2].plot(np.linalg.norm(trace['mlp_output'],axis=1),'s--',label='MLP branch')\naxes[2].set(xlabel='Token',ylabel='Branch output L2 norm',title='Two updates to a residual path')\naxes[2].legend(fontsize=9); fig.subplots_adjust(wspace=.5)\nfig.savefig(OUTPUT/'02-readback.png',bbox_inches='tight'); plt.show()\n",
   "id": "428a35de2dbe427e9b6b6620e088aed5",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "577970e5a803411eac733d98e37534eb",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-19"
   },
   "source": "**保存输出 · 2026-09-18 本地逐单元计算，非 Deepnote 云端运行**\n\n```text\nManual readback: [-0.3022 -0.0763]\nVectorized readback: [-0.3022 -0.0763]\nSoftmax row sums: [[1. 1. 1. 1.]\n [1. 1. 1. 1.]]\n```\n\n![本单元实际计算保存图](https://codingai-lec02.pages.dev/course/images/LAB11-B/figure-02.png)",
   "id": "577970e5a803411eac733d98e37534eb"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "a7533993531a49a7ac6150344dc3b99e",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-20"
   },
   "source": "在这个计算里，每个 query 都可读取 4 个 key，因此没有语言自回归的“只能看过去”。注意力权重描述一次信息混合，不是因果解释，更不是物体重要性真值。",
   "id": "a7533993531a49a7ac6150344dc3b99e"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "75434dbe01404f1899f2c824ba464c65",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-21"
   },
   "source": "## 4. 一次改一个条件",
   "id": "75434dbe01404f1899f2c824ba464c65"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "16a24d60467f4ec98c649cd51a07f76d",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-22"
   },
   "source": "- **像素条件**：只把右下 patch 的最后一个像素增加 0.4；位置与全部权重不变。\n- **位置条件**：保持像素不变，只移除全部位置向量。\n- **完全相同 token 对照**：4 个 patch 内容完全相同且不加位置。预测四行 attention 和四个输出会怎样。\n- **排列对照**：不加位置时仅排列 token，预期输出按同样方式排列。这检验实现的置换等变关系，不检验真实视觉模型的识别能力。\n\n这些是四个独立对照，不应合成一次“同时改变多项”的实验。",
   "id": "16a24d60467f4ec98c649cd51a07f76d"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "3c10693e4154421db17734a2946c22d7",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-23"
   },
   "source": "changed_image = image.copy(); changed_image[3,3] += .4\nX_pixel = patchify(changed_image) @ W_patch + position\npixel_trace = encoder(X_pixel)\nno_position = encoder(embedded)\nidentical_input = np.repeat(embedded[:1], 4, axis=0)\nidentical = encoder(identical_input)\npermutation = np.array([2,0,3,1])\npermuted = encoder(embedded[permutation])\nassert np.allclose(identical['attention'], .25)\nassert np.allclose(identical['output'], identical['output'][0])\nassert np.allclose(permuted['output'], no_position['output'][permutation])\nchanges = {\n    'one_pixel': {k: np.linalg.norm(pixel_trace[k]-trace[k],axis=-1).tolist() for k in ['input','projected_attention','residual1','mlp_output','output']},\n    'remove_position': np.linalg.norm(no_position['output']-trace['output'],axis=-1).tolist(),\n    'identical_attention_max_deviation_from_uniform': float(np.abs(identical['attention']-.25).max()),\n    'permutation_max_error': float(np.abs(permuted['output']-no_position['output'][permutation]).max())}\nprint(json.dumps(changes,indent=2))\nfig, ax=plt.subplots(figsize=(8,3.8))\nfor name, style in [('input','o--'),('projected_attention','s-'),('output','^-')]:\n    ax.plot(range(4), changes['one_pixel'][name], style, label=name)\nax.set(xlabel='Token index',ylabel='L2 change',title='Change one pixel in patch 3; all weights fixed',xticks=range(4))\nax.legend(); fig.tight_layout(); fig.savefig(OUTPUT/'03-change-one-patch.png'); plt.show()\n",
   "id": "3c10693e4154421db17734a2946c22d7",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "fbf2b7c5a7a7477b96461531c0f9693f",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-24"
   },
   "source": "**保存输出 · 2026-09-18 本地逐单元计算，非 Deepnote 云端运行**\n\n```text\n{\n  \"one_pixel\": {\n    \"input\": [\n      0.0,\n      0.0,\n      0.0,\n      0.36441734316577207\n    ],\n    \"projected_attention\": [\n      0.13845143692262174,\n      0.06075535944896897,\n      0.10211094716003967,\n      0.3623462524857455\n    ],\n    \"residual1\": [\n      0.13845143692262177,\n      0.06075535944896893,\n      0.10211094716003964,\n      0.241598612392958\n    ],\n    \"mlp_output\": [\n      0.005507455494816578,\n      0.1869619578853635,\n      0.1485668531107396,\n      0.368001638977608\n    ],\n    \"output\": [\n      0.13750777740178868,\n      0.17868250228813706,\n      0.18240298330694946,\n      0.5225156880808108\n    ]\n  },\n  \"remove_position\": [\n    0.4607686836719755,\n    0.5561112107719021,\n    0.30748062645628643,\n    0.24410837570873786\n  ],\n  \"identical_attention_max_deviation_from_uniform\": 0.0,\n  \"permutation_max_error\": 2.220446049250313e-16\n}\n```\n\n![本单元实际计算保存图](https://codingai-lec02.pages.dev/course/images/LAB11-B/figure-03.png)",
   "id": "fbf2b7c5a7a7477b96461531c0f9693f"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "95c8a859f1444dc6acb5b8b8339e14c0",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-25"
   },
   "source": "**解释结果时分开两步：** 残差直接把像素改动带到第 3 个 token；attention 让其他位置的读回也可能改变。逐 token MLP 自身不跨位置读数据，但它的输入已经含有 attention 交换的信息。因此不能看到多位置变化就说“MLP 自己做了空间通信”。\n\n相同 token 对照给出均匀权重，是这个对称输入与固定模型的数学结果；并不是“没有位置编码就永远只能平均”。不相同的图块仍可产生不同的 Q、K。",
   "id": "95c8a859f1444dc6acb5b8b8339e14c0"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "65e4e51e01534efd843f0b8c510666c3",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-26"
   },
   "source": "## 5. 保留可核查结果，再提出你的反例",
   "id": "65e4e51e01534efd843f0b8c510666c3"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "f519a75da4be4854809784537436aede",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-27"
   },
   "source": "完整中间张量保存为 JSON 和 NPZ；CSV 每行对应一个 token 的变化。学生可以只改 `image[3,3]`、`position` 或某一个投影矩阵，然后运行全本。先写出预测、保持不动的条件，再观察与解释。\n\n研究卡：如果输出变化很小，可能是 attention 变化小，也可能是输出投影抵消，或残差占比大。用哪些中间张量区分这些解释？怎样构造 attention 大幅变化而最终输出几乎不变的反例？不要把最终热图颜色当作答案。",
   "id": "f519a75da4be4854809784537436aede"
  },
  {
   "cell_type": "code",
   "metadata": {
    "deepnote": {
     "block_id": "0335500634d64a01af996b9299467921",
     "type": "code",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-28"
   },
   "source": "import csv\nartifact = {'kind':'exact_constructed_calculation_not_trained_model', 'seed':SEED,\n            'environment':{'python':sys.version,'numpy':np.__version__,'platform':platform.platform()},\n            'structure':{'norm':'pre-norm','heads':HEADS,'d_model':D,'d_head':DH,'d_ff':DFF,'epsilon':EPS,\n                         'mask':'none','dropout':0,'gelu':'tanh approximation','class_token':False},\n            'image':image.tolist(),'patches':patches.tolist(),'position':position.tolist(),\n            'patch_projection':W_patch.tolist(),'weights':{k:v.tolist() for k,v in weights.items()},\n            'trace':{k:v.tolist() for k,v in trace.items()},'changes':changes,\n            'checks':{'manual_readback':True,'softmax_rows_sum_one':True,'residuals':True,\n                      'identical_tokens_uniform':True,'no_position_permutation_equivariance':True}}\n(OUTPUT/'results.json').write_text(json.dumps(artifact,indent=2),encoding='utf-8')\nnp.savez_compressed(OUTPUT/'tensors.npz', **trace, **weights, image=image, position=position, W_patch=W_patch)\nwith (OUTPUT/'token-changes.csv').open('w',newline='') as f:\n    w=csv.writer(f); w.writerow(['token','input_l2_change','attention_branch_l2_change','output_l2_change'])\n    for i in range(4): w.writerow([i]+[changes['one_pixel'][k][i] for k in ['input','projected_attention','output']])\nprint('Saved:', sorted(p.name for p in OUTPUT.iterdir()))\n",
   "id": "0335500634d64a01af996b9299467921",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "7a39197da39241a79679b8e74a73afad",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-29"
   },
   "source": "**保存输出 · 2026-09-18 本地逐单元计算，非 Deepnote 云端运行**\n\n```text\nSaved: ['01-patches.png', '02-readback.png', '03-change-one-patch.png', 'results.json', 'tensors.npz', 'token-changes.csv']\n```",
   "id": "7a39197da39241a79679b8e74a73afad"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "9c64f0992be344e6a4230178c01a74cf",
     "type": "text-cell-h2",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-30"
   },
   "source": "## 来源与验证范围",
   "id": "9c64f0992be344e6a4230178c01a74cf"
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deepnote": {
     "block_id": "af6d77f9c1fa42549cf524e899dc8c7b",
     "type": "markdown",
     "source_block_id": null,
     "logical_key": null
    },
    "html_anchor": "cell-31"
   },
   "source": [
    "来源：[ViT 作者稿 v2，§3.1 / Eqs.2–3](https://arxiv.org/abs/2010.11929v2)，用于图块投影、位置与 pre-norm encoder 的结构约定。参考图中的 scalar q/k/v 与本册双头完整层是两组计算例，不共享权重。\n",
    "\n",
    "保存图与张量来自 2026-09-18 的本地逐单元计算，已核对手工读回、残差、相同 token 和排列对照；没有预训练或语义识别。页面上的保存输出可直接阅读，在自己的 Deepnote 副本运行时以实际新输出为准。\n",
    "\n",
    "[返回第二讲 · 后续 Attention 选读](https://codingai-lec02.pages.dev/course/reference.html#lecture05) · [网页阅读版与保存结果](https://codingai-lec02.pages.dev/course/lab11.html) · [下载 Lecture02-LAB09.ipynb](https://codingai-lec02.pages.dev/course/materials/Lecture02-LAB09.ipynb) · [课程导读](https://codingai-lec02.pages.dev/course/guide.html)"
   ],
   "id": "af6d77f9c1fa42549cf524e899dc8c7b"
  }
 ]
}