一句话定位:该研究探讨当目标物体在训练中始终被其他物体部分遮挡时,腹侧视觉通路模型能否建立跨视角不变性表征。作者利用分层神经网络 VisNet 模拟六个 3D 物体的共同旋转(中央目标 Jaimoid 始终被五个遮挡物部分遮盖),表明仅基于标准 Hebb 学习规则的连续变换学习(CT learning),即可将目标物体的不同可见局部视图绑定为视角不变的整体表征。
研究背景
灵长类腹侧视觉通路在层级加工过程中逐步形成对物体视角、尺寸及位置具有容忍度(不变性)的神经元响应,下颞叶皮层已有相应实验证据(如 Booth & Rolls 1998 记录的视角不变细胞、Tovee 等 1994 记录的平移不变面孔细胞)。在自然场景中,物体通常伴随背景及其他物体出现:既往计算模型表明,VisNet 能够在多物体同时呈现、甚至缺乏孤立呈现训练样本的情况下,依靠物体内部特征共现频率高于物体间共现频率的统计独立性,分别形成对各个物体的特异性表征(Stringer et al., 2007; Stringer & Rolls, 2008);也可识别叠加在复杂背景上的已知物体(Stringer & Rolls, 2000)。但如果训练阶段物体直接嵌入静态杂乱背景中,模型往往难以有效分离。
部分遮挡带来了进一步的计算挑战:当目标物体在训练过程中始终处于局部遮挡状态时,视觉系统一方面需要将目标与遮挡物进行空间分离,另一方面需要将不同时间段内露出的不同局部视图整合为统一的不变性表征。Stringer 等(2007)曾提出结合连续变换学习(Continuous Transformation, CT learning)与统计独立性可能解决该问题,但此前缺乏具体的仿真验证。本研究旨在通过具有光照阴影、以 1° 步长连续旋转的三维物体序列,对该机制进行定量检验。
研究思路
模型依赖两项协同机制。其一是统计独立性:训练时物体以不同的配对组合呈现(目标物 Jaimoid 依次与五个遮挡物配对旋转,五个遮挡物之间也两两配对旋转)。由于同一物体内部特征在旋转中始终保持完全共现,而两两物体之间的特征组合仅为部分共现,竞争网络的局部抑制机制倾向于将高频共现的特征集合分配给特定的输出神经元,形成针对单个物体的独立表征而非场景组合表征。其二是连续变换学习(CT learning):相邻视角之间存在输入模式的空间重叠,Hebb 学习规则使同一输出神经元随着物体旋转逐步关联新的输入连接,最终响应于该物体的大多数或全部视角,无需引入时间迹线(trace)机制。
作者对比了连续变换学习(利用视角间的空间相似性)与时间迹线学习(利用时间连续性,Földiák 1991)的差异,本实验专门测试 CT 学习。由于目标物体在训练序列中从未以完整形态呈现,网络必须关联先后出现的局部碎片视图。为验证输出神经元的表征特性,研究设置了三项测试:单独旋转测试(验证视角不变性)、同位置呈现新物体测试(排除仅对空间位置响应的可能)以及碎片局部视图测试(检验是否依赖特定关键局部标志)。
方法
仿真采用 VisNet 架构:包含四层二维神经元阵列(每层 32×32),模拟对应 V2、V4、后下颞叶(TEO)与前下颞叶(TE)皮层;前馈投射来自前一层局域、按高斯概率采样(Table 1:各层会聚半径分别为 6、6、9、12,突触数分别为 272、100、100、100,输入视网膜尺寸 128×128×32)。输入层采用模拟 V1 简单细胞的双高斯差分(DOG)滤波器组,覆盖不同空间频率(0.0625–0.5 周期/像素,4 个倍频程)、朝向(0°–135°,步长 45°)与正负极性(Table 2 给出各频段连接数)。层内侧向交互包含两步:首先通过空间高斯滤波器实现侧向抑制(Table 3:σ = 1.38/2.7/4.0/6.0,δ = 1.5/1.5/1.6/1.4),随后利用 Sigmoid 函数进行对比度增强,并依据设定百分位数阈值控制放电稀疏度(Table 4:百分位阈值 95/95/88/91,斜率 β = 190/40/75/26)。突触权重调整采用标准 Hebb 规则 δwij = α·yi·xj,每步训练结束对突触权重向量归一化以防止饱和。
视觉刺激为 Swift 3D 软件渲染的 6 个三维模型(包含环境光与单一漫反射光源),均绕竖直轴以 1° 步长旋转 360° 生成 360 帧序列。五个遮挡物为圆锥体、立方体、圆柱体、星形体与十二面体,被遮挡目标为不规则多面体 Jaimoid。空间排布呈正五边形布局(图 3):Jaimoid 位于中心,五个遮挡物位于顶点并与 Jaimoid 产生局部重合,确保各遮挡物的遮挡程度一致。训练包含五个阶段:Jaimoid 依次与各遮挡物同速同步旋转,随后呈现遮挡物之间的两两配对旋转序列;各层逐层训练 50 个 epoch(第一层学习率 0.109,其余层 0.1)。测试阶段冻结权重:执行单物体独立旋转、同位置新物体(tee、torus、hectoid、irinoid、church、pyramid)呈现,以及孤立碎片局部视图测试;性能评估采用单细胞信息量 I(s,R) 与多细胞信息量 I(S,S′)(基于贝叶斯解码,每物体选取信息量前 5 个细胞,共 30 个),6 个物体的最大理论信息量为 log₂6 ≈ 2.58 bits。
主要结果
- 训练后形成了针对目标物体的排他性视角不变神经元。 在输出层(第 4 层)中,训练前对各刺激随机放电的神经元 (4,17),训练后特异性响应于中央被遮挡物 Jaimoid 在全视角(或至少 80% 视角)的旋转,对其他遮挡物均不响应(图 12);神经元 (19,1) 则在训练后对十二面体的全部 360° 视角呈现不变性响应(图 13)。六个物体分别形成了特异性的输出表征亚群。
- 表征的不变性沿网络层级逐级提升。 记录各层对 Jaimoid 响应的典型神经元显示:第 1 层神经元仅在狭窄视角范围内放电;第 2 层显示出适度视角容忍度;第 3 层已具备较强的视角不变性与物体特异性;第 4 层达到全视角覆盖(图 14),符合层级会聚架构的理论预期。
- 神经元将分离的碎片局部视图整合为统一表征。 使用训练中从未以完整形式出现的各类碎片局部视图单独测试时,神经元 (4,17) 对不同局部视图诱发的放电模式与完整 Jaimoid 一致(图 15),表明该响应不单纯依赖某一固定可见的局部标志物。
- 神经元调谐特异于物体几何形态而非空间位置。 将六个新物体放置于与训练集完全相同的空间位置测试时,神经元 (4,17) 与 (19,1) 仅表现出零星微弱放电,未出现视角不变性响应(图 16、17),单细胞信息量均未达到饱和峰值。
- 群体信息论度量确认了无混淆的身份编码。 第 4 层神经元单细胞信息量分析显示,大量细胞对训练物体达到 2.58 bits 的理论上限(图 18);基于贝叶斯解码的多细胞信息量在训练后超过 2.5 bits,显著高于随机未训练网络(1.10 bits)以及将训练网络应用于新物体时的水平(1.7 bits;图 19)。
图注解读
图 1 · CT 学习的单层示意
原文图注:FIGURE 1 | An illustration of how CT learning functions in a feed-forward one-layer network. Activation of overlapping neurons during the transformation of the object from position to position leads to the activation of the same neuron in the output layer. Connections are strengthened according to a Hebbian learning rule after each presentation of the stimulus.
三步示意图:刺激在位置 1 时输入层 1/2/3 号神经元激活,输出层 8 号细胞经随机连接胜出并被 Hebb 规则增强;刺激移到位置 2(输入变为 2/3/4,与之前重叠两个单元)时,已增强的连接让同一个输出细胞再次胜出,新输入连接随之被加强。反复进行,同一输出细胞便逐渐学会响应变换中的同一物体。

图 2 · 六个训练物体
原文图注:FIGURE 2 | The six objects used to train the network. The objects are 3D objects each shown from 360 different views. The effect of the ambient lighting and single diffuse light source is illustrated. This allows different surfaces to be shown with different intensities. Objects are split into two groups; occluding objects are presented in the top row and the occluded object is presented in the bottom row.
训练刺激概览:上排五个遮挡物、下排被遮挡的 Jaimoid,均为 3D 渲染物体(各 360 个视角),环境光配合单漫射光源使不同表面呈现出亮度差异,具备明暗阴影与表面明暗变化。

图 3 · 五边形布阵示意
原文图注:FIGURE 3 | The pentagon formation specifying the location of the occluding and occluded objects. The occluded object, the Jaimoid, is always presented in the centre of the pentagon. Each of the occluding objects is placed at one of the five points of the pentagon, partially overlapping the Jaimoid at the center. This ensures that the occluding objects are equidistant from the center of the occluded object, helping to maintain a comparable level of partial occlusion for the Jaimoid.
空间排布示意:Jaimoid 位于正五边形中心,五个遮挡物各占一个顶点,与 Jaimoid 部分重叠且与中心等距。这一布局使不同遮挡物对 Jaimoid 的遮挡程度保持可比,控制了遮挡比例这一变量。

图 4 · Jaimoid 单独旋转的测试序列
原文图注:FIGURE 4 | Five example frames selected from the 360 frame testing image sequence of the Jaimoid rotating in depth around the vertical axis through 360° in 1° steps. The selected frames shown are for 0°, 72°, 144°, 218°, and 288°.
测试时单独呈现 Jaimoid 的 360 帧序列中的五个示例帧(0°、72°、144°、218°、288°),展示了该不规则多面体在不同深度旋转视角下的形态变化。

图 5 · VisNet 四层架构与脑区对应
原文图注:FIGURE 5 | Left: Stylised image of the 4 layer network developed by Wallis et al. (1993) and Wallis and Rolls (1997). Convergence through the network is designed to provide fourth layer neurons with information from across the entire input retina. Right: Convergence in the visual system V1, visual cortex area V1, TEO, posterior inferior temporal cortex; TE, inferior temporal cortex (IT).
左图为 VisNet 四层网络架构示意,右图对应大脑中 V1→V2→V4→TEO→TE 的层级会聚,以及感受野随离心度扩大的生理测量(1.3°/3.2°/8.0°/20°/50° 量级),说明会聚设计与腹侧通路的感受野扩大相对应。

图 6 · 输入滤波器采样范式
原文图注:FIGURE 6 | The filter sampling paradigm. Images are first filtered by a difference of Gaussian filter of the appropriate orientation, sign, and frequency. Each square represents the retinal image after filtering and the circles represent the consistent retinotopic coordinates used to provide input to a layer one cell. The orientation tuning, left to right, increases from 0° in steps of 45°, with segregated pairs of positive (P) and negative (N) filter responses. The filters double in spatial frequency toward the reader. For further details, see Rolls and Deco (2002).
每个方格表示图像经特定高斯差分(DOG)滤波器处理后的输出,圆圈标出为第 1 层细胞提供输入的视网膜拓扑坐标。滤波器覆盖不同取向与空间频率,正负极性输出(P/N)分开呈现。

图 7 · Jaimoid 与锥体成对旋转的训练序列
原文图注:FIGURE 7 | Five example frames selected from the 360 frame training image sequence of the Jaimoid and the Cone rotating through 360° in 1° steps. The selected frames shown are for 0°, 72°, 144°, 218°, and 288°. All five of the occluding objects were also presented to the network in the same manner.
训练序列示例:Jaimoid 与锥体成对、各自绕竖直轴同速旋转的五个示例帧。五段配对训练序列均采用这种形式,被遮挡物体始终伴随遮挡物出现。

图 8 · 五个遮挡物各就各位
原文图注:FIGURE 8 | Five example frames of the Jaimoid occluded by all five occluding objects in their five corresponding positions. The occluding objects are arranged around the pentagon formation so that they are equidistant from the center of the Jaimoid. The rotating objects used during training are presented in the same locations: cone, position 1; cube, position 2; cylinder, position 3; star, position 4; dodecahedron, position 5; Jaimoid, position 6.
五个示例帧展示五个遮挡物排布在五边形对应位置(锥体位置 1、立方体位置 2、圆柱位置 3、星形位置 4、十二面体位置 5,Jaimoid 位于中心位置 6)时的场景形态。所有物体在训练和测试中占据这些固定空间位置。

图 9 · 遮挡物之间的互相遮挡
原文图注:FIGURE 9 | Five example frames of two occluding stimuli (cylinder and star) rotating together, demostrating the typical overlap between the occluding objects.
圆柱与星形体成对旋转的五帧示例,展示两个遮挡物之间同样存在部分重叠,表明网络在多重重叠条件下仍需分离不同物体。

图 10 · 五对物体的互遮挡部分视图
原文图注:FIGURE 10 | Mutual overlap: Areas of mutual overlap between the occluding and occluded objects during one example frame of rotation. As the two objects rotate together in lock-step, the area of mutual overlap creates a partial view of the occluded object: (A) Shows the cone and Jaimoid; (B) Cube and Jaimoid; (C) Cylinder and Jaimoid; (D) Star and Jaimoid; (E) Dodecahedron and Jaimoid. Each pair is presented alongside the partial view it creates. VisNet must learn to associate together all of these different partial views of the occluded object to build an exclusively invariant representation.
展示五组配对旋转帧及各自暴露的局部视图:Jaimoid 分别被锥体、立方体、圆柱、星形和十二面体遮挡时露出的部分各不相同。训练过程中 Jaimoid 从未完整呈现,网络需要将各局部视图关联为统一表征。

图 11 · 星形与 Jaimoid 的互遮挡细节
原文图注:FIGURE 11 | Star and Jaimoid mutual overlap: Five example frames of the star and Jaimoid as they rotate together in lock-step. As in Figure 10, the area of mutual overlap creates a partial view of the Jaimoid. This is shown in this specific example where the star and Jaimoid are presented over five equally spaced viewing angles.
图 10 的局部放大:星形体与 Jaimoid 同步旋转经过五个等间隔视角时,互遮挡区域形成的 Jaimoid 部分视图随之变化。相邻视角间的特征空间重叠为连续变换学习提供了连接基础。

图 12 · 细胞 (4,17) 训练前后对六个物体的响应
原文图注:FIGURE 12 | The firing rate responses of cell (4, 17) in the 4th (output) layer of VisNet to the central occluded object (Jaimoid) and the five surrounding occluding objects as they rotated through 360° in 1° steps before and after training. Before training, it can be seen that the cell responds randomly to different views of different objects. After training, it can be seen that the cell's response pattern has changed. This cell responds to all the views of Jaimoid, and to none of the views of the other objects.
十二个子图分别展示细胞在训练前(A)与训练后(B)对六个物体 360 个视角的放电率曲线。训练前放电无明显规律;训练后对 Jaimoid 在全视角范围内持续放电,对其他五个物体均无响应。

图 13 · 细胞 (19,1) 成为十二面体的排他不变细胞
原文图注:FIGURE 13 | The firing rate responses of cell (19, 1) in the fourth (output) layer of VisNet to the central occluded object (Jaimoid) and the five surrounding occluding objects as they rotated through 360° in 1° steps before and after training. Before training it can be seen that the cell responds randomly to different views of different objects. After training it can be seen that the cell's response pattern has changed. This cell has become an exclusive invariant cell for the Dodecahedron. It responds invariantly to all 360 views of the Dodecahedron and to no views of any other object.
与图 12 结构一致,展示细胞 (19,1) 的响应:训练后对十二面体全部 360° 视角呈现不变性响应,对包含 Jaimoid 在内的其他物体均无放电,表明遮挡物本身也建立了排他性表征。

图 14 · 四个层级对 Jaimoid 的响应对比
原文图注:FIGURE 14 | Example Jaimoid response plots for all four layers. A prototypical example cell is presented for each of the 4 layers within the VisNet model. For each cell, its response plot is presented with respect to the example object, the Jaimoid. It can be seen that a typical layer 3 cell is highly transform invariant while a layer 2 cell shows more modest levels of view invariance. Layer 1 cells demonstrate very little view invariance and responded to a very narrow set of views. In all cases, these cells responded exclusively to their preferred object, in this case the Jaimoid, and did not learn to respond to any views of any of the other objects. Comparable responses exist for all six of the objects presented during training.
四个层级典型神经元对 Jaimoid 的响应:第 1 层神经元仅对狭窄视角段放电,第 2 层表现出适度视角容忍度,第 3 层具备较强不变性,第 4 层实现全视角覆盖,体现了不变性沿层级逐级累积的特征。

图 15 · 碎片部分视图测试
原文图注:FIGURE 15 | Partial view response plots for cell 4, 17. Cell response plots are presented after testing the network with the fragmented partial views of the Jaimoid, as exemplified in Figure 10. It can be seen that the example cell 4, 17 that responded invariantly to the complete view of the Jaimoid also responds in an identical manner to the different fragmented partial views of the Jaimoid. Cell 4, 17 has learned to bind together these different partial views into a holistic representation and responds equally well to all of them, thus proving that this cell does not rely on a specific partial view or key-marker in order to recognize the Jaimoid.
五个子图对应不同遮挡条件下生成的 Jaimoid 碎片局部视图,神经元 (4,17) 对各碎片视图的放电响应与面对完整 Jaimoid 时保持一致,表明该识别不单纯依赖某一特定可见局部标志。

图 16 · 细胞 (4,17) 对六个新物体的响应
原文图注:FIGURE 16 | The firing rate responses of cell (4, 17) in the fourth (output) layer of VisNet to six new objects that the network was not trained on, as they rotated through 360° in 1° steps after training. It can be seen that the cell's response pattern has changed compared to its response pattern to the six objects that the network was trained on (the occluded and the five occluding objects; Figure 12B). Whereas when tested with the set of objects that the network was trained on, the cell responds to at least 80% of the views of Jaimoid, and to none of the views of the other objects. When the network is tested on six novel objects presented in the same locations as the trained set of objects, the cell responds very poorly to small portions of view of a number of objects. This shows that the network has learnt to respond to the trained objects in particular, and not just to their locations.
六个新物体(tee、torus、hectoid、irinoid、church、pyramid)置于原训练位置旋转时,神经元 (4,17) 仅对少数视角产生微弱零星放电,未出现不变性响应,排除了仅对空间位置响应的可能性。

图 17 · 细胞 (19,1) 对六个新物体的响应
原文图注:FIGURE 17 | The firing rate responses of cell (19, 1) in the fourth (output) layer of VisNet to six new objects that the network was not trained on, as they rotated through 360° in 1° steps after training. It can be seen that the cell's response pattern has changed compared to its response pattern to the six objects that the network was trained on (the occluded and the five occluding objects; Figure 13B). Whereas when tested with the set of objects that the network was trained on, the cell responds invariantly to all 360 views of the Dodecahedron and to no views of any other object. When tested on the six novel objects presented in the same locations as the trained set of objects, the cell responds very poorly to small portions of view of a number of objects. This shows that the network has learnt to respond to the trained objects in particular and not just to their locations.
神经元 (19,1) 面对同位置新物体时的对照响应:同样仅出现零星微弱放电,进一步支持了选择性针对物体几何特征而非空间位置的结论。

图 18 · 单细胞信息分析
原文图注:FIGURE 18 | Single cell information results obtained when VisNet was tested with the occluded object and five occluding objects rotating through 360° in 1° steps. Single cell information analysis results are also plotted for six novel objects that the network was not previously trained on. These novel objects were rotating in the same positions as the six objects with which the network was originally trained. Results are presented having tested the trained network with the original set of trained objects (unbroken line), after testing the trained network with the novel set of objects (dashed line), and after testing a random untrained network with the original six objects (dotted line). The single cell information measure for all fourth layer neurons ranked in order of their invariance to the objects is shown. It can be seen that training the network on the object pairs has led to many fourth layer neurons attaining the maximum level of single cell information of 2.58 bits for these trained objects. These cells have learned to respond selectively to individual trained objects invariantly over all views. It can also be seen that the novel objects produce less information and no cells reached the maximal information. The random untrained plot provides a baseline comparison.
横轴为按不变性排序的第 4 层细胞秩次,纵轴为单细胞信息量(bits)。实线(训练网络测原物体)显示大量细胞达到 2.58 bits 理论上限;虚线(测新物体)信息量明显更低,无细胞达到上限;点线为随机未训练网络的基线。

图 19 · 多细胞信息分析
原文图注:FIGURE 19 | Multiple cell information results obtained when VisNet was tested with the occluded object and five occluding objects rotating through 360° in 1° steps. Multiple cell information analysis results are also plotted for six novel objects that the network was not previously trained on. These novel objects were rotating in the same positions as the six objects on which the network was originally trained. Results are presented after training the network (unbroken line), after testing the trained network with the novel set of objects (dashed line) and with a random untrained network (dotted line). After the network was trained, over 2.5 bits of information was reached, which was substantially higher than 1.10 bits reached by the untrained network or 1.7 bits reached by the trained network that was tested on the novel set of objects. This confirmed that, after training the network, there were cells that form object specific representations to each one of the six objects and do not respond to the object locations.
横轴为纳入解码的细胞数(2–20 个),纵轴为多细胞信息量。训练网络测原物体(实线)超过 2.5 bits,明显高于未训练网络(1.10 bits)与测新物体(1.7 bits),确认六个物体分别由不同细胞亚群表征。

讨论
作者讨论了计算模型对自然视觉无监督学习的启示:与依赖显式图像分割、模板匹配或固定局部特征的人工视觉算法不同,该层级竞争模型依托统计独立性结合连续变换学习,在目标物体从未单独完整呈现的条件下,实现了目标与遮挡物的分离以及不同局部视图的绑定。对于新物体测试中出现的微弱残余响应,作者归因于竞争网络基于特征相似性产生的固有泛化特性。不变性响应、同位置新物体抑制以及碎片视图响应一致性三重测试,排除了单纯记忆空间位置或依赖固定局部关键标志的解释。
作者指出了实验的局限性与后续拓展方向:首先,受计算资源限制,仿真中仅设置了一个被遮挡物体(Jaimoid),多物体互相遮挡的理论预测仍待进一步仿真验证;目标物体维持正常学习所需的最小可见比例上限尚未定量测定。其次,训练刺激采用相对简单的几何模型以保证实验变量的严格控制,更复杂物体的适用性有待检验。再次,本实验专门测试了基于空间相似性的连续变换(CT)学习,未直接对比基于时间序列连续性的迹线(trace)学习机制。最后,CT 学习依赖物体相对运动诱发的统计解耦;若物体处于完全静止状态且持续重合,纯前馈 Hebb 规则将无法实现特征分离,该极端情形可能需要结合脉冲时序依赖可塑性(STDP)等时间动力学机制。此外,扫视引起的视网膜平移不变性推广仍属理论推测,本研究未包含平移仿真的直接数据。
一句话总结
本研究表明,在目标物体始终处于部分遮挡的条件下,前馈分层神经网络通过物体运动的统计独立性与基于 Hebb 规则的连续变换学习,能够将分离的局部可见视图整合为针对该物体的视角不变性神经表征。