IT / 文献库

PAPER 075 / CLOSE READING

Learning invariant object and spatial view representations in the brain using slow unsupervised learning

语义审核:needs_revision · 图表审核:pass

本文目录 (Table of Contents)
  1. 研究背景
  2. 研究思路
  3. 方法
  4. 主要结果
  5. 图注解读
    1. 图 1 · 视觉系统的逐级汇聚与 VisNet 架构对应
    2. 图 2 · 中间层特征组合神经元的复用
    3. 图 3 · 形变不变识别的旗帜刺激
    4. 图 4 · 非意外性质与度量性质的分离
    5. 图 5 · 层级化 + 注意调控的皮层架构模型
    6. 图 6 · 自然场景中找目标与认目标的联合仿真
    7. 图 7 · 杯子实验:VisNet 能而 HMAX 不能
    8. 图 8 · 连续空间变换(CT)学习原理
    9. 图 9 · 背侧系统的位置不变全局运动
    10. 图 10 · 三级坐标变换:从视网膜坐标到海马空间视角
  6. 讨论
  7. 一句话总结

一句话定位:本文基于猕猴下颞叶皮层(IT)物体细胞与海马空间视角(spatial view)细胞的电生理发现,系统综述了利用环境统计特征的慢速无监督学习(slow unsupervised learning)模型 VisNet,阐明大脑如何通过时间连续性与局部突触可塑性形成对位置、尺寸、视角、形变及光照的不变性表征。

研究背景

视觉识别需要将同一物体在不同位置、尺寸、视角、光照和形变下的视网膜图像映射至稳定表征。电生理记录显示,灵长类 IT 皮层存在具备这些不变特性的物体与面孔选择性神经元,并采用稀疏分布式编码。监督深度卷积网络依靠大规模标注数据、误差反向传播与跨空间权重共享能够拟合类似表征,但反向传播与全局权重复制在神经生物学局部突触机制上缺乏直接对应。自然视觉环境提供了一种统计线索:同一物体的连续视网膜图像在时间序列上高度相邻,慢速无监督学习可利用该时序连续性,无需逐样本的外部类别标签。

空间认知领域存在相似的坐标变换需求。猕猴海马区的空间视角细胞表征“当前注视的外在空间位置”,对眼位、头朝向与自身所在位置具备容忍性,并能在短暂遮蔽后通过自身运动线索持续更新。这类表征需要跨越视网膜、头部及外在世界坐标系。若将时序慢学习原则与感受野增益调制、分层汇聚相结合,理论上可同时解释腹侧通路的物体不变性与背侧–海马通路的空间坐标变换,从而构建统一的皮层计算框架。

本文以 IT 皮层和海马的生理特性为约束,综述了 VisNet 模型中带时间痕迹的学习规则、低阶特征绑定与跨层部件复用机制,分析了输入时间统计对身份或姿态表征形成的影响。文章将该机制扩展至背侧通路的全局运动与多级空间坐标变换,并与 HMAX 及深度卷积网络对比。文章旨在论证慢学习计算原理在皮层中的通用性,属于生物约束下的计算建模探索,其网络缩放能力与真实皮层的映射关系仍需进一步实验验证。

研究思路

研究的核心逻辑是从实验观察中提取神经计算约束,构建具备生物合理性的分层网络模型。真实环境中物体状态随时间连续演变,连续注视期间物体图像仅发生渐进改变。因此,在 Hebb 型突触更新规则中引入短时记忆痕迹(trace),使突触后细胞保留约 100 ms 以上的激活历史,能够将时序相邻的不同视网膜投影绑定为同一物体的表征。该机制基于 Földiák(1991)提出的时序关联假说,并在四层分层网络 VisNet 中拓展至视角、形变及空间视角等复杂不变性任务。

进一步的论证在于验证该计算原理的通用性:保持核心网络架构,仅改变输入信号与统计规律。将静态 Gabor 输入替换为局部光流运动向量,可模拟背侧通路的位置不变全局运动表征;将时间连续性替换为空间重叠连续性,可实现连续空间变换学习(CT 学习);将增益调制(gain modulation)与时间痕迹规则结合,可在背侧–顶叶–海马通路中实现由视网膜坐标到外在空间坐标的多级变换。研究试图表明,分层汇聚结合局部痕迹学习构成了一种可由多个皮层系统复用的基本计算模式。

方法

VisNet 为四层前馈层级竞争网络,映射灵长类腹侧视觉通路:第 1 层对应 V2,第 4 层对应前部 IT(TE 区)。标准网络各层包含 32×32 个神经元,每个神经元从上一层局部高斯连接半径内接收约 200 个前馈突触(第 1 层为 272 个;大尺度版本扩展至每层 256×256 个神经元、1000 个突触)。层间感受野逐级扩大约 2.5 倍,使第 4 层神经元能够整合全视网膜输入(图 1)。输入层由模拟 V1 的 Gabor 滤波器组提供(4 个空间频率 × 4 个朝向)。层内竞争通过稀疏度控制实现:膜电位经点积求和后通过阈值非线性调整,各层表征稀疏度 a 控制在约 0.01 左右。

突触更新采用修饰的 Hebb 痕迹学习规则:第 2–4 层的突触增量为 δwj = α · ȳτ · xj,其中活动痕迹更新为 ȳτ = (1−η)yτ + η · ȳτ−1,痕迹衰减系数 η 常设为 0.8。第 1 层采用无时间痕迹的纯联想 Hebb 规则,以便优先在局部建立准确的特征空间相对位置绑定,防止过早产生跨位置混淆。训练时,同一物体的不同变换帧按随机顺序连续呈现,随后切换至下一物体。痕迹机制的生物学对应包括 NMDA 受体约 100 ms 的通道失活时间常数以及 IT 皮层神经元在刺激撤除后约 1 秒的持续放电。对比基线包括 HMAX(无慢学习机制的预设特征层级)与监督深度卷积网络(基于反向传播与跨空间权重共享)。在空间坐标变换部分,采用 VisNetCT 架构将增益调制信号与时间痕迹学习逐级结合。

主要结果

  1. 不变性表征的电生理特性归纳。文献系统记录了灵长类 IT 面孔与物体细胞对平移、尺寸、对比度、光照、空间频率及视角的容忍性。在复杂自然背景下,IT 神经元感受野从均匀灰背景下的约 70° 收缩至以中央凹为中心的约 8° 半径,表明自然视觉依赖注视点邻域的局部高分辨率处理,多目标位置则通过中央凹与旁中央凹响应的不对称性进行编码;新异刺激呈现初期诱发反应改变,而熟悉刺激保持稳定响应。在海马区,空间视角细胞以空间外在坐标编码“视线所向目标位置”,对眼位、头朝向与自身站位具备不变性,并在全黑或视线被遮挡条件下依靠自身运动线索持续更新数分钟,随后因路径积分误差累积逐渐漂移。
  2. VisNet 实现多种变换不变性且具有特征组合能力。实验验证了层级特征绑定机制:线段的不同几何组合能够在中间层形成独立的特征神经元,这些中间层单元可作为组合构件被顶层多个不同物体复用(图 2),为网络提供高容量基础。训练后的网络在杂乱背景与部分遮挡条件下表现出识别鲁棒性,且在连续训练序列中插入少量其他物体干扰帧时,仍能维持不变性学习。
  3. 输入时间统计决定所学表征类型。在旗帜动态形变实验中(图 3),当训练序列将同一形变/姿态下的不同个体在时间上连续呈现时,网络自发形成姿态特异、跨个体与视角不变的表征;当训练序列将同一个体的连续形变在时间上连续呈现时,网络形成个体特异、跨形变与姿态不变的表征。该结果表明相同的分层网络根据输入时序统计可自组织出不同功能的特征空间。在非意外属性(NAP)实验中(图 4),痕迹规则使网络对随视角连续变化的度量属性(如曲率程度)产生泛化,同时对非意外属性(凹、凸、直边缘)保持特异调谐。
  4. 与 HMAX 及监督深度网络的性能对照。在双杯视角识别任务中(图 7),两只杯子各提供 4 个视角。VisNet 第 4 层形成了分别对应两只杯子视角不变表征的神经元,正确率达到 100%;HMAX 无法分离两只杯子的身份,反而对表面文字产生非特异性强响应,且其末层表征分布过度离散。此外,VisNet 与 IT 神经元类似,对打乱五官的打散面孔响应显著降低,而 HMAX 对打散刺激仍产生强响应。在 50 种真实物体、每种 9 个视角(间隔 40°)的大尺度测试中,VisNet 对训练视角的测试正确率为 90%(随机基线 2%),对间隔 20° 的内插未见视角正确率为 68%。
  5. 背侧与空间系统复用相同计算原则。在全局运动实验中,将输入替换为局部光流运动矢量后,VisNet 第 3 层形成了位置不变的全局旋转运动检测神经元(图 9),解决了局部小感受野孔径问题引起的旋转方向模糊,并复现了目标中心运动(如头部旋转表征不受整体朝向颠倒影响)。在多级空间坐标变换中(图 10),视网膜坐标结合眼位增益调制形成头中心坐标(LIP/VIP);进一步结合头朝向调制形成指向地标的外在方位坐标(7a 区与后扣带回);最后结合个体站位调制形成外在空间视角坐标。各级增益调制与痕迹学习的结合抑制了多级串联中的误差发散,解释了海马空间视角细胞在短时视觉遮蔽下利用自身运动维持定向的神经基础。

图注解读

图 1 · 视觉系统的逐级汇聚与 VisNet 架构对应

原文图注:FIGURE 1 | Convergence in the visual system. (Right) Convergence in the ventral stream cortical hierarchy for object recognition. LGN, lateral geniculate nucleus; V1, visual cortex area V1; TEO, posterior inferior temporal cortex; TE, anterior inferior temporal cortex (IT). (Left) Convergence as implemented in VisNet, the model of invariant visual object recognition described here. Convergence through the hierarchical feedforward network is designed to provide Layer 4 neurons with information from across the entire input retina, by providing an increase of receptive field size of 2.5 times at each stage. Layer 1 of the VisNet model corresponds to V2 in the brain, and Layer 4 to the anterior inferior temporal visual cortex (TE). In this paper 'Layer' with a capital L indicates a Layer of a neuronal network which may correspond to a brain region as here. This is distinct from the 6 architectonic layers in neocortex, designated here with a small letter l in 'layer'.

右侧展示灵长类腹侧通路(LGN→V1→TEO→TE)感受野逐级汇聚结构;左侧为 VisNet 四层前馈模型。网络逐层将感受野扩大 2.5 倍,使第 4 层神经元能够覆盖完整视网膜输入。图中标注了功能区域映射:Layer 1 对应 V2,Layer 4 对应前部 IT(TE 区),大写“Layer”特指计算网络层,与大脑新皮层六层细胞构筑(小写 layer)区分。该结构通过有限层级汇聚提供大感受野基础。

Figure 1

图 2 · 中间层特征组合神经元的复用

原文图注:FIGURE 2 | Encoding of information in intermediate Layers of VisNet. The 13 stimuli used to investigate independent coding of different feature combinations by different neurons in intermediate Layers of VisNet. Each of the 13 stimuli was a different feature, or feature combination with adjacent features, that was learned to be a different object by VisNet, demonstrating that VisNet can learn to represent objects as different even when they have overlapping features. Moreover, these feature combination neurons could be used by further combination in higher Layers of VisNet to represented more complex objects. (After Rolls and Mills, 2018).

展示用于检验特征组合编码的 13 种线段与子结构刺激。VisNet 将包含重叠边缘特征的各个子结构分别编码为不同的激活模式,表明中间层能够表征特定的特征组合而非退化为离散无序特征。中间层组合神经元进一步作为更高层的输入组件构建复杂物体表征,展示了分层特征复用对提升网络表征容量的作用。

Figure 2

图 3 · 形变不变识别的旗帜刺激

原文图注:FIGURE 3 | Deformation-invariant object recognition. The flag stimuli used to train VisNet to demonstrate deformation-invariant object recognition. Each flag is shown with different wind forces and rotations. Starting on the left with the first pair of images for each flag, both the 0 and 180°views are shown for a windspeed of 0; and each successive pair is shown for the wind force increased by 50 Blender units. Visnet learned to categorize these 4 flags as 4 different flags provided that the different deformations of each flag were shown close together in the temporal sequence during training, to make use of the trace learning rule. (After Webb and Rolls, 2014).

测试刺激包含 4 面不同图案的旗帜,分别在 0° 和 180° 视角下展示,风力参数以 50 Blender 单位递增导致明显的非刚性几何形变。当训练序列按时间连续呈现同一面旗帜的不同形变样本时,痕迹规则促使网络成功将同一种旗帜归为同一类别,证明时序连续性足以驱动非刚性形变不变性的自组织学习。

Figure 3

图 4 · 非意外性质与度量性质的分离

原文图注:FIGURE 4 | Learning non-accidental properties of objects. The stimuli used to investigate non-accidental properties (NAP) vs. metric properties (MP) of encoding in VisNet. Each object is shown as white on a gray background. Objects 1–3 all have the non-accidental property of concave edges. Objects 1–3 are different in their metric properties, the amount of curvature. Object 4 has the non-accidental property of parallel edges. Objects 5–7 have the non-accidental property of convex edges, and different metric properties from each other, the amount of the convexity. The vertical view of each object was at 0°of tilt, with the images at –6 and 6°of tilt illustrated. Different amounts of tilt of the top toward or away from the viewer are shown at the tilt angles indicated. Each object was thin, and was cut off near at the top and bottom of each object to ensure that any view of the top or bottom of the object did not appear, so that the type of curvature of the edges (concave, straight, or convex) was the main cue available.

7 个柱状物体分为三组:1–3 为不同曲率的凹缘物体,4 为平行直边物体,5–7 为不同曲率的凸缘物体。图像上下两端被裁剪,以倾角变化(−6°、0°、6° 等)呈现。物体在倾斜时曲率绝对数值发生连续变化(度量属性 MP),但边缘凹凸走向恒定(非意外属性 NAP)。VisNet 表现出对连续度量变化的泛化,同时保持对非意外属性分类的敏感性。

Figure 4

图 5 · 层级化 + 注意调控的皮层架构模型

原文图注:FIGURE 5 | Cortical architecture for hierarchical and attention-based visual perception. The system has six modules organized so that they resemble the ventral visual stream (Left) and dorsal visual stream (Right) of the primate visual system. Information from the lateral geniculate (LGN) enters V1. The ventral visual stream leads through V2–V4 to the inferior temporal visual cortex (IT), and is mainly concerned with object recognition. The dorsal visual stream leads via areas such as MT into the posterior parietal cortex (PP), and is involved in this model in maintaining a spatial map of an object's location. The solid lines with arrows between levels show the forward connections, and the dashed lines the top-down backprojections. Short-term memory systems in the prefrontal cortex (PF46) apply top-down attentional bias to the object (from PFv) or spatial processing (from OFd) streams. (After Deco and Rolls, 2004).

计算系统整合了腹侧识别通路(LGN→V1→V2–V4→IT)与背侧空间通路(经 MT 至后顶叶 PP)。实线表示前馈投射,虚线表示自上而下的反馈投射。前额叶皮层(PFv 针对物体,OFd 针对空间)提供自上而下的注意偏置信号。该架构模拟了自然复杂场景中注意偏置对 IT 神经元感受野的动态调制效应。

Figure 5

图 6 · 自然场景中找目标与认目标的联合仿真

原文图注:FIGURE 6 | Finding and recognizing objects in natural scenes. (A) Eight of the twelve test scenes. Each scene has four objects, each shown in one of its 4 views. (B) The bottom-up saliency map generated by the GBVS code for one of the scenes. The highest levels in the saliency map are red, and the lowest blue. (C) Rectangles (384 pixels × 384 pixels) placed around each saliency peak in the scene for which the bottom-up saliency map is illustrated in (B). (After Rolls and Webb, 2014).

多阶段识别仿真流程:(A) 包含多个物体的复杂自然场景图像;(B) 基于图论视觉显著性算法(GBVS)生成的自下而上显著性图,红色区域表示高显著性;(C) 在各显著性峰值处截取 384×384 像素的局部注视窗口输入 VisNet 进行识别。系统在目标未精确居中时保持约 90% 正确率,并能泛化至未训练的内插视角,说明识别系统具备局部平移与视角容忍能力。

Figure 6

图 7 · 杯子实验:VisNet 能而 HMAX 不能

原文图注:FIGURE 7 | View invariant representations by VisNet but not by HMAX. The two objects, cups, each with four views. HMAX of Riesenhuber and Poggio (1999) fails to categorize these objects correctly, because, unlike VisNet, it has no slow learning mechanism to associate together different views of the same object. (After Robinson and Rolls, 2015).

实验选取两只结构相近但表面带有不同字符的杯子,各包含 4 个旋转视角。VisNet 依赖时间痕迹规则将同一物体的不同视角输入绑定,在第 4 层形成了独立的视角不变神经表征;HMAX 缺乏时序关联学习机制,主要响应局部图像高频纹理与文字特征,未能形成稳定区分两个物体身份的表征。

Figure 7

图 8 · 连续空间变换(CT)学习原理

原文图注:FIGURE 8 | Continuous spatial transformation learning of transform-invariant visual representations of objects. This illustrates how continuous spatial transformation (CT) learning would operate in a network with forward synaptic connections between an input Layer of neurons and an output Layer. Initially the forward synaptic connection weights are set to random values. (A) The presentation of a stimulus to the network in position 1. Activation from the active (shaded black) input neurons is transmitted through the initially random forward connections to activate the neurons in the output Layer. The neuron shaded black in the output Layer wins the competition in the output Layer. The synaptic weights from the active input neurons to the active output neuron are then strengthened using an associative synaptic learning rule. (B) The situation after the stimulus is shifted by a small amount to a new partially overlapping position 2. Because some of the active input neurons are the same as those that were active when the stimulus was presented in position 1, the same output neuron is driven by these previously strengthened synaptic afferents to win the competition. The rightmost input neuron shown in black is activated by the stimulus in position 2, and was inactive when the stimulus was in position 1, now has its synaptic connection to the active output neuron strengthened (denoted by the dashed line). Thus the same neuron in the output Layer has learned to respond to the two input patterns that have vector elements that overlap. The process can be continued for subsequent shifts, provided that a sufficient proportion of input neurons is activated by each new shift to activate the same output neuron. (After Stringer et al., 2006).

连续空间变换(CT)学习机制示意:(A) 刺激处于位置 1,输入神经元通过前馈突触激活竞争获胜的输出神经元,突触发生 Hebb 增强;(B) 刺激小幅平移至位置 2,由于激活的输入模式部分重叠,先前强化的突触促使同一输出神经元再次获胜,新激活的输入单元突触被联想强化(虚线所示)。该机制表明,在空间位移具有连续重叠的前提下,无需依赖时间痕迹亦可形成连续平移不变性。

Figure 8

图 9 · 背侧系统的位置不变全局运动

原文图注:FIGURE 9 | Invariant object-based global motion in the dorsal visual system. This shows two wheels at different locations in the visual field rotating in the same direction. One rotating wheel is presented at a time, and a representation is needed in the case illustrated that the rotating flow field produced by the wheel in either location is always clockwise. The local flow field in V1 and V2 is ambiguous about the direction of rotation of the two wheels, because of the small receptive field size. Rotation that is clockwise or counterclockwise is can only be identified by a global flow computation, with larger receptive fields. The diagram shows how a network with stages like those found in the brain can solve the problem to produce position invariant global motion-sensitive neurons by Layer 3. The computation involved is convergence from stage to stage as illustrated, combined with a short-term memory trace synaptic learning rule to help the network learn that it is the same wheel rotating in the same direction as it moves across the visual field during training (during development). This is the computational architecture of VisNet. It was demonstrated that VisNet can learn translation invariant representations of these types of object-based motion, by substituting the normal Gabor filters as the input neurons in the input Layer corresponding to V1 with local optic flow motion neurons also present in V1. (After Rolls and Stringer, 2006b).

两个顺时针旋转的轮子呈现在不同空间位置。V1/V2 局部运动敏感细胞由于感受野过小无法确定整体旋转方向。通过分层汇聚结合时间痕迹突触规则,并在输入层采用局部光流检测单元,网络在第 3 层产生了对旋转方向敏感且具备空间平移不变性的全局运动表征单元。

Figure 9

图 10 · 三级坐标变换:从视网膜坐标到海马空间视角

原文图注:FIGURE 10 | Coordinate transforms in the primate dorsal visual system. Three stages of coordinate transforms that take place at different levels of the primate dorsal visual system are shown. At each stage the coordinate transform is performed by gain modulation of the receptive field by an appropriate modulator, that is usefully combined with slow learning of the type implemented in VisNet which helps the same neurons at a particular stage to develop what are effectively representations that become independent of the modulating signal. In Layer 1 gain modulation by eye position combined with slow learning enables neurons to develop representations in head-centered coordinates that are invariant with respect to retinal and eye position. In Layer 2 gain modulation by head direction combined with slow learning enables neurons to develop representations in allocentric bearing to a stimulus such as a landmark coordinates that are invariant with respect to head direction. In Layer 3 gain modulation by the place where the individual is located combined with slow learning enables neurons to develop representations of a stimulus such as a landmark that are in allocentric spatial view coordinates with invariance with respect to where the individual is located. The diagram shows the architecture of the VisNetCT model in which gain modulation combined with short-term memory trace associative learning was shown to implement these transforms (Rolls, 2020). Each neuron in a Layer (or cortical area in the hierarchy) receives from neurons in a small region of the preceding Layer. It is proposed that idiothetic update through this dorsal visual cortical stream is used for idiothetic update of hippocampal spatial view cells useful for navigation when the environment may not be visible for short periods (Rolls, 2020, 2021b). PCC, posterior cingulate cortex; RSC, retrosplenial cortex.

背侧通路多级坐标变换体系:Layer 1 通过眼位增益调制结合慢学习,将视网膜坐标转换为独立于眼位的头中心坐标;Layer 2 通过头朝向增益调制转换为独立于头朝向的外在地标方位坐标;Layer 3 通过个体所在位置增益调制转换为独立于个体站位的外在空间视角坐标,最终投射至海马结构。增益调制提供坐标转换,时间痕迹规则绑定不同变换下的表征,支持视线暂时被遮蔽时通过自身运动线索持续导航。

Figure 10

讨论

作者讨论了模型结构与新皮层解剖构筑的对应性:沿灵长类腹侧通路(V1 至 TE),第 3 层锥体神经元的基树突树覆盖范围、树突总长度及棘突密度逐步增加,基树突棘突总数由 V1 的约 640 个升至 TE 的约 7400 个。解剖梯度的差异反映了功能需求:初级皮层维持高空间分辨率以精确绑定局部边缘组合,因而初级网络层采用无痕迹的纯联想 Hebb 规则;高级皮层依靠大感受野与丰富的侧向连接维持吸引子网络放电,支持痕迹学习在时程上的突触强化(如 IT 皮层在刺激撤除后约 1 秒的持续放电)。

在人工网络与神经机制的对比中,作者指出标准深度卷积网络依赖全局误差反传和跨位置卷积核权重复制,多采用大量打散的静态样本进行拟合,易受局部像素微小扰动与对抗样本干扰。通过符合自然规律的连续时间变换序列训练,有助于网络自发学习具有几何容忍性的结构表征。

模型同时存在若干局限性:当前测试主要在受控刺激与中小规模网络中完成,VisNet 能否按比例扩展至灵长类皮层规模(皮层模块含约十万神经元,单个锥体细胞接收数万突触)仍需大尺度验证;此外,利用复发连接吸引子网络实现视角不变性的方案存在随视角数增加导致容量快速衰减的问题,吸引子可能更适合负责模糊输入的模式补全。在物种比较方面,啮齿类动物缺乏中央凹,其视网膜–顶叶系统的中央凹依赖性不如灵长类显著,可能更依赖以自身位置为中心的导航系统;而灵长类的空间视角系统则提供了基于外部显著地标的导航支持。

一句话总结

本综述阐明了基于环境时空连续性的慢速无监督学习规则与分层前馈网络结合的机制,展示了局部突触可塑性在无教师指导下实现物体不变性与多级空间坐标变换的计算可行性。

审校与证据追溯

当前来源未附审校评注。请参照上方登记状态。