本文系统探讨了灵长类腹侧视觉流(ventral visual stream)中“类别正交属性”(category-orthogonal object properties,如物体的位置、尺寸及三维姿态)的神经表征机制。作者在猕猴 V4 区与下颞叶(IT)皮层开展大规模微电极阵列记录,并结合 V1 计算模型,在同一套高变异自然图像集上对比了 16 项解码任务;进一步引入人类在线心理物理实验作为基准参照,并利用层级卷积神经网络(HCNN)复现神经计算过程。研究发现,沿腹侧视觉通路的层级递进,线性可读出的正交属性信息不仅未被丢弃,反而显著增长;IT 群体表征能够与人类在多属性估计上的行为表现精准对齐。该结果否定了“高级视皮层通过牺牲度量属性以实现类别不变性”的传统假设,证实腹侧通路协同保留并优化了丰富的度量属性信息。
研究背景
腹侧视觉通路如何实现对位置、尺寸和三维视角变化的不变性(invariance),是视觉物体识别研究的核心问题。沿着初级视皮层到前部下颞叶(IT)皮层的层级通路,神经元感受野逐渐增大,对局部物理特征的敏感性逐步转化为对高阶物体类别的抽象表征。经典理论常将这一过程解释为“特征过滤与信息舍弃”——即系统通过逐步丢弃或主动抑制与类别无关的度量变化,从而建立对几何变换具有容忍度的类别流形。
然而,这一传统观点面临重要的理论与行为挑战。在日常视觉行为中,观察者不仅需要判断“眼前是什么物体”(类别识别),还需要准确感知“物体在什么位置、多大尺寸、朝向何处”,以指导空间抓取与交互行为。视觉系统若在识别过程中将这些度量信息彻底滤除,势必需要额外构建并行的独立处理系统。另一种可能性是,IT 皮层表征的不变性并不等同于度量信息的物理丢失:物体在群体神经反应空间中可能被映射为一个高度解开的流形(untangled manifold),该流形既支持超平面线性划分物体类别,其内部自由度亦同时线性保留了丰富的位置、大小和姿态等度量属性。
既往研究在探讨该问题时往往受到刺激维度与任务设置的局限:若刺激中仅包含孤立、摆正或低变异的简单物体,神经系统的度量表征特性难以充分激活;且以往实验多将分类任务与位置估计割裂在不同研究中。本研究通过构建跨越宽广参数变化的高变异自然渲染图像库,在 V4 和 IT 皮层开展多位点同步电生理记录,并结合严格的线性支持向量机(SVM)解码,系统评估度量属性在各层级的信息可读性。需客观指出的是:本研究主要基于被动注视任务下的电生理响应,未涵盖执行特定主动抓取任务时的自上而下注意调制;图像刺激为二维合成渲染图像,与三维动态双眼立体环境仍有区别;线性可读性(explicit information)界定了下游网络单突触读取的便捷性,并不等同于皮层编码了全部潜在的原始像素香农熵。
研究思路
研究的核心策略是将“统一刺激空间、多任务解码、跨层级生理记录、人类行为对标与深度网络建模”统合在严密的实验框架内。刺激集构建采用高变异(high-variation)自然图像:将 64 个具有精细三维网格的三维物体模型(涵盖 8 个常见类别),在三维空间中连续随机变化二维平面位置(方位角与仰角)、视角尺寸及三维旋转姿态(偏航角、俯仰角与翻滚角),并合成于真实杂乱自然场景背景中,确保不同属性在统计上相互独立。
分析方法以“线性可读出信息”(explicit / linearly accessible information)作为核心量化标尺,利用线性支持向量机(SVM)对来自 V1 计算模型、记录自猕猴 V4 和 IT 皮层的神经群体尖峰发放率开展跨试次解码测试。解码体系涵盖 16 项独立任务:包含 8 种主要属性(粗分类、细分类、二维位置、视角大小、三维姿态角)在两种难度条件下的交叉组合。最后,将解码性能与人类心理物理测试进行定量对标,并输入目标驱动的深度卷积神经网络(HCNN),检验前馈计算模型能否自发涌现出一致的信息层级演化格局。
方法
实验使用两只成年恒河猴(Macaca mulatta)。在两猴左半球 IT 皮层腹侧前部 TE 区与视皮层 V4 区分别长期慢性植入高密度多微电极阵列(Utah 阵列与微驱动深部电极),每次实验同步记录数十至上百个独立电生理单元。猴在中央注视点(0.5° 容限)维持注视期间,屏幕以快速串行视觉呈现(RSVP)方式呈现测试图像,刺激呈现时长为 100 ms,间隔为 100 ms。神经响应提取刺激呈现后 70 至 170 ms 时窗内的动作电位计数并开展均值归一化。
共生成 5,760 张高变异合成测试图像(涵盖 8 个物体类别、64 个物体实例),每张图像重复呈现 20–47 次。分析引入了四种对照与模型输入:(1) 像素对照(Pixel);(2) 经典的 V1 复杂细胞计算模型;(3) V4 记录群体;(4) IT 记录群体。解码采用线性支持向量机(线性 SVM,正则化参数 C 经交叉验证优化),性能评估采用折半交叉验证并计算皮尔逊相关系数 r 或分类准确率。人类心理物理实验通过 Amazon Mechanical Turk 在线采集(数百名正常受试者),采用快速掩蔽呈现范式完成完全相同的 16 项视觉估计任务;深度计算模型采用经 ImageNet 物体识别任务预训练的深度卷积神经网络(Zeiler-Fergus 与 AlexNet 架构),逐层提取特征图计算线性解码指标。
主要结果
- 高变异图像中所有物体属性信息沿腹侧通路层级单调递增(图 3):在全部 16 项解码任务中(包括物体粗分类、细分类、平面位置、尺寸及三维旋转角度),IT 神经群体的线性解码准确率均系统性显著优于 V4 群体(在绝大多数任务中 P < 0.005);IT 与 V4 的表现均显著超越初级视皮层 V1 模型;原始像素表征的表现则显著垫底。在单位点层面,最佳 IT 单个电极的解码性能亦显著优于最佳 V4 单个位点(P < 0.01)。
- IT 皮层神经表征精确预测人类行为表现(图 4):跨 16 个任务的对比显示,IT 神经群体的解码表现与人类行为表现的相关系数高达 r = 0.99(P < 10⁻¹²),各任务在 IT 表征空间中的相对难度排序与人类知觉完全吻合;而 V4 群体与人类表现的相关性(r = 0.77)显著较弱,V1 模型(r = 0.44)与像素基准(r = 0.32)则无法有效预测人类行为。即使在单一任务内部,IT 神经表征亦能准确预测人类在各测试图像子集上的判断误差模式。
- 度量信息在 IT 皮层呈广泛分布式表征(图 5):信息并非集中于极少数特异性位点,随机抽取 IT 位点构建子群体显示,随着采样位点数量增加(从 10 个增加至 100 个以上),全部属性的解码准确率呈渐近饱和上升曲线;属性重叠度分析表明,同一批 IT 位点通常同时携带位置、尺寸、姿态与类别信息,各属性在单神经元上的绝对调谐权重表现为独立的正交混合,而非功能模块化的隔离分布。
- 性能优化的深度卷积网络自发复现生物层级信息格局(图 6):在 ImageNet 上进行物体分类训练的层级深度卷积神经网络(HCNN),其内部各层特征表现出与灵长类腹侧通路完全一致的演变规律——随着网络层级加深,不仅类别解码能力显著提高,位置、尺寸及姿态等正交属性的线性可读出信息亦同步持续上升,且最高特征层(FC7)最能预测 IT 神经响应与人类行为。
- 信息递增格局高度依赖刺激环境的现实变异度(图 7):当将测试图像的人工变异量(位置、尺寸、姿态的扰动方差)人为压缩至低变异或无变异时,沿层级递增的信息格局发生显著重塑甚至反转:在无姿态与尺寸变异的简化图像中,V1 模型即可通过低级像素重叠轻松读出位置,导致 V1 或 V4 的线性解码性能看似超越 IT。这表明只有在高度复杂的自然变异约束下,高级视皮层整合表征的计算优势才得以充分展现。
图注解读
图 1 · 四种假说的图解
原文图注:Figure 1 Illustration of possible scenarios. (a) Prior to this study, extensive research has shown that invariant category recognition performance increases along the ventral pathway (top), whereas lower and intermediate visual areas are sensitive to various categorical-orthogonal properties (position, border continuity, etc.) in simple stimuli. It was also known that IT contains some information for category-orthogonal properties: as illustrated (bottom), performance in IT must be above floor. (b) However, the previous literature determined neither the relative amounts of explicitly decodable information for category-orthogonal properties between ventral cortical areas nor the ratio of neural decode performance in IT (or elsewhere) to measured behavioral performance levels. In other words, there were multiple qualitatively different hypotheses consistent with the known data as to both the red curve's shape and its height on the y axis. In hypothesis H1a, there is a tradeoff between increasing receptive field size and categorization ability, and performance on the orthogonal task. Early areas match human performance on these tasks, whereas later areas do not. This is probably the dominant view in the visual neuroscience community. In hypothesis H1b, the same tradeoff holds as in H1a, except that the human performance is matched in IT, rather than early layers. In hypothesis H2, explicitly decodable information peaks (for at least some non-categorical properties) in intermediate visual areas, analogous with the results for V2 border-ownership cells that have been found in the context of simple visual stimuli. In hypothesis H3, information is neither lost nor gained for the orthogonal variable tasks up through the ventral stream, it is simply preserved. This view is suggested as one possibility in previous studies from our group and is consistent with ideas of hyperacuity. Finally, in hypothesis H4, information increases for the orthogonal tasks along with the categorization tasks. Aspects of this possibility are consistent with coarse coding.
解读:上排画的是已知事实(分类信息递增;低级区对简单刺激的正交属性敏感;IT 至少有"高于地板"的一些信息),下排画的是五种候选曲线——H1a/H1b 都主张递减(差别只在何处达到人类水平),H2 主张中间区峰值,H3 主张水平保持,H4 主张与分类同向上升。红色曲线即"类别正交性能随腹侧层级深度"的走向,纵轴以人类水平为 1。这张图界定了全文要裁决的问题空间;正文图 3 显示只有 H4 在复杂刺激上成立。

图 2 · 刺激、记录与解码流水线
原文图注:Figure 2 Large-scale electrophysiological measurement of neural responses in macaque IT and V4 cortex to visual object stimuli containing high levels of object viewpoint variation. (a) We recorded neural responses to 5,760 high-variation naturalistic images consisting of 64 exemplar objects in eight categories (animals, boats, cars, chairs, faces, fruits, planes, tables), placed on natural scene backgrounds, at a wide range of positions, sizes and poses. (b) Stimuli were presented to awake fixating animals for 100 ms in a rapid serial visual presentation (RSVP) procedure (horizontal black bars indicate stimulus-presentation period). Object centers varied within 8° of fixation center. Recordings were made using chronically implanted electrode arrays, collecting a total of 392 neuronal sites in IT (n = 266) and V4 (n = 126) visual cortex. Each stimulus was repeated between 25 and 50 times. Spike counts were binned in the time window 70–170 ms post stimulus presentation (as indicated by shaded regions) and averaged across repetitions, to produce a 5,760 × 392 neural response pattern array. (c) We then used linear readouts to decode a variety of types of image information from the neural responses, including categorical data such as object category and exemplar identity, as well as continuous data such as object position, retinal and three-dimensional object size, two- and three-dimensional pose angles, object perimeter, and aspect ratio.
解读:三栏分别是刺激集示意(类别、型号、位置/姿态/尺度变异、自然场景背景)、RSVP 时间线与阵列记录(阴影标出 70–170 ms 计数窗),以及群体读出流程:5,760 图 × 392 位的响应矩阵接入线性解码器,输出从分类到位置、尺寸、姿态、周长、长宽比等各种属性。读懂这张图就等于握住了全文的坐标系。

图 3 · 各区域对 16 个任务的解码对比
原文图注:Figure 3 Comparison between ventral cortical areas of object property information encoding in high-variation stimuli. (a) Performance of single best sites from IT (blue bars) and V4 (green bars) on each task measured task. Best sites were chosen in a cross-validated manner, with performance being evaluated on held-out images. Chance performance is at 0. Error bars represent s.d. of the mean taken over subsets of images used to choose the best site for each task. n.s. indicates IT-V4 difference not significant, P < 0.05, P < 0.005. (b) Population decoding. For each task, we trained a linear decoder on neural output. For discrete-valued tasks, including object categorization and subordinate identification, we used support vector machine (SVM) classifiers with L2-regularization. For continuous-valued estimation tasks, we used linear regression with L2-regularization. We compared decoding performance for our recorded IT population sample (blue bars) and V4 population sample (green bars), as well as for a performance-optimized V1 Gabor wavelet model with competitive normalization (gray bars) and the trivial pixel control (black bars). For categorical properties, bar height represents balanced accuracy (0 = chance, 1 = perfect). For continuous properties, bar height represents the Pearson correlation between the predicted value and the actual ground-truth value. All values are shown on cross-validated testing images held out during classifier and regressor training. All evaluations are performed with n = 126 sites and a fixed number of training and testing examples. Error bars represent s.d. of the mean over cross-validation image splits and, in the case of pixel, V1 and IT data, over multiple subsamplings of 126 units from the whole population. In b, IT-V4 and V4-V1 separations were significant at P < 0.005 except where notated; n.s. indicates difference not significant, P < 0.05. See Supplementary Table 1 for statistical details.
解读:a 是逐任务的"最佳单电极"对比(蓝 IT、绿 V4,机遇水平为 0),b 是固定 126 位点的群体解码对比(另加灰色的 V1 模型与黑色的像素基线)。纵轴对离散任务是平衡准确率、对连续任务是预测-真值 Pearson 相关。读图要点:绝大多数任务上蓝柱 > 绿柱 > 灰柱 > 黑柱,IT-V4 差异多标注 P<0.005,只有少数任务(如图中 3D 尺度附近)标 n.s. 或仅 P<0.05。这张图是主要结果第 1 条的直接证据。

图 4 · 神经解码与人类表现的对标
原文图注:Figure 4 Comparison of neural population decoding performance to human psychophysical measurements. (a) Human-relative performance as a function of number of subsampled sites used to decode the property, for selected tasks. The x axis represents the number of sites. For each task, the y axis represents the performance of the decoder with the indicated number of sites, as a fraction of median human performance for that task, with a value of 1 indicating human performance parity. Decoder performance metrics and training procedures are as described in Figure 3b. Solid lines represent measured data and dotted lines represent log-linear extrapolations based on the measured data. Shaded areas around each solid line represent s.e. assessed by bootstrap resampling of sites and images. We evaluated our measured IT (blue lines) and V4 (green lines) neural populations out to the entire recorded populations of 266 and 126 sites, respectively, and evaluated V1 model (gray lines) and pixels (black lines) out to 2,000 units. Human performance for each indicated task was measured using large-scale web-based psychophysics. 1 s.d. in the human performance is indicated by gray shading flanking y = 1 (median performance level). (b) Scatters show human performance (x axis) versus neural performance (y axis) for a variety of tasks. Large squares (n = 14) correspond to the tasks indicated in Table 1. Small circles (n = 41) indicate values for further breakdown of the data into subordinate identification and pose estimation tasks on a per-category basis. (c) Summary of data from b. Bar height represents Spearman's R correlation between human and neural decode for the aggregated large-square tasks (left) and disaggregated small-circle tasks (right). Error bars are s.d. of the mean due to task and image variation. The dotted line represents the mean self-consistency of the measured human population, averaged across multiple subsets of the population sample. Horizontal gray bars represent the s.d. of the mean of human self-consistency, across population, task and image subsets. n.s. indicates difference not significant, P < 0.05, *P < 0.005. See Supplementary Table 1 for statistical details.
解读:a 是核心曲线——纵轴以人类中位表现为 1,横轴位点数(对数刻度):IT 蓝线最先撞到 y=1 的"人类平价带",V4 绿线按外推要数十万甚至更多位点,V1 模型与像素线始终够不到;dotted 线是 log-线性外推。b 把每任务的人类表现(横轴)对神经解码性能(纵轴)画成散点,跨任务走势的匹配度即 c 中的 Spearman R——IT 最高、显著高于 V4/V1/像素。这组图支撑主要结果第 2、3 条,即"多少位点、哪种格局"两层意义上 IT 都最像行为的发生地。

图 5 · 信息在 IT 位点间的分布与任务间重叠
原文图注:Figure 5 Distribution and overlap of IT cortex site contribution across tasks. (a) Histograms of values of sparseness over all tasks. Sparseness is measured via excess kurtosis (γ2). Reference values show fractions of 'high-relevance' sites, as determined by three-point distribution method. Gray band represents 1 s.d. of distribution of sparseness values taken on size-matched samples from a Gaussian distribution. (b) Histograms of values of imbalance over all tasks. Imbalance is measured via skewness (γ1). Reference values at the top of the imbalance panel show fractions of values above versus below means, ranging from 1.3 to 0.7. Gray band represents 1 s.d. of distribution of imbalance values taken on size-matched samples from a Gaussian distribution. (c) Sparseness (left) and imbalance (right) of weight distributions for selected tasks. Error bars represent s.d. over image splits on which weights were determined. Gray bands here are defined as in a and b. (d) Quantification of weight pattern overlap for pairs of tasks. Each colored square in the heat map is the Pearson correlation between the absolute value of the weight vectors for a pair of tasks. A high value (red color) indicates that the weight pattern for the pair of tasks is similar; a low value (blue color) indicates the opposite. White indicates a value that is not statistically significantly different from zero. The order of tasks is the same as in c.
解读:a、b 的直方图看每个任务权重分布的形状——尖峰度(sparseness)衡量"是否只有少数位点被高权重选中",偏度(imbalance)衡量正相关与负相关位点谁多。多数任务落在同规模高斯样本的灰色带内(高权重位点 15–35%、Gaussian 对照 32.5%),比例接近对称(0.7–1.3),说明没有任务专属的小集团。d 的热图看任务两两重叠:多数格偏红(56.5% 正重叠),语义相近的任务(各类尺寸任务)成块发红,而面孔检测一行/列发蓝色。此图支撑主要结果第 4 条。

图 6 · 计算模型复现跨层级信息格局
原文图注:Figure 6 Computational modeling results. (a) Performance of fully-trained model at each hidden layer. y axes are as described in Figure 3b for corresponding tasks. (b) Scatter plots of performance of computational model's top hidden layer on training-set categorization performance versus testing-set estimation accuracy for selected non-categorical tasks. Each dot represents a state of the model during training. (c) Quantification of relationship in b, shown for all tested tasks aside from categorization itself (n = 15). Bar height represents Pearson correlation of accuracy on indicated task with test-set categorization performance, taken across training time steps. Error bars represent s.d. of the mean taken across both time steps as well as splits of images used for performance assessment. (d) Scatter plot of performance of top hidden layer of fully trained model versus performance of IT neural representation, on each task measured in Table 1. As in Figure 4b, large squares represent aggregated tasks (n = 16) and small circles represent disaggregated tasks (n = 43). Unlike Figure 4b, several tasks are included for which human data were not collected. (e) Consistency of fully trained model with neural performance pattern across layers, using the same metric described in Figure 4c. y axis and error bars are as described in Figure 4c. See Supplementary Table 1 for statistical details.
解读:a 显示六层模型从浅到深在所有任务上的性能都单调上升;b、c 是训练时程分析——顶层在测试集上对非类别任务的表现随分类表现同步增长(每个点是一个训练时刻的模型快照),意味着"为分类学习而堆出来的表征顺手把位置、姿态都表征了";d 为模型顶层 vs IT 神经的逐任务散点(大方块 16 个聚合任务、小圈 43 个拆分任务);e 用与图 4c 相同的一致性指标显示这种吻合随模型层级加深而增强。此图支撑主要结果第 5 条。

图 7 · 信息格局依赖刺激变异量
原文图注:Figure 7 Dependence of linearly accessible information on the amount of variation in stimuli. (a) Population neural decoding results for position and orientation tasks defined on a simpler stimulus set consisting of grating patches placed on gray backgrounds. y axis, bar colors and error bars are as described in Figure 3b. (b) Performance of three selected neural network model layers (layer 1, yellow; layer 3, olive green; layer 6, cyan) for the tasks shown in a. (c) Population decoding performance as a function of amount of rotational variation in classifier training and testing data sets, for each of several representative object tasks, for measured IT neural population, V4 neural population, V1 model and for the three model layers. x axis represents (absolute value of) the amount of rotational variation allowed in all three rotational axes; for example, a value of 10 corresponds to rotation in x, y and z axes ranging from −10 to 10 degrees. y axis is performance evaluated using the same metrics and decoder training procedures as described in Figure 3b. Error bars are computed over selections of sites and units as well as image training splits (see Supplementary Table 1 for statistical details).
解读:a 用灰底光栅块重复图 3b 式的分析——此时灰色 V1 模型柱反超,V4 与 IT 显著高于机遇但 IT 不再优于 V4;b 显示模型同样的层内倒序(第 1 层最高、第 6 层最低)。c 把训练/测试集按三轴旋转变异量(如 x 轴取 10 即 ±10°)切分:总体上变异越小,V4-IT 与顶层-中层模型差距越小,个别任务甚至反转。此图支撑主要结果第 6 条:跨区域的信息格局由变异复杂度决定,而非任务本身的类型。

讨论
作者将实证结果归结为一个核心结论:在复杂自然图像域中,“所有行为相关的物体属性沿腹侧视觉通路层级被协同提取”(支持 H4 假说),而经典的不变性权衡假设(H1a 与 H1b)被证伪,独立模块特化假说(H3)亦未得到支持。从神经计算机制来看,这一现象与高维群体流形解缠绕(manifold untangling)理论高度自洽:层级通路的计算目标并非主动滤除或丢弃度量几何信息,而是通过非线性特征映射,将纠缠在一起的复杂图像流形展开为低维平滑子空间。在这一展开过程中,支持超平面划分不仅使得物体类别可以被单层线性感知机轻松读取,由物体平移、缩放与视角变换构成的正交坐标轴亦同时被线性解耦,使得度量信息的可读性大幅提高。
该发现深刻改变了关于腹侧“what”视觉通路的传统认知:腹侧流不仅负责抽象语义分类,同时也是一个高保真的几何测量引擎,能够为空间交互与抓取提供精细的度量参数。结合深度卷积网络的研究表明,以通用物体分类为目标训练的前馈层级网络能够自发复现这一神经表征格局,表明两类信息的协同提取是层级视觉前馈计算在真实世界高维变异约束下的普适优化结果。局限性方面,作者指出实验在被动注视范式下完成,主动行为任务中的注意力偏置与反馈回路对局部属性权重的瞬态重构效应,仍需结合认知调控任务进一步探讨。
一句话总结
本研究通过大规模电生理记录、计算建模与人类心理物理实验,证实腹侧视觉通路在逐层构建物体类别表征的同时,位置、尺寸与姿态等类别正交度量信息的线性可读性亦同步单调递增,确立了高级视皮层表征具有协同保留与优化几何度量信息的计算架构。