Deep Residual Network Predicts Cortical Representation and Organization of Visual Features for Rapid Categorization

Haiguang Wen, Junxing Shi, Wei Chen, Zhongming Liu

Research output: Contribution to journalArticlepeer-review

17 Scopus citations

Abstract

The brain represents visual objects with topographic cortical patterns. To address how distributed visual representations enable object categorization, we established predictive encoding models based on a deep residual network, and trained them to predict cortical responses to natural movies. Using this predictive model, we mapped human cortical representations to 64,000 visual objects from 80 categories with high throughput and accuracy. Such representations covered both the ventral and dorsal pathways, reflected multiple levels of object features, and preserved semantic relationships between categories. In the entire visual cortex, object representations were organized into three clusters of categories: Biological objects, non-biological objects, and background scenes. In a finer scale specific to each cluster, object representations revealed sub-clusters for further categorization. Such hierarchical clustering of category representations was mostly contributed by cortical representations of object features from middle to high levels. In summary, this study demonstrates a useful computational strategy to characterize the cortical organization and representations of visual features for rapid categorization.

Original languageEnglish (US)
Article number3752
JournalScientific reports
Volume8
Issue number1
DOIs
StatePublished - Dec 1 2018

Bibliographical note

Funding Information:
The authors are thankful to Dr. Xiaohong Zhu and Dr. Byeong-Yeul Lee for constructive discussion, and Kuan Han for his assistance in collecting natural images, and Yizhen Zhang for her help in acquiring fMRI. The research was supported by NIH R01MH104402, MH111413, and Purdue University.

Publisher Copyright:
© 2018 The Author(s).

Fingerprint Dive into the research topics of 'Deep Residual Network Predicts Cortical Representation and Organization of Visual Features for Rapid Categorization'. Together they form a unique fingerprint.

Cite this