Bayesian Reconstruction of Natural Images from Human Brain Activity

Thomas Naselaris, Ryan J. Prenger, Kendrick N. Kay, Michael Oliver, Jack L. Gallant

Research output: Contribution to journalArticlepeer-review

308 Scopus citations

Abstract

Recent studies have used fMRI signals from early visual areas to reconstruct simple geometric patterns. Here, we demonstrate a new Bayesian decoder that uses fMRI signals from early and anterior visual areas to reconstruct complex natural images. Our decoder combines three elements: a structural encoding model that characterizes responses in early visual areas, a semantic encoding model that characterizes responses in anterior visual areas, and prior information about the structure and semantic content of natural images. By combining all these elements, the decoder produces reconstructions that accurately reflect both the spatial structure and semantic category of the objects contained in the observed natural image. Our results show that prior information has a substantial effect on the quality of natural image reconstructions. We also demonstrate that much of the variance in the responses of anterior visual areas to complex natural images is explained by the semantic category of the image alone.

Original languageEnglish (US)
Pages (from-to)902-915
Number of pages14
JournalNeuron
Volume63
Issue number6
DOIs
StatePublished - Sep 24 2009

Bibliographical note

Funding Information:
This work was supported by an NRSA postdoctoral fellowship (T.N.), the National Institutes of Health, and University of California, Berkeley, intramural funds. We thank B. Inglis for assistance with MRI, K. Hansen for assistance with retinotopic mapping, D. Woods and X. Kang for acquisition of whole-brain anatomical data, and A. Rokem for assistance with scanner operation. We thank A. Berg for assistance with the natural image database, B. Yu and T. Griffiths for consultation on the mathematical analyses, and S. Nishimoto, and D. Stansbury for their help in various aspects of this research.

Keywords

  • SYSNEURO

Fingerprint

Dive into the research topics of 'Bayesian Reconstruction of Natural Images from Human Brain Activity'. Together they form a unique fingerprint.

Cite this