Predicting human gaze beyond pixels

Juan Xu, Ming Jiang, Shuo Wang, Mohan S. Kankanhalli, Qi Zhao

Research output: Contribution to journalArticlepeer-review

151 Scopus citations

Abstract

A large body of previous models to predict where people look in natural scenes focused on pixel-level image attributes. To bridge the semantic gap between the predictive power of computational saliency models and human behavior, we propose a new saliency architecture that incorporates information at three layers: pixel-level image attributes, object-level attributes, and semanticlevel attributes. Object- and semantic-level information is frequently ignored, or only a few sample object categories are discussed where scaling to a large number of object categories is not feasible nor neurally plausible. To address this problem, this work constructs a principled vocabulary of basic attributes to describe object- and semantic-level information thus not restricting to a limited number of object categories. We build a new dataset of 700 images with eye-tracking data of 15 viewers and annotation data of 5,551 segmented objects with fine contours and 12 semantic attributes (publicly available with the paper). Experimental results demonstrate the importance of the object- and semantic-level information in the prediction of visual attention.

Original languageEnglish (US)
Article number28
JournalJournal of vision
Volume14
Issue number1
DOIs
StatePublished - 2014

Keywords

  • Computational model
  • Dataset
  • Object saliency
  • Saliency attribute
  • Semantic saliency
  • Visual saliency

Fingerprint Dive into the research topics of 'Predicting human gaze beyond pixels'. Together they form a unique fingerprint.

Cite this