G -Softmax: Improving Intraclass Compactness and Interclass Separability of Features

Yan Luo, Yongkang Wong, Mohan Kankanhalli, Qi Zhao

Research output: Contribution to journalArticlepeer-review

6 Scopus citations

Abstract

Intraclass compactness and interclass separability are crucial indicators to measure the effectiveness of a model to produce discriminative features, where intraclass compactness indicates how close the features with the same label are to each other and interclass separability indicates how far away the features with different labels are. In this paper, we investigate intraclass compactness and interclass separability of features learned by convolutional networks and propose a Gaussian-based softmax (G -softmax) function that can effectively improve intraclass compactness and interclass separability. The proposed function is simple to implement and can easily replace the softmax function. We evaluate the proposed G -softmax function on classification data sets (i.e., CIFAR-10, CIFAR-100, and Tiny ImageNet) and on multilabel classification data sets (i.e., MS COCO and NUS-WIDE). The experimental results show that the proposed G -softmax function improves the state-of-the-art models across all evaluated data sets. In addition, the analysis of the intraclass compactness and interclass separability demonstrates the advantages of the proposed function over the softmax function, which is consistent with the performance improvement. More importantly, we observe that high intraclass compactness and interclass separability are linearly correlated with average precision on MS COCO and NUS-WIDE. This implies that the improvement of intraclass compactness and interclass separability would lead to the improvement of average precision.

Original languageEnglish (US)
Article number8712413
Pages (from-to)685-699
Number of pages15
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume31
Issue number2
DOIs
StatePublished - Feb 2020

Bibliographical note

Funding Information:
Manuscript received June 5, 2018; revised December 20, 2018; accepted March 30, 2019. Date of publication May 10, 2019; date of current version February 5, 2020. This research was funded in part by the NSF under Grant 1849107, in part by the University of Minnesota Department of Computer Science and Engineering Start-up Fund (QZ), and in part by the National Research Foundation, Prime Minister’s Office, Singapore under its Strategic Capability Research Centres Funding Initiative. (Corresponding author: Qi Zhao.) Y. Luo and Q. Zhao are with the Department of Computer Science and Engineering, University of Minnesota, Minneapolis, MN 55455 USA (e-mail: luoxx648@umn.edu; qzhao@cs.umn.edu).

Publisher Copyright:
© 2012 IEEE.

Keywords

  • Compactness and separability
  • Gaussian-based softmax
  • deep learning
  • multilabel classification

PubMed: MeSH publication types

  • Journal Article

Fingerprint

Dive into the research topics of 'G -Softmax: Improving Intraclass Compactness and Interclass Separability of Features'. Together they form a unique fingerprint.

Cite this