Active learning for top-K rank aggregation from noisy comparisons

Soheil Mohajer; Changho Suh; Adel Elmahdy

Active learning for top-K rank aggregation from noisy comparisons

Soheil Mohajer, Changho Suh, Adel Elmahdy

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

12 Scopus citations

Abstract

We explore an active top-A" ranking problem based on pairwise comparisons that are collected possibly in a sequential manner as per our design choice. We consider two settings: (1) top-K sorting in which the goal is to recover the top-A" items in order out of n items; (2) top-K partitioning where only the set of top-A' items is desired. Under a fairly general model which subsumes as special cases various models (e.g., Strong Stochastic Transitivity model, BTL model and uniform noise model), we characterize upper bounds on the sample size required for top-A∗sorting as well as for top-A" partitioning. As a consequence, we demonstrate that active ranking can offer significant multiplicative gains in sample complexity over passive ranking. Depending on the underlying stochastic noise model, such gain varies from around logn/log log n to n² log n/log log n. We also Present an algorithm that is applicable to both settings.

Original language	English (US)
Title of host publication	34th International Conference on Machine Learning, ICML 2017
Publisher	International Machine Learning Society (IMLS)
Pages	3842-3853
Number of pages	12
ISBN (Electronic)	9781510855144
State	Published - 2017
Event	34th International Conference on Machine Learning, ICML 2017 - Sydney, Australia Duration: Aug 6 2017 → Aug 11 2017

Publication series

Name	34th International Conference on Machine Learning, ICML 2017
Volume	5

Other

Other	34th International Conference on Machine Learning, ICML 2017
Country/Territory	Australia
City	Sydney
Period	8/6/17 → 8/11/17

Bibliographical note

Funding Information:
The authors would like to thank the reviewers who gave useful comments. C. Suh was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP; Ministry of Science, ICT & Future Planning) (No. 2015R1C1A1A02036561).

Publisher Copyright:
© 2017 by the author(s).

OpenUrl availability

Full text

Cite this

Active learning for top-K rank aggregation from noisy comparisons. / Mohajer, Soheil; Suh, Changho; Elmahdy, Adel.
34th International Conference on Machine Learning, ICML 2017. International Machine Learning Society (IMLS), 2017. p. 3842-3853 (34th International Conference on Machine Learning, ICML 2017; Vol. 5).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Mohajer, S, Suh, C & Elmahdy, A 2017, Active learning for top-K rank aggregation from noisy comparisons. in 34th International Conference on Machine Learning, ICML 2017. 34th International Conference on Machine Learning, ICML 2017, vol. 5, International Machine Learning Society (IMLS), pp. 3842-3853, 34th International Conference on Machine Learning, ICML 2017, Sydney, Australia, 8/6/17.

@inproceedings{27c6517235a44ee689fa8b1b39c61670,

title = "Active learning for top-K rank aggregation from noisy comparisons",

abstract = "We explore an active top-A{"} ranking problem based on pairwise comparisons that are collected possibly in a sequential manner as per our design choice. We consider two settings: (1) top-K sorting in which the goal is to recover the top-A{"} items in order out of n items; (2) top-K partitioning where only the set of top-A' items is desired. Under a fairly general model which subsumes as special cases various models (e.g., Strong Stochastic Transitivity model, BTL model and uniform noise model), we characterize upper bounds on the sample size required for top-A∗sorting as well as for top-A{"} partitioning. As a consequence, we demonstrate that active ranking can offer significant multiplicative gains in sample complexity over passive ranking. Depending on the underlying stochastic noise model, such gain varies from around logn/log log n to n2 log n/log log n. We also Present an algorithm that is applicable to both settings.",

author = "Soheil Mohajer and Changho Suh and Adel Elmahdy",

note = "Funding Information: The authors would like to thank the reviewers who gave useful comments. C. Suh was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP; Ministry of Science, ICT & Future Planning) (No. 2015R1C1A1A02036561). Publisher Copyright: {\textcopyright} 2017 by the author(s).; 34th International Conference on Machine Learning, ICML 2017 ; Conference date: 06-08-2017 Through 11-08-2017",

year = "2017",

language = "English (US)",

series = "34th International Conference on Machine Learning, ICML 2017",

publisher = "International Machine Learning Society (IMLS)",

pages = "3842--3853",

booktitle = "34th International Conference on Machine Learning, ICML 2017",

}

TY - GEN

T1 - Active learning for top-K rank aggregation from noisy comparisons

AU - Mohajer, Soheil

AU - Suh, Changho

AU - Elmahdy, Adel

N1 - Funding Information: The authors would like to thank the reviewers who gave useful comments. C. Suh was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP; Ministry of Science, ICT & Future Planning) (No. 2015R1C1A1A02036561). Publisher Copyright: © 2017 by the author(s).

PY - 2017

Y1 - 2017

N2 - We explore an active top-A" ranking problem based on pairwise comparisons that are collected possibly in a sequential manner as per our design choice. We consider two settings: (1) top-K sorting in which the goal is to recover the top-A" items in order out of n items; (2) top-K partitioning where only the set of top-A' items is desired. Under a fairly general model which subsumes as special cases various models (e.g., Strong Stochastic Transitivity model, BTL model and uniform noise model), we characterize upper bounds on the sample size required for top-A∗sorting as well as for top-A" partitioning. As a consequence, we demonstrate that active ranking can offer significant multiplicative gains in sample complexity over passive ranking. Depending on the underlying stochastic noise model, such gain varies from around logn/log log n to n2 log n/log log n. We also Present an algorithm that is applicable to both settings.

AB - We explore an active top-A" ranking problem based on pairwise comparisons that are collected possibly in a sequential manner as per our design choice. We consider two settings: (1) top-K sorting in which the goal is to recover the top-A" items in order out of n items; (2) top-K partitioning where only the set of top-A' items is desired. Under a fairly general model which subsumes as special cases various models (e.g., Strong Stochastic Transitivity model, BTL model and uniform noise model), we characterize upper bounds on the sample size required for top-A∗sorting as well as for top-A" partitioning. As a consequence, we demonstrate that active ranking can offer significant multiplicative gains in sample complexity over passive ranking. Depending on the underlying stochastic noise model, such gain varies from around logn/log log n to n2 log n/log log n. We also Present an algorithm that is applicable to both settings.

UR - http://www.scopus.com/inward/record.url?scp=85048474867&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048474867&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85048474867

T3 - 34th International Conference on Machine Learning, ICML 2017

SP - 3842

EP - 3853

BT - 34th International Conference on Machine Learning, ICML 2017

PB - International Machine Learning Society (IMLS)

T2 - 34th International Conference on Machine Learning, ICML 2017

Y2 - 6 August 2017 through 11 August 2017

ER -

Active learning for top-K rank aggregation from noisy comparisons

Abstract

Publication series

Other

Bibliographical note

OpenUrl availability

Other files and links

Fingerprint

Cite this