Alternating direction method of multipliers for regularized multiclass support vector machines

Yangyang Xu; Ioannis Akrotirianakis; Amit Chakraborty

doi:10.1007/978-3-319-27926-8_10

Alternating direction method of multipliers for regularized multiclass support vector machines

Yangyang Xu, Ioannis Akrotirianakis, Amit Chakraborty

Institute for Mathematics and its Applications

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Scopus citations

Abstract

The support vector machine (SVM) was originally designed for binary classifications. A lot of effort has been put to generalize the binary SVM to multiclass SVM (MSVM) which are more complex problems. Initially, MSVMs were solved by considering their dual formulations which are quadratic programs and can be solved by standard second-order methods. However, the duals of MSVMs with regularizers are usually more difficult to formulate and computationally very expensive to solve. This paper focuses on several regularized MSVMs and extends the alternating direction method of multiplier (ADMM) to these MSVMs. Using a splitting technique, all considered MSVMs are written as two-block convex programs, for which the ADMM has global convergence guarantees. Numerical experiments on synthetic and real data demonstrate the high efficiency and accuracy of our algorithms.

Original language	English (US)
Title of host publication	Machine Learning, Optimization, and Big Data - 1st International Workshop, MOD 2015 Taormina, Revised Selected Papers
Editors	Mario Pavone, Giovanni Maria Farinella, Vincenzo Cutello, Panos Pardalos
Publisher	Springer- Verlag
Pages	105-117
Number of pages	13
ISBN (Print)	9783319279251
DOIs	https://doi.org/10.1007/978-3-319-27926-8_10
State	Published - Jan 1 2015
Event	1st International Workshop on Machine Learning, Optimization, and Big Data, MOD 2015 - Taormina, Sicily, Italy Duration: Jul 21 2015 → Jul 23 2015

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9432
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	1st International Workshop on Machine Learning, Optimization, and Big Data, MOD 2015
Country	Italy
City	Taormina, Sicily
Period	7/21/15 → 7/23/15

Keywords

Alternating direction method of multipliers
Elastic net
Group lasso
Multiclass classification
Supnorm
Support vector machine

Access

10.1007/978-3-319-27926-8_10

Fingerprint Dive into the research topics of 'Alternating direction method of multipliers for regularized multiclass support vector machines'. Together they form a unique fingerprint.

Cite this

Xu, Y., Akrotirianakis, I., & Chakraborty, A. (2015). Alternating direction method of multipliers for regularized multiclass support vector machines. In M. Pavone, G. M. Farinella, V. Cutello, & P. Pardalos (Eds.), Machine Learning, Optimization, and Big Data - 1st International Workshop, MOD 2015 Taormina, Revised Selected Papers (pp. 105-117). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9432). Springer- Verlag. https://doi.org/10.1007/978-3-319-27926-8_10

Alternating direction method of multipliers for regularized multiclass support vector machines. / Xu, Yangyang; Akrotirianakis, Ioannis; Chakraborty, Amit.

Machine Learning, Optimization, and Big Data - 1st International Workshop, MOD 2015 Taormina, Revised Selected Papers. ed. / Mario Pavone; Giovanni Maria Farinella; Vincenzo Cutello; Panos Pardalos. Springer- Verlag, 2015. p. 105-117 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9432).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Xu, Y, Akrotirianakis, I & Chakraborty, A 2015, Alternating direction method of multipliers for regularized multiclass support vector machines. in M Pavone, GM Farinella, V Cutello & P Pardalos (eds), Machine Learning, Optimization, and Big Data - 1st International Workshop, MOD 2015 Taormina, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9432, Springer- Verlag, pp. 105-117, 1st International Workshop on Machine Learning, Optimization, and Big Data, MOD 2015, Taormina, Sicily, Italy, 7/21/15. https://doi.org/10.1007/978-3-319-27926-8_10

Xu Y, Akrotirianakis I, Chakraborty A. Alternating direction method of multipliers for regularized multiclass support vector machines. In Pavone M, Farinella GM, Cutello V, Pardalos P, editors, Machine Learning, Optimization, and Big Data - 1st International Workshop, MOD 2015 Taormina, Revised Selected Papers. Springer- Verlag. 2015. p. 105-117. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-27926-8_10

Xu, Yangyang ; Akrotirianakis, Ioannis ; Chakraborty, Amit. / Alternating direction method of multipliers for regularized multiclass support vector machines. Machine Learning, Optimization, and Big Data - 1st International Workshop, MOD 2015 Taormina, Revised Selected Papers. editor / Mario Pavone ; Giovanni Maria Farinella ; Vincenzo Cutello ; Panos Pardalos. Springer- Verlag, 2015. pp. 105-117 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{3d1e4558115245249d48b7f311fd403c,

title = "Alternating direction method of multipliers for regularized multiclass support vector machines",

abstract = "The support vector machine (SVM) was originally designed for binary classifications. A lot of effort has been put to generalize the binary SVM to multiclass SVM (MSVM) which are more complex problems. Initially, MSVMs were solved by considering their dual formulations which are quadratic programs and can be solved by standard second-order methods. However, the duals of MSVMs with regularizers are usually more difficult to formulate and computationally very expensive to solve. This paper focuses on several regularized MSVMs and extends the alternating direction method of multiplier (ADMM) to these MSVMs. Using a splitting technique, all considered MSVMs are written as two-block convex programs, for which the ADMM has global convergence guarantees. Numerical experiments on synthetic and real data demonstrate the high efficiency and accuracy of our algorithms.",

keywords = "Alternating direction method of multipliers, Elastic net, Group lasso, Multiclass classification, Supnorm, Support vector machine",

author = "Yangyang Xu and Ioannis Akrotirianakis and Amit Chakraborty",

year = "2015",

month = jan,

day = "1",

doi = "10.1007/978-3-319-27926-8_10",

language = "English (US)",

isbn = "9783319279251",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer- Verlag",

pages = "105--117",

editor = "Mario Pavone and Farinella, {Giovanni Maria} and Vincenzo Cutello and Panos Pardalos",

booktitle = "Machine Learning, Optimization, and Big Data - 1st International Workshop, MOD 2015 Taormina, Revised Selected Papers",

note = "1st International Workshop on Machine Learning, Optimization, and Big Data, MOD 2015 ; Conference date: 21-07-2015 Through 23-07-2015",

}

TY - GEN

T1 - Alternating direction method of multipliers for regularized multiclass support vector machines

AU - Xu, Yangyang

AU - Akrotirianakis, Ioannis

AU - Chakraborty, Amit

PY - 2015/1/1

Y1 - 2015/1/1

N2 - The support vector machine (SVM) was originally designed for binary classifications. A lot of effort has been put to generalize the binary SVM to multiclass SVM (MSVM) which are more complex problems. Initially, MSVMs were solved by considering their dual formulations which are quadratic programs and can be solved by standard second-order methods. However, the duals of MSVMs with regularizers are usually more difficult to formulate and computationally very expensive to solve. This paper focuses on several regularized MSVMs and extends the alternating direction method of multiplier (ADMM) to these MSVMs. Using a splitting technique, all considered MSVMs are written as two-block convex programs, for which the ADMM has global convergence guarantees. Numerical experiments on synthetic and real data demonstrate the high efficiency and accuracy of our algorithms.

AB - The support vector machine (SVM) was originally designed for binary classifications. A lot of effort has been put to generalize the binary SVM to multiclass SVM (MSVM) which are more complex problems. Initially, MSVMs were solved by considering their dual formulations which are quadratic programs and can be solved by standard second-order methods. However, the duals of MSVMs with regularizers are usually more difficult to formulate and computationally very expensive to solve. This paper focuses on several regularized MSVMs and extends the alternating direction method of multiplier (ADMM) to these MSVMs. Using a splitting technique, all considered MSVMs are written as two-block convex programs, for which the ADMM has global convergence guarantees. Numerical experiments on synthetic and real data demonstrate the high efficiency and accuracy of our algorithms.

KW - Alternating direction method of multipliers

KW - Elastic net

KW - Group lasso

KW - Multiclass classification

KW - Supnorm

KW - Support vector machine

UR - http://www.scopus.com/inward/record.url?scp=84955317353&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84955317353&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-27926-8_10

DO - 10.1007/978-3-319-27926-8_10

M3 - Conference contribution

AN - SCOPUS:84955317353

SN - 9783319279251

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 105

EP - 117

BT - Machine Learning, Optimization, and Big Data - 1st International Workshop, MOD 2015 Taormina, Revised Selected Papers

A2 - Pavone, Mario

A2 - Farinella, Giovanni Maria

A2 - Cutello, Vincenzo

A2 - Pardalos, Panos

PB - Springer- Verlag

T2 - 1st International Workshop on Machine Learning, Optimization, and Big Data, MOD 2015

Y2 - 21 July 2015 through 23 July 2015

ER -