Kernel-based methods exhibit well-documented performance in various nonlinear learning tasks. Most of them rely on a preselected kernel, whose prudent choice presumes task-specific prior information. To cope with this limitation, multi-kernel learning has gained popularity thanks to its flexibility in choosing kernels from a prescribed kernel dictionary. Leveraging the random feature approximation and its recent orthogonality-promoting variant, the present contribution develops an online multi-kernel learning scheme to infer the intended nonlinear function ‘on the fly.’ To further boost performance in non-stationary environments, an adaptive multi-kernel learning scheme (termed AdaRaker) is developed with affordable computation and memory complexity. Performance is analyzed in terms of both static and dynamic regret. To our best knowledge, AdaRaker is the first algorithm that can optimally track nonlinear functions in non-stationary settings with theoretical guarantees. Numerical tests on real datasets are carried out to showcase the effectiveness of the proposed algorithms.
|Original language||English (US)|
|Number of pages||10|
|State||Published - 2018|
|Event||21st International Conference on Artificial Intelligence and Statistics, AISTATS 2018 - Playa Blanca, Lanzarote, Canary Islands, Spain|
Duration: Apr 9 2018 → Apr 11 2018
|Conference||21st International Conference on Artificial Intelligence and Statistics, AISTATS 2018|
|City||Playa Blanca, Lanzarote, Canary Islands|
|Period||4/9/18 → 4/11/18|
Bibliographical noteFunding Information:
This work is supported in part by the National Science Foundation under Grant 1500713 and 1711471, and NIH 1R01GM104975-01. Tianyi Chen is also supported by the Doctoral Dissertation Fellowship from the University of Minnesota.