Projection-Based Regularized Dual Averaging for Stochastic Optimization

Asahi Ushio, Masahiro Yukawa

研究成果: Article査読

2 被引用数 (Scopus)

抄録

We propose a novel stochastic-optimization framework based on the regularized dual averaging (RDA) method. The proposed approach differs from the previous studies of RDA in three major aspects. First, the squared-distance loss function to a 'random' closed convex set is employed for stability. Second, a sparsity-promoting metric (used implicitly by a certain proportionate-type adaptive filtering algorithm) and a quadratically-weighted \ell -1 regularizer are used simultaneously. Third, the step size and regularization parameters are both constant due to the smoothness of the loss function. These three differences yield an excellent sparsity-seeking property, high estimation accuracy, and insensitivity to the choice of the regularization parameter. Numerical examples show the remarkable advantages of the proposed method over the existing methods (including AdaGrad and the adaptive proximal forward-backward splitting method) in applications to regression and classification with real/synthetic data.

本文言語English
論文番号8680689
ページ(範囲)2720-2733
ページ数14
ジャーナルIEEE Transactions on Signal Processing
67
10
DOI
出版ステータスPublished - 2019 5 15

ASJC Scopus subject areas

  • 信号処理
  • 電子工学および電気工学

フィンガープリント

「Projection-Based Regularized Dual Averaging for Stochastic Optimization」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル