p-regularized least squares (0 < p < 1) and critical path

Masahiro Yukawa, Shun Ichi Amari

研究成果: Article査読

7 被引用数 (Scopus)

抄録

This paper elucidates the underlying structures of ℓp-regularized least squares problems in the nonconvex case of 0 < p < 1. The difference between two formulations is highlighted (which does not occur in the convex case of p = 1): 1) an ℓp -constrained optimization (Ppc) and 2) an ℓp-penalized (unconstrained) optimization (Lλ p). It is shown that the solution path of (Lλp) is discontinuous and also a part of the solution path of (Ppc). As an alternative to the solution path, a critical path is considered, which is a maximal continuous curve consisting of critical points. Critical paths are piecewise smooth, as can be seen from the viewpoint of the variational method, and generally contain non-optimal points, such as saddle points and local maxima as well as global/local minima. Our study reveals multiplicity (non-monotonicity) in the correspondence between the regularization parameters of (Ppc) and (Lλp). Two particular paths of critical points connecting the origin and an ordinary least squares (OLS) solution are studied further. One is a main path starting at an OLS solution, and the other is a greedy path starting at the origin. Part of the greedy path can be constructed with a generalized Minkowskian gradient. This paper of greedy path leads to a nontrivial close-link between the optimization problem of ℓp -regularized least squares and the greedy method of orthogonal matching pursuit.

本文言語English
論文番号7330004
ページ(範囲)488-502
ページ数15
ジャーナルIEEE Transactions on Information Theory
62
1
DOI
出版ステータスPublished - 2016 1 1

ASJC Scopus subject areas

  • 情報システム
  • コンピュータ サイエンスの応用
  • 図書館情報学

フィンガープリント

「ℓ<sub>p</sub>-regularized least squares (0 < p < 1) and critical path」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル