Can critical-point paths under ℓp-regularization (0 < p < 1) reach the sparsest least squares solutions?

Kwangjin Jeong, Masahiro Yukawa, Shun Ichi Amari

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

The solution path of the least square problem under ℓp- regularization (0<p<1) is studied, where the Lagrangian multiplier λ due to the constraint is the parameter of the path. It is first proven that the least square solution of an unconstrained overdetermined linear system is connected with the origin, under a mild condition, by a continuous path of critical points of an p-regularized squared error function. Based on this fact, it is proven that every sparsest least square solution of an underdetermined system is connected with the origin by a critical-point path. The existence theorem holds more generally for any least square solution whose support has its associated submatrix of the fat sensing matrix be full column rank. This is a sufficient condition for the existence, and allows to reduce the underdetermined problem to an overdetermined one with the off-support variable(s) nullified. A necessary condition is that the gradient of the ℓp regularizer with respect to the support variables lies in the row space of the submatrix (which is not necessarily full column rank).

Original languageEnglish
Article number6776531
Pages (from-to)2960-2968
Number of pages9
JournalIEEE Transactions on Information Theory
Volume60
Issue number5
DOIs
Publication statusPublished - 2014

Fingerprint

multiplier
Oils and fats
Linear systems

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Cite this

Can critical-point paths under ℓp-regularization (0 < p < 1) reach the sparsest least squares solutions? / Jeong, Kwangjin; Yukawa, Masahiro; Amari, Shun Ichi.

In: IEEE Transactions on Information Theory, Vol. 60, No. 5, 6776531, 2014, p. 2960-2968.

Research output: Contribution to journalArticle

@article{cfe202592d8f40339534d8c279fabab8,
title = "Can critical-point paths under ℓp-regularization (0 < p < 1) reach the sparsest least squares solutions?",
abstract = "The solution path of the least square problem under ℓp- regularization (0p-regularized squared error function. Based on this fact, it is proven that every sparsest least square solution of an underdetermined system is connected with the origin by a critical-point path. The existence theorem holds more generally for any least square solution whose support has its associated submatrix of the fat sensing matrix be full column rank. This is a sufficient condition for the existence, and allows to reduce the underdetermined problem to an overdetermined one with the off-support variable(s) nullified. A necessary condition is that the gradient of the ℓp regularizer with respect to the support variables lies in the row space of the submatrix (which is not necessarily full column rank).",
author = "Kwangjin Jeong and Masahiro Yukawa and Amari, {Shun Ichi}",
year = "2014",
doi = "10.1109/TIT.2014.2312723",
language = "English",
volume = "60",
pages = "2960--2968",
journal = "IEEE Transactions on Information Theory",
issn = "0018-9448",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "5",

}

TY - JOUR

T1 - Can critical-point paths under ℓp-regularization (0 < p < 1) reach the sparsest least squares solutions?

AU - Jeong, Kwangjin

AU - Yukawa, Masahiro

AU - Amari, Shun Ichi

PY - 2014

Y1 - 2014

N2 - The solution path of the least square problem under ℓp- regularization (0p-regularized squared error function. Based on this fact, it is proven that every sparsest least square solution of an underdetermined system is connected with the origin by a critical-point path. The existence theorem holds more generally for any least square solution whose support has its associated submatrix of the fat sensing matrix be full column rank. This is a sufficient condition for the existence, and allows to reduce the underdetermined problem to an overdetermined one with the off-support variable(s) nullified. A necessary condition is that the gradient of the ℓp regularizer with respect to the support variables lies in the row space of the submatrix (which is not necessarily full column rank).

AB - The solution path of the least square problem under ℓp- regularization (0p-regularized squared error function. Based on this fact, it is proven that every sparsest least square solution of an underdetermined system is connected with the origin by a critical-point path. The existence theorem holds more generally for any least square solution whose support has its associated submatrix of the fat sensing matrix be full column rank. This is a sufficient condition for the existence, and allows to reduce the underdetermined problem to an overdetermined one with the off-support variable(s) nullified. A necessary condition is that the gradient of the ℓp regularizer with respect to the support variables lies in the row space of the submatrix (which is not necessarily full column rank).

UR - http://www.scopus.com/inward/record.url?scp=84899654836&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84899654836&partnerID=8YFLogxK

U2 - 10.1109/TIT.2014.2312723

DO - 10.1109/TIT.2014.2312723

M3 - Article

AN - SCOPUS:84899654836

VL - 60

SP - 2960

EP - 2968

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

SN - 0018-9448

IS - 5

M1 - 6776531

ER -