## Abstract

This paper elucidates the underlying structures of ℓ_{p}-regularized least squares problems in the nonconvex case of 0 < p < 1. The difference between two formulations is highlighted (which does not occur in the convex case of p = 1): 1) an ℓ_{p} -constrained optimization (P^{p}_{c}) and 2) an ℓ_{p}-penalized (unconstrained) optimization (L_{λ} ^{p}). It is shown that the solution path of (L_{λ}^{p}) is discontinuous and also a part of the solution path of (P^{p}_{c}). As an alternative to the solution path, a critical path is considered, which is a maximal continuous curve consisting of critical points. Critical paths are piecewise smooth, as can be seen from the viewpoint of the variational method, and generally contain non-optimal points, such as saddle points and local maxima as well as global/local minima. Our study reveals multiplicity (non-monotonicity) in the correspondence between the regularization parameters of (P^{p}_{c}) and (L_{λ}^{p}). Two particular paths of critical points connecting the origin and an ordinary least squares (OLS) solution are studied further. One is a main path starting at an OLS solution, and the other is a greedy path starting at the origin. Part of the greedy path can be constructed with a generalized Minkowskian gradient. This paper of greedy path leads to a nontrivial close-link between the optimization problem of ℓ_{p} -regularized least squares and the greedy method of orthogonal matching pursuit.

Original language | English |
---|---|

Article number | 7330004 |

Pages (from-to) | 488-502 |

Number of pages | 15 |

Journal | IEEE Transactions on Information Theory |

Volume | 62 |

Issue number | 1 |

DOIs | |

Publication status | Published - 2016 Jan 1 |

## Keywords

- Critical points
- LARS
- Nonconvex optimization
- Sparse solution
- ℓ quasi-norm (0 < p < 1)

## ASJC Scopus subject areas

- Information Systems
- Computer Science Applications
- Library and Information Sciences