Nonlinear logistic discrimination via regularized radial basis functions for classifying high-dimensional data

Tomohiro Ando, Sadanori Konishi

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

A flexible nonparametric method is proposed for classifying high- dimensional data with a complex structure. The proposed method can be regarded as an extended version of linear logistic discriminant procedures, in which the linear predictor is replaced by a radial-basis-expansion predictor. Radial basis functions with a hyperparameter are used to take the information on covariates and class labels into account; this was nearly impossible within the previously proposed hybrid learning framework. The penalized maximum likelihood estimation procedure is employed to obtain stable parameter estimates. A crucial issue in the model-construction process is the choice of a suitable model from candidates. This issue is examined from information-theoretic and Bayesian viewpoints and we employed Ando et al. (Japanese Journal of Applied Statistics, 31, 123-139, 2002)'s model evaluation criteria. The proposed method is available not only for the high-dimensional data but also for the variable selection problem. Real data analysis and Monte Carlo experiments show that our proposed method performs well in classifying future observations in practical situations. The simulation results also show that the use of the hyperparameter in the basis functions improves the prediction performance.

Original languageEnglish
Pages (from-to)331-353
Number of pages23
JournalAnnals of the Institute of Statistical Mathematics
Volume61
Issue number2
DOIs
Publication statusPublished - 2009 Jun

Fingerprint

High-dimensional Data
Radial Functions
Logistics
Discrimination
Basis Functions
Hyperparameters
Predictors
Penalized Maximum Likelihood
Hybrid Learning
Model Evaluation
Monte Carlo Experiment
Nonparametric Methods
Performance Prediction
Variable Selection
Complex Structure
Discriminant
Maximum Likelihood Estimation
Covariates
Data analysis
Statistics

Keywords

  • Bayes approach
  • Information criteria
  • Maximum penalized likelihood method
  • Radial basis functions

ASJC Scopus subject areas

  • Statistics and Probability

Cite this

Nonlinear logistic discrimination via regularized radial basis functions for classifying high-dimensional data. / Ando, Tomohiro; Konishi, Sadanori.

In: Annals of the Institute of Statistical Mathematics, Vol. 61, No. 2, 06.2009, p. 331-353.

Research output: Contribution to journalArticle

@article{557ca993c3cf4b738bb92368c72c8144,
title = "Nonlinear logistic discrimination via regularized radial basis functions for classifying high-dimensional data",
abstract = "A flexible nonparametric method is proposed for classifying high- dimensional data with a complex structure. The proposed method can be regarded as an extended version of linear logistic discriminant procedures, in which the linear predictor is replaced by a radial-basis-expansion predictor. Radial basis functions with a hyperparameter are used to take the information on covariates and class labels into account; this was nearly impossible within the previously proposed hybrid learning framework. The penalized maximum likelihood estimation procedure is employed to obtain stable parameter estimates. A crucial issue in the model-construction process is the choice of a suitable model from candidates. This issue is examined from information-theoretic and Bayesian viewpoints and we employed Ando et al. (Japanese Journal of Applied Statistics, 31, 123-139, 2002)'s model evaluation criteria. The proposed method is available not only for the high-dimensional data but also for the variable selection problem. Real data analysis and Monte Carlo experiments show that our proposed method performs well in classifying future observations in practical situations. The simulation results also show that the use of the hyperparameter in the basis functions improves the prediction performance.",
keywords = "Bayes approach, Information criteria, Maximum penalized likelihood method, Radial basis functions",
author = "Tomohiro Ando and Sadanori Konishi",
year = "2009",
month = "6",
doi = "10.1007/s10463-007-0143-3",
language = "English",
volume = "61",
pages = "331--353",
journal = "Annals of the Institute of Statistical Mathematics",
issn = "0020-3157",
publisher = "Springer Netherlands",
number = "2",

}

TY - JOUR

T1 - Nonlinear logistic discrimination via regularized radial basis functions for classifying high-dimensional data

AU - Ando, Tomohiro

AU - Konishi, Sadanori

PY - 2009/6

Y1 - 2009/6

N2 - A flexible nonparametric method is proposed for classifying high- dimensional data with a complex structure. The proposed method can be regarded as an extended version of linear logistic discriminant procedures, in which the linear predictor is replaced by a radial-basis-expansion predictor. Radial basis functions with a hyperparameter are used to take the information on covariates and class labels into account; this was nearly impossible within the previously proposed hybrid learning framework. The penalized maximum likelihood estimation procedure is employed to obtain stable parameter estimates. A crucial issue in the model-construction process is the choice of a suitable model from candidates. This issue is examined from information-theoretic and Bayesian viewpoints and we employed Ando et al. (Japanese Journal of Applied Statistics, 31, 123-139, 2002)'s model evaluation criteria. The proposed method is available not only for the high-dimensional data but also for the variable selection problem. Real data analysis and Monte Carlo experiments show that our proposed method performs well in classifying future observations in practical situations. The simulation results also show that the use of the hyperparameter in the basis functions improves the prediction performance.

AB - A flexible nonparametric method is proposed for classifying high- dimensional data with a complex structure. The proposed method can be regarded as an extended version of linear logistic discriminant procedures, in which the linear predictor is replaced by a radial-basis-expansion predictor. Radial basis functions with a hyperparameter are used to take the information on covariates and class labels into account; this was nearly impossible within the previously proposed hybrid learning framework. The penalized maximum likelihood estimation procedure is employed to obtain stable parameter estimates. A crucial issue in the model-construction process is the choice of a suitable model from candidates. This issue is examined from information-theoretic and Bayesian viewpoints and we employed Ando et al. (Japanese Journal of Applied Statistics, 31, 123-139, 2002)'s model evaluation criteria. The proposed method is available not only for the high-dimensional data but also for the variable selection problem. Real data analysis and Monte Carlo experiments show that our proposed method performs well in classifying future observations in practical situations. The simulation results also show that the use of the hyperparameter in the basis functions improves the prediction performance.

KW - Bayes approach

KW - Information criteria

KW - Maximum penalized likelihood method

KW - Radial basis functions

UR - http://www.scopus.com/inward/record.url?scp=67349126855&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67349126855&partnerID=8YFLogxK

U2 - 10.1007/s10463-007-0143-3

DO - 10.1007/s10463-007-0143-3

M3 - Article

AN - SCOPUS:67349126855

VL - 61

SP - 331

EP - 353

JO - Annals of the Institute of Statistical Mathematics

JF - Annals of the Institute of Statistical Mathematics

SN - 0020-3157

IS - 2

ER -