Online model-selection and learning for nonlinear estimation based on multikernel adaptive filtering

Osamu Toda, Masahiro Yukawa

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

We study a use of Gaussian kernels with a wide range of scales for nonlinear function estimation. The estimation task can then be split into two sub-tasks: (i) model selection and (ii) learning (parameter estimation) under the selected model. We propose a fully-adaptive and all-in-one scheme that jointly carries out the two sub-tasks based on the multikernel adaptive filtering framework. The task is cast as an asymptotic minimization problem of an instantaneous fidelity function penalized by two types of block l1-norm regularizers. Those regularizers enhance the sparsity of the solution in two different block structures, leading to effi- cient model selection and dictionary refinement. The adaptive generalized forward-backward splitting method is derived to deal with the asymptotic minimization problem. Numerical examples show that the scheme achieves the model selection and learning simultaneously, and demonstrate its strik- ing advantages over the multiple kernel learning (MKL) method called SimpleMKL.

Original languageEnglish
Pages (from-to)236-250
Number of pages15
JournalIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
VolumeE100A
Issue number1
DOIs
Publication statusPublished - 2017 Jan 1

Fingerprint

Nonlinear Estimation
Adaptive Filtering
Adaptive filtering
Model Selection
Minimization Problem
Block Structure
Gaussian Kernel
Function Estimation
L1-norm
Splitting Method
Sparsity
Nonlinear Function
Fidelity
Instantaneous
Parameter Estimation
Refinement
Glossaries
Parameter estimation
kernel
Numerical Examples

Keywords

  • Adaptive filter
  • Convex projection
  • Proximity operator
  • Reproducing kernels

ASJC Scopus subject areas

  • Signal Processing
  • Computer Graphics and Computer-Aided Design
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this

@article{8ea4cdeec73e45d8b297a3b8c8fea3d1,
title = "Online model-selection and learning for nonlinear estimation based on multikernel adaptive filtering",
abstract = "We study a use of Gaussian kernels with a wide range of scales for nonlinear function estimation. The estimation task can then be split into two sub-tasks: (i) model selection and (ii) learning (parameter estimation) under the selected model. We propose a fully-adaptive and all-in-one scheme that jointly carries out the two sub-tasks based on the multikernel adaptive filtering framework. The task is cast as an asymptotic minimization problem of an instantaneous fidelity function penalized by two types of block l1-norm regularizers. Those regularizers enhance the sparsity of the solution in two different block structures, leading to effi- cient model selection and dictionary refinement. The adaptive generalized forward-backward splitting method is derived to deal with the asymptotic minimization problem. Numerical examples show that the scheme achieves the model selection and learning simultaneously, and demonstrate its strik- ing advantages over the multiple kernel learning (MKL) method called SimpleMKL.",
keywords = "Adaptive filter, Convex projection, Proximity operator, Reproducing kernels",
author = "Osamu Toda and Masahiro Yukawa",
year = "2017",
month = "1",
day = "1",
doi = "10.1587/transfun.E100.A.236",
language = "English",
volume = "E100A",
pages = "236--250",
journal = "IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences",
issn = "0916-8508",
publisher = "Maruzen Co., Ltd/Maruzen Kabushikikaisha",
number = "1",

}

TY - JOUR

T1 - Online model-selection and learning for nonlinear estimation based on multikernel adaptive filtering

AU - Toda, Osamu

AU - Yukawa, Masahiro

PY - 2017/1/1

Y1 - 2017/1/1

N2 - We study a use of Gaussian kernels with a wide range of scales for nonlinear function estimation. The estimation task can then be split into two sub-tasks: (i) model selection and (ii) learning (parameter estimation) under the selected model. We propose a fully-adaptive and all-in-one scheme that jointly carries out the two sub-tasks based on the multikernel adaptive filtering framework. The task is cast as an asymptotic minimization problem of an instantaneous fidelity function penalized by two types of block l1-norm regularizers. Those regularizers enhance the sparsity of the solution in two different block structures, leading to effi- cient model selection and dictionary refinement. The adaptive generalized forward-backward splitting method is derived to deal with the asymptotic minimization problem. Numerical examples show that the scheme achieves the model selection and learning simultaneously, and demonstrate its strik- ing advantages over the multiple kernel learning (MKL) method called SimpleMKL.

AB - We study a use of Gaussian kernels with a wide range of scales for nonlinear function estimation. The estimation task can then be split into two sub-tasks: (i) model selection and (ii) learning (parameter estimation) under the selected model. We propose a fully-adaptive and all-in-one scheme that jointly carries out the two sub-tasks based on the multikernel adaptive filtering framework. The task is cast as an asymptotic minimization problem of an instantaneous fidelity function penalized by two types of block l1-norm regularizers. Those regularizers enhance the sparsity of the solution in two different block structures, leading to effi- cient model selection and dictionary refinement. The adaptive generalized forward-backward splitting method is derived to deal with the asymptotic minimization problem. Numerical examples show that the scheme achieves the model selection and learning simultaneously, and demonstrate its strik- ing advantages over the multiple kernel learning (MKL) method called SimpleMKL.

KW - Adaptive filter

KW - Convex projection

KW - Proximity operator

KW - Reproducing kernels

UR - http://www.scopus.com/inward/record.url?scp=85008311763&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85008311763&partnerID=8YFLogxK

U2 - 10.1587/transfun.E100.A.236

DO - 10.1587/transfun.E100.A.236

M3 - Article

AN - SCOPUS:85008311763

VL - E100A

SP - 236

EP - 250

JO - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

JF - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

SN - 0916-8508

IS - 1

ER -