Optimal regularization for system identification from noisy input and output signals

Jingmin Xin, Hiromitsu Ohmori, Akira Sano

Research output: Contribution to journalArticle

Abstract

In identification of a finite impulse response (FIR) model using noise-corrupted input and output data, the least squares type of estimation schemes such as the ordinary least squares (LS), the corrected least squares (CLS) and the total least squares (TLS) methods become often numerically unstable, when the true input signal to the system is strongly correlated. To overcome this ill-conditioned problem, we propose a regularized CLS estimation method by introducing multiple regularization parameters to minimize the mean squares error (MSE) of the regularized CLS estimate of the FIR model. The asymptotic MSE can be evaluated by considering the third and fourth order cross moments of the input and output measurement noises, and an analytical expression of the optimal regularization parameters minimizing the MSE is also clarified. Furthermore, an effective regularization algorithm is given by using the only accessible input-output data without using any true unknown parameters. The effectiveness of the proposed data-based regularization algorithm is demonstrated and compared with the ordinary LS, CLS and TLS estimates through numerical examples.

Original languageEnglish
Pages (from-to)1805-1815
Number of pages11
JournalIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
VolumeE78-A
Issue number12
Publication statusPublished - 1995 Dec

Fingerprint

System Identification
Mean square error
Total Least Squares
Least Squares
Regularization
Identification (control systems)
Least Squares Estimate
Ordinary Least Squares
Regularization Parameter
Impulse Response
Impulse response
Output
Least Squares Estimation
Optimal Parameter
Least Square Method
Unknown Parameters
Fourth Order
Unstable
Moment
Minimise

ASJC Scopus subject areas

  • Hardware and Architecture
  • Information Systems
  • Electrical and Electronic Engineering

Cite this

Optimal regularization for system identification from noisy input and output signals. / Xin, Jingmin; Ohmori, Hiromitsu; Sano, Akira.

In: IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Vol. E78-A, No. 12, 12.1995, p. 1805-1815.

Research output: Contribution to journalArticle

@article{5c395a0ed3214615ad5e2518b1228f31,
title = "Optimal regularization for system identification from noisy input and output signals",
abstract = "In identification of a finite impulse response (FIR) model using noise-corrupted input and output data, the least squares type of estimation schemes such as the ordinary least squares (LS), the corrected least squares (CLS) and the total least squares (TLS) methods become often numerically unstable, when the true input signal to the system is strongly correlated. To overcome this ill-conditioned problem, we propose a regularized CLS estimation method by introducing multiple regularization parameters to minimize the mean squares error (MSE) of the regularized CLS estimate of the FIR model. The asymptotic MSE can be evaluated by considering the third and fourth order cross moments of the input and output measurement noises, and an analytical expression of the optimal regularization parameters minimizing the MSE is also clarified. Furthermore, an effective regularization algorithm is given by using the only accessible input-output data without using any true unknown parameters. The effectiveness of the proposed data-based regularization algorithm is demonstrated and compared with the ordinary LS, CLS and TLS estimates through numerical examples.",
author = "Jingmin Xin and Hiromitsu Ohmori and Akira Sano",
year = "1995",
month = "12",
language = "English",
volume = "E78-A",
pages = "1805--1815",
journal = "IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences",
issn = "0916-8508",
publisher = "Maruzen Co., Ltd/Maruzen Kabushikikaisha",
number = "12",

}

TY - JOUR

T1 - Optimal regularization for system identification from noisy input and output signals

AU - Xin, Jingmin

AU - Ohmori, Hiromitsu

AU - Sano, Akira

PY - 1995/12

Y1 - 1995/12

N2 - In identification of a finite impulse response (FIR) model using noise-corrupted input and output data, the least squares type of estimation schemes such as the ordinary least squares (LS), the corrected least squares (CLS) and the total least squares (TLS) methods become often numerically unstable, when the true input signal to the system is strongly correlated. To overcome this ill-conditioned problem, we propose a regularized CLS estimation method by introducing multiple regularization parameters to minimize the mean squares error (MSE) of the regularized CLS estimate of the FIR model. The asymptotic MSE can be evaluated by considering the third and fourth order cross moments of the input and output measurement noises, and an analytical expression of the optimal regularization parameters minimizing the MSE is also clarified. Furthermore, an effective regularization algorithm is given by using the only accessible input-output data without using any true unknown parameters. The effectiveness of the proposed data-based regularization algorithm is demonstrated and compared with the ordinary LS, CLS and TLS estimates through numerical examples.

AB - In identification of a finite impulse response (FIR) model using noise-corrupted input and output data, the least squares type of estimation schemes such as the ordinary least squares (LS), the corrected least squares (CLS) and the total least squares (TLS) methods become often numerically unstable, when the true input signal to the system is strongly correlated. To overcome this ill-conditioned problem, we propose a regularized CLS estimation method by introducing multiple regularization parameters to minimize the mean squares error (MSE) of the regularized CLS estimate of the FIR model. The asymptotic MSE can be evaluated by considering the third and fourth order cross moments of the input and output measurement noises, and an analytical expression of the optimal regularization parameters minimizing the MSE is also clarified. Furthermore, an effective regularization algorithm is given by using the only accessible input-output data without using any true unknown parameters. The effectiveness of the proposed data-based regularization algorithm is demonstrated and compared with the ordinary LS, CLS and TLS estimates through numerical examples.

UR - http://www.scopus.com/inward/record.url?scp=0029509406&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0029509406&partnerID=8YFLogxK

M3 - Article

VL - E78-A

SP - 1805

EP - 1815

JO - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

JF - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

SN - 0916-8508

IS - 12

ER -