A robust ensemble learning using zero-one loss function

Natsuki Sano, Hideo Suzuki, Masato Koda

Research output: Contribution to journalArticle

Abstract

Classifier is used for pattern recognition in various fields including data mining. Boosting is an ensemble learning method to boost (enhance) an accuracy of single classifier. We propose a new, robust boosting method by using a zero-one step function as a loss function. In deriving the method, the MarginBoost technique is blended with the stochastic gradient approximation algorithm, called Stochastic Noise Reaction (SNR). Based on intensive numerical experiments, we show that the proposed method is actually better than AdaBoost on test error rates in the case of noisy, mislabeled situation.

Original languageEnglish
Pages (from-to)95-110
Number of pages16
JournalJournal of the Operations Research Society of Japan
Volume51
Issue number1
Publication statusPublished - 2008 Mar
Externally publishedYes

Fingerprint

Loss function
Ensemble learning
Boosting
Classifier
Approximation algorithms
Gradient
Learning methods
Data mining
Numerical experiment
Pattern recognition

Keywords

  • AdaBoost
  • Data Analysis
  • Data mining
  • Stochastic noise reaction
  • Zero-one loss function

ASJC Scopus subject areas

  • Management Science and Operations Research
  • Decision Sciences(all)

Cite this

A robust ensemble learning using zero-one loss function. / Sano, Natsuki; Suzuki, Hideo; Koda, Masato.

In: Journal of the Operations Research Society of Japan, Vol. 51, No. 1, 03.2008, p. 95-110.

Research output: Contribution to journalArticle

@article{2a471c74a14c47688b4c15fad8bbe308,
title = "A robust ensemble learning using zero-one loss function",
abstract = "Classifier is used for pattern recognition in various fields including data mining. Boosting is an ensemble learning method to boost (enhance) an accuracy of single classifier. We propose a new, robust boosting method by using a zero-one step function as a loss function. In deriving the method, the MarginBoost technique is blended with the stochastic gradient approximation algorithm, called Stochastic Noise Reaction (SNR). Based on intensive numerical experiments, we show that the proposed method is actually better than AdaBoost on test error rates in the case of noisy, mislabeled situation.",
keywords = "AdaBoost, Data Analysis, Data mining, Stochastic noise reaction, Zero-one loss function",
author = "Natsuki Sano and Hideo Suzuki and Masato Koda",
year = "2008",
month = "3",
language = "English",
volume = "51",
pages = "95--110",
journal = "Journal of the Operations Research Society of Japan",
issn = "0453-4514",
publisher = "Operations Research Society of Japan",
number = "1",

}

TY - JOUR

T1 - A robust ensemble learning using zero-one loss function

AU - Sano, Natsuki

AU - Suzuki, Hideo

AU - Koda, Masato

PY - 2008/3

Y1 - 2008/3

N2 - Classifier is used for pattern recognition in various fields including data mining. Boosting is an ensemble learning method to boost (enhance) an accuracy of single classifier. We propose a new, robust boosting method by using a zero-one step function as a loss function. In deriving the method, the MarginBoost technique is blended with the stochastic gradient approximation algorithm, called Stochastic Noise Reaction (SNR). Based on intensive numerical experiments, we show that the proposed method is actually better than AdaBoost on test error rates in the case of noisy, mislabeled situation.

AB - Classifier is used for pattern recognition in various fields including data mining. Boosting is an ensemble learning method to boost (enhance) an accuracy of single classifier. We propose a new, robust boosting method by using a zero-one step function as a loss function. In deriving the method, the MarginBoost technique is blended with the stochastic gradient approximation algorithm, called Stochastic Noise Reaction (SNR). Based on intensive numerical experiments, we show that the proposed method is actually better than AdaBoost on test error rates in the case of noisy, mislabeled situation.

KW - AdaBoost

KW - Data Analysis

KW - Data mining

KW - Stochastic noise reaction

KW - Zero-one loss function

UR - http://www.scopus.com/inward/record.url?scp=44649181131&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=44649181131&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:44649181131

VL - 51

SP - 95

EP - 110

JO - Journal of the Operations Research Society of Japan

JF - Journal of the Operations Research Society of Japan

SN - 0453-4514

IS - 1

ER -