The power-integrated discriminant improvement

An accurate measure of the incremental predictive value of additional biomarkers

Kenichi Hayashi, Shinto Eguchi

Research output: Contribution to journalArticle

Abstract

The predictive performance of biomarkers is a central concern in biomedical research. This is often evaluated by comparing two statistical models: a “new” model incorporating additional biomarkers and an “old” model without them. In 2008, the integrated discrimination improvement (IDI) was proposed for cases when the response variable is binary, and it is now widely applied as a promising alternative to conventional measures, such as the difference of the area under the receiver operating characteristic curve. However, the IDI can erroneously identify a significant improvement in the new model even if no additional information has been provided by new biomarkers. In order to overcome problems with existing measures, in this study, we propose the power-IDI as a measure of incremental predictive value. Our study explains why the IDI cannot avoid false detection of apparent improvements in a new model and we show that our proposed measure is better able to capture improvements in prediction. Numerical simulations and examples using real empirical data reveal that the power-IDI is not only more powerful but also incurs fewer false detections of improvement.

Original languageEnglish
JournalStatistics in Medicine
DOIs
Publication statusPublished - 2019 Jan 1

Fingerprint

Biomarkers
Discriminant
Discrimination
Statistical Models
ROC Curve
Biomedical Research
Receiver Operating Characteristic Curve
Model
Statistical Model
Binary
Numerical Simulation
Numerical Examples
Prediction
Alternatives

Keywords

  • area under the ROC curve
  • Bayes-risk consistency
  • fisher consistency
  • integrated discrimination improvement
  • logistic regression
  • net reclassification index

ASJC Scopus subject areas

  • Epidemiology
  • Statistics and Probability

Cite this

The power-integrated discriminant improvement : An accurate measure of the incremental predictive value of additional biomarkers. / Hayashi, Kenichi; Eguchi, Shinto.

In: Statistics in Medicine, 01.01.2019.

Research output: Contribution to journalArticle

@article{a45c80bb8e07428f99dfc09a0ccfbb79,
title = "The power-integrated discriminant improvement: An accurate measure of the incremental predictive value of additional biomarkers",
abstract = "The predictive performance of biomarkers is a central concern in biomedical research. This is often evaluated by comparing two statistical models: a “new” model incorporating additional biomarkers and an “old” model without them. In 2008, the integrated discrimination improvement (IDI) was proposed for cases when the response variable is binary, and it is now widely applied as a promising alternative to conventional measures, such as the difference of the area under the receiver operating characteristic curve. However, the IDI can erroneously identify a significant improvement in the new model even if no additional information has been provided by new biomarkers. In order to overcome problems with existing measures, in this study, we propose the power-IDI as a measure of incremental predictive value. Our study explains why the IDI cannot avoid false detection of apparent improvements in a new model and we show that our proposed measure is better able to capture improvements in prediction. Numerical simulations and examples using real empirical data reveal that the power-IDI is not only more powerful but also incurs fewer false detections of improvement.",
keywords = "area under the ROC curve, Bayes-risk consistency, fisher consistency, integrated discrimination improvement, logistic regression, net reclassification index",
author = "Kenichi Hayashi and Shinto Eguchi",
year = "2019",
month = "1",
day = "1",
doi = "10.1002/sim.8135",
language = "English",
journal = "Statistics in Medicine",
issn = "0277-6715",
publisher = "John Wiley and Sons Ltd",

}

TY - JOUR

T1 - The power-integrated discriminant improvement

T2 - An accurate measure of the incremental predictive value of additional biomarkers

AU - Hayashi, Kenichi

AU - Eguchi, Shinto

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The predictive performance of biomarkers is a central concern in biomedical research. This is often evaluated by comparing two statistical models: a “new” model incorporating additional biomarkers and an “old” model without them. In 2008, the integrated discrimination improvement (IDI) was proposed for cases when the response variable is binary, and it is now widely applied as a promising alternative to conventional measures, such as the difference of the area under the receiver operating characteristic curve. However, the IDI can erroneously identify a significant improvement in the new model even if no additional information has been provided by new biomarkers. In order to overcome problems with existing measures, in this study, we propose the power-IDI as a measure of incremental predictive value. Our study explains why the IDI cannot avoid false detection of apparent improvements in a new model and we show that our proposed measure is better able to capture improvements in prediction. Numerical simulations and examples using real empirical data reveal that the power-IDI is not only more powerful but also incurs fewer false detections of improvement.

AB - The predictive performance of biomarkers is a central concern in biomedical research. This is often evaluated by comparing two statistical models: a “new” model incorporating additional biomarkers and an “old” model without them. In 2008, the integrated discrimination improvement (IDI) was proposed for cases when the response variable is binary, and it is now widely applied as a promising alternative to conventional measures, such as the difference of the area under the receiver operating characteristic curve. However, the IDI can erroneously identify a significant improvement in the new model even if no additional information has been provided by new biomarkers. In order to overcome problems with existing measures, in this study, we propose the power-IDI as a measure of incremental predictive value. Our study explains why the IDI cannot avoid false detection of apparent improvements in a new model and we show that our proposed measure is better able to capture improvements in prediction. Numerical simulations and examples using real empirical data reveal that the power-IDI is not only more powerful but also incurs fewer false detections of improvement.

KW - area under the ROC curve

KW - Bayes-risk consistency

KW - fisher consistency

KW - integrated discrimination improvement

KW - logistic regression

KW - net reclassification index

UR - http://www.scopus.com/inward/record.url?scp=85062767081&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062767081&partnerID=8YFLogxK

U2 - 10.1002/sim.8135

DO - 10.1002/sim.8135

M3 - Article

JO - Statistics in Medicine

JF - Statistics in Medicine

SN - 0277-6715

ER -