Automated Assessment of Existing Patient's Revised Cardiac Risk Index Using Algorithmic Software

Ira S. Hofer, Drew Cheng, Tristan Grogan, Yohei Fujimoto, Takashige Yamada, Lauren Beck, Maxime Cannesson, Aman Mahajan

Research output: Contribution to journalArticle

Abstract

BACKGROUND: Previous work in the field of medical informatics has shown that rules-based algorithms can be created to identify patients with various medical conditions; however, these techniques have not been compared to actual clinician notes nor has the ability to predict complications been tested. We hypothesize that a rules-based algorithm can successfully identify patients with the diseases in the Revised Cardiac Risk Index (RCRI). METHODS: Patients undergoing surgery at the University of California, Los Angeles Health System between April 1, 2013 and July 1, 2016 and who had at least 2 previous office visits were included. For each disease in the RCRI except renal failure-congestive heart failure, ischemic heart disease, cerebrovascular disease, and diabetes mellitus-diagnosis algorithms were created based on diagnostic and standard clinical treatment criteria. For each disease state, the prevalence of the disease as determined by the algorithm, International Classification of Disease (ICD) code, and anesthesiologist's preoperative note were determined. Additionally, 400 American Society of Anesthesiologists classes III and IV cases were randomly chosen for manual review by an anesthesiologist. The sensitivity, specificity, accuracy, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve were determined using the manual review as a gold standard. Last, the ability of the RCRI as calculated by each of the methods to predict in-hospital mortality was determined, and the time necessary to run the algorithms was calculated. RESULTS: A total of 64,151 patients met inclusion criteria for the study. In general, the incidence of definite or likely disease determined by the algorithms was higher than that detected by the anesthesiologist. Additionally, in all disease states, the prevalence of disease was always lowest for the ICD codes, followed by the preoperative note, followed by the algorithms. In the subset of patients for whom the records were manually reviewed, the algorithms were generally the most sensitive and the ICD codes the most specific. When computing the modified RCRI using each of the methods, the modified RCRI from the algorithms predicted in-hospital mortality with an area under the receiver operating characteristic curve of 0.70 (0.67-0.73), which compared to 0.70 (0.67-0.72) for ICD codes and 0.64 (0.61-0.67) for the preoperative note. On average, the algorithms took 12.64 ± 1.20 minutes to run on 1.4 million patients. CONCLUSIONS: Rules-based algorithms for disease in the RCRI can be created that perform with a similar discriminative ability as compared to physician notes and ICD codes but with significantly increased economies of scale.

Original languageEnglish
Pages (from-to)909-916
Number of pages8
JournalAnesthesia and analgesia
Volume128
Issue number5
DOIs
Publication statusPublished - 2019 May 1

Fingerprint

Software
International Classification of Diseases
Hospital Mortality
ROC Curve
Cerebrovascular Disorders
Office Visits
Medical Informatics
Los Angeles
Renal Insufficiency
Myocardial Ischemia
Diabetes Mellitus
Heart Failure
Physicians
Sensitivity and Specificity
Incidence
Health
Anesthesiologists

ASJC Scopus subject areas

  • Anesthesiology and Pain Medicine

Cite this

Automated Assessment of Existing Patient's Revised Cardiac Risk Index Using Algorithmic Software. / Hofer, Ira S.; Cheng, Drew; Grogan, Tristan; Fujimoto, Yohei; Yamada, Takashige; Beck, Lauren; Cannesson, Maxime; Mahajan, Aman.

In: Anesthesia and analgesia, Vol. 128, No. 5, 01.05.2019, p. 909-916.

Research output: Contribution to journalArticle

Hofer, IS, Cheng, D, Grogan, T, Fujimoto, Y, Yamada, T, Beck, L, Cannesson, M & Mahajan, A 2019, 'Automated Assessment of Existing Patient's Revised Cardiac Risk Index Using Algorithmic Software', Anesthesia and analgesia, vol. 128, no. 5, pp. 909-916. https://doi.org/10.1213/ANE.0000000000003440
Hofer, Ira S. ; Cheng, Drew ; Grogan, Tristan ; Fujimoto, Yohei ; Yamada, Takashige ; Beck, Lauren ; Cannesson, Maxime ; Mahajan, Aman. / Automated Assessment of Existing Patient's Revised Cardiac Risk Index Using Algorithmic Software. In: Anesthesia and analgesia. 2019 ; Vol. 128, No. 5. pp. 909-916.
@article{762d7c97e3f940c1b9ba10011f4ec90b,
title = "Automated Assessment of Existing Patient's Revised Cardiac Risk Index Using Algorithmic Software",
abstract = "BACKGROUND: Previous work in the field of medical informatics has shown that rules-based algorithms can be created to identify patients with various medical conditions; however, these techniques have not been compared to actual clinician notes nor has the ability to predict complications been tested. We hypothesize that a rules-based algorithm can successfully identify patients with the diseases in the Revised Cardiac Risk Index (RCRI). METHODS: Patients undergoing surgery at the University of California, Los Angeles Health System between April 1, 2013 and July 1, 2016 and who had at least 2 previous office visits were included. For each disease in the RCRI except renal failure-congestive heart failure, ischemic heart disease, cerebrovascular disease, and diabetes mellitus-diagnosis algorithms were created based on diagnostic and standard clinical treatment criteria. For each disease state, the prevalence of the disease as determined by the algorithm, International Classification of Disease (ICD) code, and anesthesiologist's preoperative note were determined. Additionally, 400 American Society of Anesthesiologists classes III and IV cases were randomly chosen for manual review by an anesthesiologist. The sensitivity, specificity, accuracy, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve were determined using the manual review as a gold standard. Last, the ability of the RCRI as calculated by each of the methods to predict in-hospital mortality was determined, and the time necessary to run the algorithms was calculated. RESULTS: A total of 64,151 patients met inclusion criteria for the study. In general, the incidence of definite or likely disease determined by the algorithms was higher than that detected by the anesthesiologist. Additionally, in all disease states, the prevalence of disease was always lowest for the ICD codes, followed by the preoperative note, followed by the algorithms. In the subset of patients for whom the records were manually reviewed, the algorithms were generally the most sensitive and the ICD codes the most specific. When computing the modified RCRI using each of the methods, the modified RCRI from the algorithms predicted in-hospital mortality with an area under the receiver operating characteristic curve of 0.70 (0.67-0.73), which compared to 0.70 (0.67-0.72) for ICD codes and 0.64 (0.61-0.67) for the preoperative note. On average, the algorithms took 12.64 ± 1.20 minutes to run on 1.4 million patients. CONCLUSIONS: Rules-based algorithms for disease in the RCRI can be created that perform with a similar discriminative ability as compared to physician notes and ICD codes but with significantly increased economies of scale.",
author = "Hofer, {Ira S.} and Drew Cheng and Tristan Grogan and Yohei Fujimoto and Takashige Yamada and Lauren Beck and Maxime Cannesson and Aman Mahajan",
year = "2019",
month = "5",
day = "1",
doi = "10.1213/ANE.0000000000003440",
language = "English",
volume = "128",
pages = "909--916",
journal = "Anesthesia and Analgesia",
issn = "0003-2999",
publisher = "Lippincott Williams and Wilkins",
number = "5",

}

TY - JOUR

T1 - Automated Assessment of Existing Patient's Revised Cardiac Risk Index Using Algorithmic Software

AU - Hofer, Ira S.

AU - Cheng, Drew

AU - Grogan, Tristan

AU - Fujimoto, Yohei

AU - Yamada, Takashige

AU - Beck, Lauren

AU - Cannesson, Maxime

AU - Mahajan, Aman

PY - 2019/5/1

Y1 - 2019/5/1

N2 - BACKGROUND: Previous work in the field of medical informatics has shown that rules-based algorithms can be created to identify patients with various medical conditions; however, these techniques have not been compared to actual clinician notes nor has the ability to predict complications been tested. We hypothesize that a rules-based algorithm can successfully identify patients with the diseases in the Revised Cardiac Risk Index (RCRI). METHODS: Patients undergoing surgery at the University of California, Los Angeles Health System between April 1, 2013 and July 1, 2016 and who had at least 2 previous office visits were included. For each disease in the RCRI except renal failure-congestive heart failure, ischemic heart disease, cerebrovascular disease, and diabetes mellitus-diagnosis algorithms were created based on diagnostic and standard clinical treatment criteria. For each disease state, the prevalence of the disease as determined by the algorithm, International Classification of Disease (ICD) code, and anesthesiologist's preoperative note were determined. Additionally, 400 American Society of Anesthesiologists classes III and IV cases were randomly chosen for manual review by an anesthesiologist. The sensitivity, specificity, accuracy, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve were determined using the manual review as a gold standard. Last, the ability of the RCRI as calculated by each of the methods to predict in-hospital mortality was determined, and the time necessary to run the algorithms was calculated. RESULTS: A total of 64,151 patients met inclusion criteria for the study. In general, the incidence of definite or likely disease determined by the algorithms was higher than that detected by the anesthesiologist. Additionally, in all disease states, the prevalence of disease was always lowest for the ICD codes, followed by the preoperative note, followed by the algorithms. In the subset of patients for whom the records were manually reviewed, the algorithms were generally the most sensitive and the ICD codes the most specific. When computing the modified RCRI using each of the methods, the modified RCRI from the algorithms predicted in-hospital mortality with an area under the receiver operating characteristic curve of 0.70 (0.67-0.73), which compared to 0.70 (0.67-0.72) for ICD codes and 0.64 (0.61-0.67) for the preoperative note. On average, the algorithms took 12.64 ± 1.20 minutes to run on 1.4 million patients. CONCLUSIONS: Rules-based algorithms for disease in the RCRI can be created that perform with a similar discriminative ability as compared to physician notes and ICD codes but with significantly increased economies of scale.

AB - BACKGROUND: Previous work in the field of medical informatics has shown that rules-based algorithms can be created to identify patients with various medical conditions; however, these techniques have not been compared to actual clinician notes nor has the ability to predict complications been tested. We hypothesize that a rules-based algorithm can successfully identify patients with the diseases in the Revised Cardiac Risk Index (RCRI). METHODS: Patients undergoing surgery at the University of California, Los Angeles Health System between April 1, 2013 and July 1, 2016 and who had at least 2 previous office visits were included. For each disease in the RCRI except renal failure-congestive heart failure, ischemic heart disease, cerebrovascular disease, and diabetes mellitus-diagnosis algorithms were created based on diagnostic and standard clinical treatment criteria. For each disease state, the prevalence of the disease as determined by the algorithm, International Classification of Disease (ICD) code, and anesthesiologist's preoperative note were determined. Additionally, 400 American Society of Anesthesiologists classes III and IV cases were randomly chosen for manual review by an anesthesiologist. The sensitivity, specificity, accuracy, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve were determined using the manual review as a gold standard. Last, the ability of the RCRI as calculated by each of the methods to predict in-hospital mortality was determined, and the time necessary to run the algorithms was calculated. RESULTS: A total of 64,151 patients met inclusion criteria for the study. In general, the incidence of definite or likely disease determined by the algorithms was higher than that detected by the anesthesiologist. Additionally, in all disease states, the prevalence of disease was always lowest for the ICD codes, followed by the preoperative note, followed by the algorithms. In the subset of patients for whom the records were manually reviewed, the algorithms were generally the most sensitive and the ICD codes the most specific. When computing the modified RCRI using each of the methods, the modified RCRI from the algorithms predicted in-hospital mortality with an area under the receiver operating characteristic curve of 0.70 (0.67-0.73), which compared to 0.70 (0.67-0.72) for ICD codes and 0.64 (0.61-0.67) for the preoperative note. On average, the algorithms took 12.64 ± 1.20 minutes to run on 1.4 million patients. CONCLUSIONS: Rules-based algorithms for disease in the RCRI can be created that perform with a similar discriminative ability as compared to physician notes and ICD codes but with significantly increased economies of scale.

UR - http://www.scopus.com/inward/record.url?scp=85064967850&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064967850&partnerID=8YFLogxK

U2 - 10.1213/ANE.0000000000003440

DO - 10.1213/ANE.0000000000003440

M3 - Article

C2 - 29847379

AN - SCOPUS:85064967850

VL - 128

SP - 909

EP - 916

JO - Anesthesia and Analgesia

JF - Anesthesia and Analgesia

SN - 0003-2999

IS - 5

ER -