Inter-rater reliability of the BIOCHIP indirect immunofluorescence dermatology mosaic in bullous pemphigoid and pemphigus patients

A. Yang, R. R. Xuan, W. Melbourne, T. Hashimoto, S. Uzun, M. Daneshpazhooh, Jun Yamagami, G. Di Zenzo, J. M. Mascaro, H. Mahmoudi, A. Patsatsi, K. Drenovska, S. Vassileva, D. F. Murrell

Research output: Contribution to journalArticle

Abstract

Background: The BIOCHIP (Dermatology Mosaic 7; EUROIMMUN, Lubeck, Germany) is a novel multiplex indirect immunofluorescence (IIF) technique used in the serological diagnosis of bullous pemphigoid (BP) and pemphigus. Objective: To validate the accuracy and inter-rater reliability (IRR) of the BIOCHIP in the diagnosis of BP, pemphigus foliaceus (PF) and pemphigus vulgaris (PV). Methods: Sera from patients with BP (n = 38), PF (n = 8), PV (n = 23), control patients (n = 64) and healthy control volunteers (n = 39) were tested. Sera were collected and analysed during the course of the disease at 1–5 different time points. The BIOCHIP was performed for all patients, digital images were captured of each incubated field, and the images were shared with 10 dermatologists experienced in reading IF from around the world to report. There were 312 BIOCHIP slides consisting of 1872 photos in total. All patients were de-identified. Fleiss Kappa was used to estimate the IRR. Results: Fleiss Kappa was computed for each category (Oesophagus, Oesophagus immunofluorescence pattern, Salt-Split Skin (SSS), SSS immunofluorescence location, BP180, BP230, Dsg 1 and Ds3). The inter-rater agreement between the 10 raters varied between fair and moderate for all categories. Those that demonstrated fair concordance included monkey oesophagus (k = 0.257, P < 0.0001), oesophagus pattern (k = 0.357, P < 0.0001), Dsg1 (k = 0.390, P < 0.0001) and BP230 (k = 0.281, P < 0.0001). Moderate agreement was demonstrated for SSS (k = 0.416, P < 0.0001), SSS immunofluorescence location (k = 0.505, P < 0.0001), Dsg3 (k = 0.437, P < 0.0001) and BP180 (k = 0.559, P < 0.0001). Conclusion: The BIOCHIP mosaic-based immunofluorescence test is a simple, time and effort saving test that can aid in the diagnosis and screening of BP, PV and PF. However, the level of agreement was relatively low. The authors found the most common causes to be variable levels of training, indicating the presence of a learning curve in the interpretation of the results and ambiguous staining patterns leading to incongruent results.

Original languageEnglish
JournalJournal of the European Academy of Dermatology and Venereology
DOIs
Publication statusAccepted/In press - 2019 Jan 1

Fingerprint

Bullous Pemphigoid
Pemphigus
Indirect Fluorescent Antibody Technique
Dermatology
Esophagus
Fluorescent Antibody Technique
Salts
Skin
Learning Curve
Serum
Haplorhini
Germany
Reading
Healthy Volunteers
Staining and Labeling

ASJC Scopus subject areas

  • Dermatology
  • Infectious Diseases

Cite this

Inter-rater reliability of the BIOCHIP indirect immunofluorescence dermatology mosaic in bullous pemphigoid and pemphigus patients. / Yang, A.; Xuan, R. R.; Melbourne, W.; Hashimoto, T.; Uzun, S.; Daneshpazhooh, M.; Yamagami, Jun; Di Zenzo, G.; Mascaro, J. M.; Mahmoudi, H.; Patsatsi, A.; Drenovska, K.; Vassileva, S.; Murrell, D. F.

In: Journal of the European Academy of Dermatology and Venereology, 01.01.2019.

Research output: Contribution to journalArticle

Yang, A, Xuan, RR, Melbourne, W, Hashimoto, T, Uzun, S, Daneshpazhooh, M, Yamagami, J, Di Zenzo, G, Mascaro, JM, Mahmoudi, H, Patsatsi, A, Drenovska, K, Vassileva, S & Murrell, DF 2019, 'Inter-rater reliability of the BIOCHIP indirect immunofluorescence dermatology mosaic in bullous pemphigoid and pemphigus patients', Journal of the European Academy of Dermatology and Venereology. https://doi.org/10.1111/jdv.15817
Yang, A. ; Xuan, R. R. ; Melbourne, W. ; Hashimoto, T. ; Uzun, S. ; Daneshpazhooh, M. ; Yamagami, Jun ; Di Zenzo, G. ; Mascaro, J. M. ; Mahmoudi, H. ; Patsatsi, A. ; Drenovska, K. ; Vassileva, S. ; Murrell, D. F. / Inter-rater reliability of the BIOCHIP indirect immunofluorescence dermatology mosaic in bullous pemphigoid and pemphigus patients. In: Journal of the European Academy of Dermatology and Venereology. 2019.
@article{7450c768660a4170a6290dcc5dd107f6,
title = "Inter-rater reliability of the BIOCHIP indirect immunofluorescence dermatology mosaic in bullous pemphigoid and pemphigus patients",
abstract = "Background: The BIOCHIP (Dermatology Mosaic 7; EUROIMMUN, Lubeck, Germany) is a novel multiplex indirect immunofluorescence (IIF) technique used in the serological diagnosis of bullous pemphigoid (BP) and pemphigus. Objective: To validate the accuracy and inter-rater reliability (IRR) of the BIOCHIP in the diagnosis of BP, pemphigus foliaceus (PF) and pemphigus vulgaris (PV). Methods: Sera from patients with BP (n = 38), PF (n = 8), PV (n = 23), control patients (n = 64) and healthy control volunteers (n = 39) were tested. Sera were collected and analysed during the course of the disease at 1–5 different time points. The BIOCHIP was performed for all patients, digital images were captured of each incubated field, and the images were shared with 10 dermatologists experienced in reading IF from around the world to report. There were 312 BIOCHIP slides consisting of 1872 photos in total. All patients were de-identified. Fleiss Kappa was used to estimate the IRR. Results: Fleiss Kappa was computed for each category (Oesophagus, Oesophagus immunofluorescence pattern, Salt-Split Skin (SSS), SSS immunofluorescence location, BP180, BP230, Dsg 1 and Ds3). The inter-rater agreement between the 10 raters varied between fair and moderate for all categories. Those that demonstrated fair concordance included monkey oesophagus (k = 0.257, P < 0.0001), oesophagus pattern (k = 0.357, P < 0.0001), Dsg1 (k = 0.390, P < 0.0001) and BP230 (k = 0.281, P < 0.0001). Moderate agreement was demonstrated for SSS (k = 0.416, P < 0.0001), SSS immunofluorescence location (k = 0.505, P < 0.0001), Dsg3 (k = 0.437, P < 0.0001) and BP180 (k = 0.559, P < 0.0001). Conclusion: The BIOCHIP mosaic-based immunofluorescence test is a simple, time and effort saving test that can aid in the diagnosis and screening of BP, PV and PF. However, the level of agreement was relatively low. The authors found the most common causes to be variable levels of training, indicating the presence of a learning curve in the interpretation of the results and ambiguous staining patterns leading to incongruent results.",
author = "A. Yang and Xuan, {R. R.} and W. Melbourne and T. Hashimoto and S. Uzun and M. Daneshpazhooh and Jun Yamagami and {Di Zenzo}, G. and Mascaro, {J. M.} and H. Mahmoudi and A. Patsatsi and K. Drenovska and S. Vassileva and Murrell, {D. F.}",
year = "2019",
month = "1",
day = "1",
doi = "10.1111/jdv.15817",
language = "English",
journal = "Journal of the European Academy of Dermatology and Venereology",
issn = "0926-9959",
publisher = "Wiley-Blackwell",

}

TY - JOUR

T1 - Inter-rater reliability of the BIOCHIP indirect immunofluorescence dermatology mosaic in bullous pemphigoid and pemphigus patients

AU - Yang, A.

AU - Xuan, R. R.

AU - Melbourne, W.

AU - Hashimoto, T.

AU - Uzun, S.

AU - Daneshpazhooh, M.

AU - Yamagami, Jun

AU - Di Zenzo, G.

AU - Mascaro, J. M.

AU - Mahmoudi, H.

AU - Patsatsi, A.

AU - Drenovska, K.

AU - Vassileva, S.

AU - Murrell, D. F.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Background: The BIOCHIP (Dermatology Mosaic 7; EUROIMMUN, Lubeck, Germany) is a novel multiplex indirect immunofluorescence (IIF) technique used in the serological diagnosis of bullous pemphigoid (BP) and pemphigus. Objective: To validate the accuracy and inter-rater reliability (IRR) of the BIOCHIP in the diagnosis of BP, pemphigus foliaceus (PF) and pemphigus vulgaris (PV). Methods: Sera from patients with BP (n = 38), PF (n = 8), PV (n = 23), control patients (n = 64) and healthy control volunteers (n = 39) were tested. Sera were collected and analysed during the course of the disease at 1–5 different time points. The BIOCHIP was performed for all patients, digital images were captured of each incubated field, and the images were shared with 10 dermatologists experienced in reading IF from around the world to report. There were 312 BIOCHIP slides consisting of 1872 photos in total. All patients were de-identified. Fleiss Kappa was used to estimate the IRR. Results: Fleiss Kappa was computed for each category (Oesophagus, Oesophagus immunofluorescence pattern, Salt-Split Skin (SSS), SSS immunofluorescence location, BP180, BP230, Dsg 1 and Ds3). The inter-rater agreement between the 10 raters varied between fair and moderate for all categories. Those that demonstrated fair concordance included monkey oesophagus (k = 0.257, P < 0.0001), oesophagus pattern (k = 0.357, P < 0.0001), Dsg1 (k = 0.390, P < 0.0001) and BP230 (k = 0.281, P < 0.0001). Moderate agreement was demonstrated for SSS (k = 0.416, P < 0.0001), SSS immunofluorescence location (k = 0.505, P < 0.0001), Dsg3 (k = 0.437, P < 0.0001) and BP180 (k = 0.559, P < 0.0001). Conclusion: The BIOCHIP mosaic-based immunofluorescence test is a simple, time and effort saving test that can aid in the diagnosis and screening of BP, PV and PF. However, the level of agreement was relatively low. The authors found the most common causes to be variable levels of training, indicating the presence of a learning curve in the interpretation of the results and ambiguous staining patterns leading to incongruent results.

AB - Background: The BIOCHIP (Dermatology Mosaic 7; EUROIMMUN, Lubeck, Germany) is a novel multiplex indirect immunofluorescence (IIF) technique used in the serological diagnosis of bullous pemphigoid (BP) and pemphigus. Objective: To validate the accuracy and inter-rater reliability (IRR) of the BIOCHIP in the diagnosis of BP, pemphigus foliaceus (PF) and pemphigus vulgaris (PV). Methods: Sera from patients with BP (n = 38), PF (n = 8), PV (n = 23), control patients (n = 64) and healthy control volunteers (n = 39) were tested. Sera were collected and analysed during the course of the disease at 1–5 different time points. The BIOCHIP was performed for all patients, digital images were captured of each incubated field, and the images were shared with 10 dermatologists experienced in reading IF from around the world to report. There were 312 BIOCHIP slides consisting of 1872 photos in total. All patients were de-identified. Fleiss Kappa was used to estimate the IRR. Results: Fleiss Kappa was computed for each category (Oesophagus, Oesophagus immunofluorescence pattern, Salt-Split Skin (SSS), SSS immunofluorescence location, BP180, BP230, Dsg 1 and Ds3). The inter-rater agreement between the 10 raters varied between fair and moderate for all categories. Those that demonstrated fair concordance included monkey oesophagus (k = 0.257, P < 0.0001), oesophagus pattern (k = 0.357, P < 0.0001), Dsg1 (k = 0.390, P < 0.0001) and BP230 (k = 0.281, P < 0.0001). Moderate agreement was demonstrated for SSS (k = 0.416, P < 0.0001), SSS immunofluorescence location (k = 0.505, P < 0.0001), Dsg3 (k = 0.437, P < 0.0001) and BP180 (k = 0.559, P < 0.0001). Conclusion: The BIOCHIP mosaic-based immunofluorescence test is a simple, time and effort saving test that can aid in the diagnosis and screening of BP, PV and PF. However, the level of agreement was relatively low. The authors found the most common causes to be variable levels of training, indicating the presence of a learning curve in the interpretation of the results and ambiguous staining patterns leading to incongruent results.

UR - http://www.scopus.com/inward/record.url?scp=85070721550&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070721550&partnerID=8YFLogxK

U2 - 10.1111/jdv.15817

DO - 10.1111/jdv.15817

M3 - Article

C2 - 31325388

AN - SCOPUS:85070721550

JO - Journal of the European Academy of Dermatology and Venereology

JF - Journal of the European Academy of Dermatology and Venereology

SN - 0926-9959

ER -