Inter-rater reliability of the BIOCHIP indirect immunofluorescence dermatology mosaic in bullous pemphigoid and pemphigus patients

Background: The BIOCHIP (Dermatology Mosaic 7; EUROIMMUN, Lubeck, Germany) is a novel multiplex indirect immunofluorescence (IIF) technique used in the serological diagnosis of bullous pemphigoid (BP) and pemphigus. Objective: To validate the accuracy and inter-rater reliability (IRR) of the BIOCHIP in the diagnosis of BP, pemphigus foliaceus (PF) and pemphigus vulgaris (PV). Methods: Sera from patients with BP (n = 38), PF (n = 8), PV (n = 23), control patients (n = 64) and healthy control volunteers (n = 39) were tested. Sera were collected and analysed during the course of the disease at 1–5 different time points. The BIOCHIP was performed for all patients, digital images were captured of each incubated field, and the images were shared with 10 dermatologists experienced in reading IF from around the world to report. There were 312 BIOCHIP slides consisting of 1872 photos in total. All patients were de-identified. Fleiss Kappa was used to estimate the IRR. Results: Fleiss Kappa was computed for each category (Oesophagus, Oesophagus immunofluorescence pattern, Salt-Split Skin (SSS), SSS immunofluorescence location, BP180, BP230, Dsg 1 and Ds3). The inter-rater agreement between the 10 raters varied between fair and moderate for all categories. Those that demonstrated fair concordance included monkey oesophagus (k = 0.257, P < 0.0001), oesophagus pattern (k = 0.357, P < 0.0001), Dsg1 (k = 0.390, P < 0.0001) and BP230 (k = 0.281, P < 0.0001). Moderate agreement was demonstrated for SSS (k = 0.416, P < 0.0001), SSS immunofluorescence location (k = 0.505, P < 0.0001), Dsg3 (k = 0.437, P < 0.0001) and BP180 (k = 0.559, P < 0.0001). Conclusion: The BIOCHIP mosaic-based immunofluorescence test is a simple, time and effort saving test that can aid in the diagnosis and screening of BP, PV and PF. However, the level of agreement was relatively low. The authors found the most common causes to be variable levels of training, indicating the presence of a learning curve in the interpretation of the results and ambiguous staining patterns leading to incongruent results.