1005: Increasing the Proficiency for Scoring Sacroiliac Joint Radiographs According to the Modified New York Criteria for Ankylosing Spondylitis by Using an Online Real-time Iterative Calibration (RETIC) Module

Anna Hadsbjerg¹, Mikkel Østergaard², Joel Paschke³, Adrian Ciurea⁴, Michael Nissen⁵, Stephanie Wichuk⁶, Ashish J Mathew⁷, Marie Wetterslev⁸, Raphael Micheroli⁹, Susanne Pedersen¹⁰, Simon Krabbe¹⁰, Kristyna Bubova¹¹, Monika gregova¹², Alexander Bernatschek⁴, Maurice Donzallaz⁴, Burkhard Moeller¹³, Ziga Snoj¹⁴, Karlo Pintaric¹⁴, Manouk de Hooge¹⁵, Karel Gorican¹⁶, Robert G Lambert⁶ and Walter P Maksymowych¹⁷, ¹Copenhagen Center for Arthritis Research, COPECARE, Rigshospitalet, Glostrup, Denmark, ²Rigshospitalet, University of Copenhagen, Glostrup, Denmark, ³CARE Arthritis LTD, Edmonton, AB, Canada, ⁴University Hospital Zurich, Zürich, Switzerland, ⁵Hopitaux Universitaires de Genève, Geneva, Switzerland, ⁶University of Alberta, Edmonton, AB, Canada, ⁷Rigshospitalet, Copenhagen, Denmark, ⁸Rigshospitalet, Copenhagen, Glostrup, Denmark, ⁹University Hospital Zurich, Department of Rheumatology, Zürich, Switzerland, ¹⁰Rigshospitalet, København, Denmark, ¹¹First Faculty of Medicine, Charles University,, Prague, Czech Republic, ¹²Institue of Rheumatology, Prague, ¹³Inselspital - University Hospital Bern, Bern, Switzerland, ¹⁴Radiology, University Medical Centre Ljubljana, Ljubljana, Slovenia, ¹⁵Ghent University Hospital, Luxembourg, Luxembourg, ¹⁶MSK radiology unit, Radiology section – Diagnostic department, Geneva University Hospitals, Geneva, Switzerland, ¹⁷Department of Medicine, University of Alberta, Edmonton, AB, Canada

Background/Purpose: Radiographic assessment of the sacroiliac joints (SIJ) according to the modified New York (mNY) criteria is essential in the classification of ankylosing spondylitis, but low to moderate inter-reader agreement and limited improvement by training have been reported. A web-based real-time iterative calibration (RETIC) module for scoring SIJ radiographs according to the mNY criteria has been developed by experienced readers to allow remote standardized training. In this study, we aimed to test the performance of this online RETIC training module in enhancing scoring proficiency with an expert reader as gold standard reference.

Methods: The RETIC module for scoring according to the mNY criteria consisted of 50 cases with radiographs of the SIJ at one timepoint with an integrated scoring interface with real time feedback to the reader's grading compared to the expert readers' grading. The mNY grade (0-4) is assessed for each joint. Reliability for fulfilling (mNY+) versus not fulfilling (mNY-) mNY criteria was assessed in real-time by kappa data being provided every 10 cases, until proficiency targets (kappa > 0.5) were achieved. In this study, participants (n=17) from the EuroSpA Imaging project were randomized to one of two reader training strategies (groups A and B) that comprised of 3 training steps, each followed by a reading exercise (Exercises 1-3; 25 patients per exercise). Training steps were as follows: Group A. 1. Review of original manuscripts describing the mNY method. 2. Review of PowerPoint summary of mNY method and video with grading examples PLUS completion of mNY_RETIC module. 3. Re-review of PowerPoint summary and video. Group B. Same 3-step strategy as group A except that the mNY_RETIC module was completed at step 3, i.e. immediately before Exercise 3. The reliability of scoring was compared to an expert radiologist (RL), using kappa statistics for positivity of mNY criteria and intraclass correlation coefficients (ICCs) for the grades of the mNY criteria.

Results: The mean agreement for all readers on fulfillment of the mNY criteria (mNY+ vs mNY-) with kappa was 0.69 in exercise 1, 0.72 in exercise 2 and 0.83 in exercise 3. Corresponding ICCs for mNY scoring 0-4 were 0.77, 0.77 and 0.82 in exercises 1, 2 and 3, respectively, see Table. For Reader Group A (performed RETIC module before exercise 2), kappas for exercises 1-3 were 0.65, 0.76 and 0.83, respectively and ICCs 0.71, 0.82 and 0.81 (exercise 3). Interestingly, the main increase in agreement occurred between Exercises 1 and 2, i.e. after the RETIC module was performed. For Reader group B (performed RETIC module before exercise 3), corresponding kappa's were 0.71, 0.77 and 0.83 and ICCs 0.80, 0.72 and 0.80, for exercises 1-3, respectively. Similarly, agreements predominantly improved directly after the RETIC module was performed.

Conclusion: Agreement on scoring radiographs of the SIJ according to the mNY criteria was noticeably improved when using a systematic online training module and in general readers achieved very good agreement with experts implying that systematic training of readers before scoring of images can enhance scoring proficiency.

Disclosures: A. Hadsbjerg, Novartis; M. Østergaard, AbbVie/Abbott, Amgen, Bristol-Myers Squibb(BMS), Celgene, Eli Lilly, Janssen, Gilead, Novartis, Pfizer, UCB; J. Paschke, None; A. Ciurea, AbbVie, Novartis, Merck/MSD; M. Nissen, AbbVie/Abbott, Pfizer, Amgen, Novartis, Janssen; S. Wichuk, None; A. Mathew, None; M. Wetterslev, None; R. Micheroli, None; S. Pedersen, Novartis, AbbVie/Abbott, UCB, Pfizer, Merck/MSD; S. Krabbe, Novartis; K. Bubova, None; M. gregova, None; A. Bernatschek, None; M. Donzallaz, None; B. Moeller, None; Z. Snoj, None; K. Pintaric, None; M. de Hooge, None; K. Gorican, None; R. Lambert, Calyx, CARE Arthritis, Image Analysis Group; W. Maksymowych, AbbVie, Boehringer-Ingelheim, Celgene, Eli Lilly, Galapagos, Janssen, Novartis, Pfizer, UCB, CARE Arthritis Limited.