Analysis of ILD-RADS reproducibility and validity: Can different degrees of experience affect the interpretation?

Abstract

Purpose: To evaluate the reliability, validity and applicability of ILD-RADS among readers with different levels of experience.

Methods: This prospective tri-center study included 159 patients with clinically diagnosed ILD who underwent high-resolution CT (HRCT). Two experienced thoracic radiologists, two general radiologists, and one pulmonologist independently evaluated the HRCT images blinded to the patient's clinical data and assigned ILD-RADS category for each patient. The Fleiss kappa test was employed to estimate the inter-reader agreement among all readers. Cohen's kappa test was applied to measure the pairwise inter-reader agreement. The multi-disciplinary team discussion (MDD) was used as a reference test to estimate the validity of ILD-RADS for diagnosing idiopathic pulmonary fibrosis (IPF). A 5-point Likert short survey was accomplished by the pulmonologists about the applicability of ILD-RADS in clinical practice.

Results: The current study included 124 non-IPF and 35 IPF cases. Based on the radiologists, the ILD-RADS showed moderate inter-reader agreement (K = 0.515, P < 0.001) while being fair after the inclusion of the pulmonologist's input (K = 0.333, P < 0.001). The agreement was substantial among thoracic radiologists (K = 0.716, p < 0.001) and moderate among general radiologists (K = 0.461, p < 0.001). ILD-RADS ≤ 2 was the optimal cut-off for predicting IPF, with an accuracy ranging from 62.84 % to 80.54 %. Seventy-five percent of pulmonologists rated ILD-RADS as highly applicable in practice.

Conclusions: ILD-RADS is reliable and valid among radiologists but requires further refinement to enhance consistency and applicability in diverse clinical settings. Moreover, pulmonologists support its use in clinical practice.

Keywords: Agreement; HRCT; ILDs; IPF; Standardized.