Validation of an artificial intelligence-based algorithm for predictive performance and risk stratification of sepsis using real-world data from hospitalised patients: a prospective observational study

Abstract

Objective: The heterogeneous nature of sepsis renders determining its underlying causes difficult, which may delay diagnosis and intervention. VitalCare-SEPsis Score (VC-SEPS) is a deep learning-based algorithm that predicts sepsis and monitors patient conditions based on electronic medical record data. However, few studies have prospectively compared medical artificial intelligence software algorithms and traditional scoring systems to predict sepsis. This prospective observational study attempted to validate the predictive performance and risk stratification of VC-SEPS for early prediction of sepsis.

Methods: In this prospective observational study, we collected electronic medical record data from 6,797 patients hospitalised at Keimyung University Dongsan Hospital, Daegu, South Korea. The final version of the analysed set included 6,455 patients, 325 of whom were diagnosed with sepsis.

Results: The area under the receiver operating characteristic curve of VC-SEPS was 0.880, indicating its superiority over traditional scoring systems. The algorithm performance showed a consistent trend within 24 hours. On patients' initial admission, the VC-SEPS was associated with the risk of developing sepsis, and the score accurately predicted sepsis by an average of 68.05 min compared with diagnosis time by an operational definition of sepsis.

Discussion: VC-SEPS could assist medical staff with early diagnosis and intervention in clinical practice by providing a sepsis risk score. Prompt recognition assisting recognition can significantly help shorten the time between recognition and intervention in clinical decision-making processes.

Conclusion: This study suggests that using a clinical decision support system can help improve hospital workflows as well as the quality of medical care.

Keywords: Artificial intelligence; Decision Support Systems, Clinical; Deep Learning; Information Science; Medical Records.