An interpretable machine learning model for predicting early liver metastasis after pancreatic cancer surgery

Abstract

Background: Liver metastasis is the most frequent site of distant metastasis in pancreatic ductal adenocarcinoma (PDAC), significantly contributing to poor prognosis. This study aims to develop and validate a machine learning (ML) model for predicting early liver metastasis (ELM) following pancreatic cancer surgery.

Method: This retrospective study included 407 pancreatic cancer patients who underwent surgery at the First Affiliated Hospital of Soochow University between January 2015 and December 2023, aiming to develop and validate a predictive model. Seven ML algorithms were employed to predict the risk of liver metastasis within one year after surgery. The training cohort (n = 284) was used for model development and hyperparameter tuning, while the internal validation cohort (n = 123) was employed to assess predictive performance. Shapley additive explanations (SHAP) were applied to elucidate the decision-making process of the best-performing model. To assess the generalizability of the model, 131 PDAC patients from the Affiliated Hospital of Nantong University were included as an external validation cohort.

Result: A total of 194 patients (36.1%) were diagnosed with ELM during the 1-year postoperative follow-up across the two centers. Out of 22 disease characteristics, nine key features were selected for the development of the model. XGBoost exhibited the highest performance, achieving an AUC of 0.901, accuracy of 0.846, sensitivity of 0.756, specificity of 0.897, and an F1 score of 0.782. The Brier score of 0.12 indicated excellent calibration. Furthermore, both the internal and external validation datasets demonstrated consistent and robust performance, as evidenced by ROC curves, calibration plots, decision curves, and clinical impact curves, thereby supporting its clinical utility.

Conclusion: An XGBoost model was developed to predict the likelihood of ELM after PDAC surgery with high accuracy. Additionally, the model was implemented as an application, providing clinicians with an accessible visual tool to support personalized clinical strategies and ultimately enhance patient outcomes.

Keywords: Early liver metastasis; Machine learning; PDAC; Pancreatic cancer; Prediction model.