Hybrid Machine Learning and Deep Learning Framework for Predicting Lymph Node Metastasis in Papillary Thyroid Carcinoma with Hashimoto's Thyroiditis: A Retrospective Cohort Study

Abstract

Background:  Papillary thyroid carcinoma (PTC) is commonly present in the co-occurrence with Hashimoto thyroiditis (HT); the inflammatory microenvironment, however, presents obstacles to preoperative predictability of lymph node metastasis (LNM). Accurate risk stratification also clinically cannot be done without to inform surgical decision-making and reduce unnecessary central neck dissections. In line with this, this study assesses the viability of machine-learning (ML), deep-learning (DL), and the hybrid modelling approaches to forecast LNM in HT-related PTC by using real-world clinical data.

Methods: A retrospective cohort study involving 197 patients who had been diagnosed with HT-PTC was examined. Following data cleaning and feature engineering, 68 patients with complete LNM labels were used in supervised learning. Models considered included logistic regression, random forest, gradient boosting, XGBoost, a fully connected artificial neural network (ANN) and an XGBoost-ANN hybrid architecture. The model performance was measured based on accuracy, precision, recall, F1-score and area under receiver operating characteristic curve (AUC). The explainability of the model was estimated using SHAP values.

Results: Random Forest produced the best area under the receiver operating characteristic curve (AUC = 0.617), traditional and hybrid models showed rather modest discriminative outcomes (AUC range 0.36-0.53). The artificial neural network provided showed significant overfitting, which can probably be explained by the small sample size. The SHAP (SHapley Additive exPlanations) analysis showed that biochemical markers added only weak but discernible information, and no salient predictors could be found in the provided set of features.

Conclusion: ML/DL models trained on demographic and biochemical features alone show limited utility in predicting LNM in HT-PTC. Integration of structural ultrasound and pathological features, along with larger multicentre datasets, is necessary to achieve clinically deployable predictive performance.