Skip to main content

Table 2 Discrimination and reclassification ability of the 6-metabolite model based on 4 machine learning algorithms and logistic regression

From: A novel 6-metabolite signature for prediction of clinical outcomes in type 2 diabetic patients undergoing percutaneous coronary intervention

Model Discrimination statistics Seven-fold CV Reclassification index
AUC P valuea Sensitivity Specificity PPV NPV AUC NRI P value
Discovery set
 Logistic regression 0.89 (0.81–0.94) Ref. 80 (67–89) 91 (80–97) 61 (40–79) 96 (94–98) 0.89 Ref. Ref.
 Random forest 0.99 (0.95–1.00) < 0.001 98 (90–99) 96 (87–99) 83 (55–95) 99 (98–100) 0.99 1.93 (1.83–2.03) < 0.001
 XGBoost 0.99 (0.96–1.00) < 0.001 98 (90–99) 96 (87–99) 83 (55–95) 99 (98–100) 0.99 1.89 (1.77–2.01) < 0.001
SVM 0.93 (0.86–0.97) 0.005 85 (73–93) 93 (82–98) 68 (45–84) 97 (95–99) 0.91 1.00 (0.68–1.32) < 0.001
 DNN 0.91 (0.85–0.96) 0.092 96 (87–99) 87 (75–95) 58 (40–73) 99 (97–100) 0.92 0.96 (0.66–1.26) < 0.001
Internal validation
 Logistic regression 0.85 (0.76–0.91) Ref. 72 (58–84) 94 (85–99) 70 (44–88) 95 (92–97) 0.85 Ref. Ref.
 Random forest 0.93 (0.87–0.97) 0.003 83 (71–92) 94 (85–99) 73 (48–89) 97 (95–98) 0.91 1.33 (1.05–1.61) < 0.001
 XGBoost 0.91 (0.84–0.95) 0.023 85 (73–93) 94 (85–99) 73 (48–89) 97 (95–99) 0.91 1.33 (1.05–1.61) < 0.001
 SVM 0.89 (0.82–0.94) 0.119 85 (73–93) 83 (71–92) 48 (34–63) 97 (94–98) 0.88 0.22 (− 0.15 to 0.59) 0.240
 DNN 0.82 (0.74–0.89) 0.329 69 (54–81) 94 (85–99) 69 (42–87) 94 (92–96) 0.84 0.22 (− 0.14 to 0.59) 0.233
  1. AUC area under the curve; PPV positive predictive value; NPV negative predictive value; CV cross validation; NRI net reclassification improvement; XGBoost extreme gradient boosting tree; SVM Support Vector Machines; DNN deep neural network
  2. aPaired comparisons of AUCs between logistic regression and each machine learning model are conducted using a DeLong test