A novel 6-metabolite signature for prediction of clinical outcomes in type 2 diabetic patients undergoing percutaneous coronary intervention

Table 2 Discrimination and reclassification ability of the 6-metabolite model based on 4 machine learning algorithms and logistic regression

Model	Discrimination statistics						Seven-fold CV	Reclassification index
Model	AUC	P value^a	Sensitivity	Specificity	PPV	NPV	AUC	NRI	P value
Discovery set
Logistic regression	0.89 (0.81–0.94)	Ref.	80 (67–89)	91 (80–97)	61 (40–79)	96 (94–98)	0.89	Ref.	Ref.
Random forest	0.99 (0.95–1.00)	< 0.001	98 (90–99)	96 (87–99)	83 (55–95)	99 (98–100)	0.99	1.93 (1.83–2.03)	< 0.001
XGBoost	0.99 (0.96–1.00)	< 0.001	98 (90–99)	96 (87–99)	83 (55–95)	99 (98–100)	0.99	1.89 (1.77–2.01)	< 0.001
SVM	0.93 (0.86–0.97)	0.005	85 (73–93)	93 (82–98)	68 (45–84)	97 (95–99)	0.91	1.00 (0.68–1.32)	< 0.001
DNN	0.91 (0.85–0.96)	0.092	96 (87–99)	87 (75–95)	58 (40–73)	99 (97–100)	0.92	0.96 (0.66–1.26)	< 0.001
Internal validation
Logistic regression	0.85 (0.76–0.91)	Ref.	72 (58–84)	94 (85–99)	70 (44–88)	95 (92–97)	0.85	Ref.	Ref.
Random forest	0.93 (0.87–0.97)	0.003	83 (71–92)	94 (85–99)	73 (48–89)	97 (95–98)	0.91	1.33 (1.05–1.61)	< 0.001
XGBoost	0.91 (0.84–0.95)	0.023	85 (73–93)	94 (85–99)	73 (48–89)	97 (95–99)	0.91	1.33 (1.05–1.61)	< 0.001
SVM	0.89 (0.82–0.94)	0.119	85 (73–93)	83 (71–92)	48 (34–63)	97 (94–98)	0.88	0.22 (− 0.15 to 0.59)	0.240
DNN	0.82 (0.74–0.89)	0.329	69 (54–81)	94 (85–99)	69 (42–87)	94 (92–96)	0.84	0.22 (− 0.14 to 0.59)	0.233

AUC area under the curve; PPV positive predictive value; NPV negative predictive value; CV cross validation; NRI net reclassification improvement; XGBoost extreme gradient boosting tree; SVM Support Vector Machines; DNN deep neural network
^aPaired comparisons of AUCs between logistic regression and each machine learning model are conducted using a DeLong test

ISSN: 1475-2840