Citation

van der Ploeg T, Nieboer D, Steyerberg EW. J. Clin. Epidemiol. 2016; 78: 83-89.

Affiliation

Department of Public Health, Erasmus MC - University Medical Center Rotterdam, P.O. Box 2040, 3000 CA Rotterdam, The Netherlands.

Copyright

(Copyright © 2016, Elsevier Publishing)

DOI

10.1016/j.jclinepi.2016.03.002

PMID

26987507

Abstract

BACKGROUND: Prediction of medical outcomes may potentially benefit from using modern statistical modeling techniques. We aimed to externally validate modeling strategies for prediction of 6-month mortality of patients suffering from traumatic brain injury (TBI) with predictor sets of increasing complexity.

METHODS: We analyzed individual patient data from fifteen different studies including 11,026 TBI patients. We consecutively considered a core set of predictors (age, motor score and pupillary reactivity), an extended set with CT scan characteristics, and a further extension with 2 laboratory measurements (glucose and hemoglobin). With each of these sets, we predicted 6-month mortality using default settings with five statistical modelling techniques: logistic regression (LR), classification and regression trees (CART), random forests (RF), support vector machines (SVM) and neural nets (NN). For external validation, a model developed on one of the fifteen data sets was applied to each of the fourteen remaining sets. This process was repeated fifteen times for a total of 630 validations. The area under the ROC-curve (AUC) was used to assess the discriminative ability of the models.

RESULTS: For the most complex predictor set, the LR models performed best (median validated AUC value 0.757), followed by RF and SVM models (median validated AUC value 0.735 and 0.732 respectively). With each predictor set, the CART models showed poor performance (median validated AUC value <0.7). The variability in performance across the studies was smallest for the RF and LR based models (IQR for validated AUC values from 0.07 to 0.10).

CONCLUSIONS: In the area of predicting mortality from traumatic brain injury, non-linear and non-additive effects are not pronounced enough to make modern prediction methods beneficial.

Copyright © 2016 Elsevier Inc. All rights reserved.

Language: en