Adil Syed M, Seas Andreas, Sexton Daniel P, Warman Pranav I, Wissel Benjamin D, Carpenter Kennedy L, Carter Lacey, Kolls Brad J, Fuller Anthony T, Lad Shivanand P, Dunn Timothy W, Fuchs Herbert, Vestal Matthew, Grant Gerald A
Departments of1Neurosurgery and.
2Department of Biomedical Engineering, Duke University, Durham, North Carolina; and.
J Neurosurg Pediatr. 2024 Dec 6;35(3):246-254. doi: 10.3171/2024.9.PEDS24146. Print 2025 Mar 1.
The Endoscopic Third Ventriculostomy Success Score (ETVSS) is a useful decision-making heuristic when considering the probability of surgical success, defined traditionally as no repeat cerebrospinal fluid diversion surgery needed within 6 months. Nonetheless, the performance of the logistic regression (LR) model in the original 2009 study was modest, with an area under the receiver operating characteristic curve (AUROC) of 0.68. The authors sought to use a larger dataset to develop more accurate machine learning (ML) models to predict endoscopic third ventriculostomy (ETV) success and also to perform the largest validation of the ETVSS to date.
The authors queried the MarketScan national database for the years 2005-2022 to identify patients < 18 years of age who underwent first-time ETV and subsequently had at least 6 months of continuous enrollment in the database. The authors collected data on predictors matching the original ETVSS: age, etiology of hydrocephalus, and history of any previous shunt placement. Next, they used 6 ML algorithms-LR, support vector classifier, random forest, k-nearest neighbors, Extreme Gradient Boosted Regression (XGBoost), and naive Bayes-to develop predictive models. Finally, the authors used nested cross-validation to assess the models' comparative performances on unseen data.
The authors identified 2047 patients who met inclusion criteria, and 1261 (61.6%) underwent successful ETV. The performances of most ML models were similar to that of the original ETVSS, which had an AUROC of 0.693 on the validation set and 0.661 (95% CI 0.600-0.722) on the test set. The authors' new LR model performed comparably with AUROCs of 0.693 on both the validation and test sets, with 95% CI 0.633-0.754 on the test set. Among the more complex ML algorithms, XGBoost performed best, with AUROCs of 0.683 and 0.672 (95% CI 0.609-0.734) on the validation and test sets, respectively.
This is the largest external validation of the ETVSS, and it confirms modest performance. More sophisticated ML algorithms do not meaningfully improve predictive performance compared to ETVSS; this underscores the need for higher utility, novelty, and dimensionality of input data rather than changes in modeling strategies.
内镜下第三脑室造瘘术成功评分（ETVSS）在考虑手术成功概率时是一种有用的决策启发式方法，传统上定义为6个月内无需再次进行脑脊液分流手术。尽管如此，2009年原始研究中的逻辑回归（LR）模型表现一般，受试者操作特征曲线（AUROC）下面积为0.68。作者试图使用更大的数据集来开发更准确的机器学习（ML）模型，以预测内镜下第三脑室造瘘术（ETV）的成功率，并对ETVSS进行迄今为止最大规模的验证。
作者查询了2005 - 2022年的MarketScan国家数据库，以识别年龄小于18岁且首次接受ETV并随后在数据库中连续登记至少6个月的患者。作者收集了与原始ETVSS匹配的预测因素数据：年龄、脑积水病因以及既往任何分流置管史。接下来，他们使用6种ML算法——LR、支持向量分类器、随机森林、k近邻、极端梯度提升回归（XGBoost）和朴素贝叶斯——来开发预测模型。最后，作者使用嵌套交叉验证来评估模型在未见数据上的比较性能。
作者确定了2047例符合纳入标准的患者，其中1261例（61.6%）ETV手术成功。大多数ML模型的表现与原始ETVSS相似，原始ETVSS在验证集上的AUROC为0.693，在测试集上为0.661（95%CI 0.600 - 0.722）。作者的新LR模型在验证集和测试集上的AUROC均为0.693，测试集上的95%CI为0.633 - 0.754。在更复杂的ML算法中，XGBoost表现最佳，在验证集和测试集上的AUROC分别为0.683和0.672（95%CI 0.609 - 0.734）。
这是对ETVSS最大规模的外部验证，证实了其表现一般。与ETVSS相比，更复杂的ML算法并不能显著提高预测性能；这凸显了对输入数据更高的实用性、新颖性和维度的需求，而非建模策略的改变。
