Chillakuru Yeshwant Reddy, Munjal Shourya, Laguna Benjamin, Chen Timothy L, Chaudhari Gunvant R, Vu Thienkhai, Seo Youngho, Narvid Jared, Sohn Jae Ho
Radiology & Biomedical Imaging, University of California San Francisco (UCSF), 505 Parnassus Ave, San Francisco, CA, 94158, USA.
The George Washington University School of Medicine and Health Sciences, 2300 I St NW, Washington, DC, 20052, USA.
BMC Med Inform Decis Mak. 2021 Jul 12;21(1):213. doi: 10.1186/s12911-021-01574-y.
A systematic approach to MRI protocol assignment is essential for the efficient delivery of safe patient care. Advances in natural language processing (NLP) allow for the development of accurate automated protocol assignment. We aim to develop, evaluate, and deploy an NLP model that automates protocol assignment, given the clinician indication text.
We collected 7139 spine MRI protocols (routine or contrast) and 990 head MRI protocols (routine brain, contrast brain, or other) from a single institution. Protocols were split into training (n = 4997 for spine MRI; n = 839 for head MRI), validation (n = 1071 for spine MRI, fivefold cross-validation used for head MRI), and test (n = 1071 for spine MRI; n = 151 for head MRI) sets. fastText and XGBoost were used to develop 2 NLP models to classify spine and head MRI protocols, respectively. A Flask-based web app was developed to be deployed via Heroku.
The spine MRI model had an accuracy of 83.38% and a receiver operator characteristic area under the curve (ROC-AUC) of 0.8873. The head MRI model had an accuracy of 85.43% with a routine brain protocol ROC-AUC of 0.9463 and contrast brain protocol ROC-AUC of 0.9284. Cancer, infectious, and inflammatory related keywords were associated with contrast administration. Structural anatomic abnormalities and stroke/altered mental status were indicative of routine spine and brain MRI, respectively. Error analysis revealed increasing the sample size may improve performance for head MRI protocols. A web version of the model is provided for demonstration and deployment.
We developed and web-deployed two NLP models that accurately predict spine and head MRI protocol assignment, which could improve radiology workflow efficiency.
采用系统方法进行MRI协议分配对于安全有效地提供患者护理至关重要。自然语言处理（NLP）的进展使得开发准确的自动协议分配成为可能。我们旨在开发、评估和部署一种NLP模型，该模型可根据临床医生的指示文本自动进行协议分配。
我们从一家机构收集了7139份脊柱MRI协议（常规或增强）和990份头部MRI协议（常规脑、增强脑或其他）。协议被分为训练集（脊柱MRI为n = 4997；头部MRI为n = 839）、验证集（脊柱MRI为n = 1071，头部MRI采用五折交叉验证）和测试集（脊柱MRI为n = 1071；头部MRI为n = 151）。使用fastText和XGBoost分别开发了2个NLP模型来对脊柱和头部MRI协议进行分类。开发了一个基于Flask的网络应用程序，以便通过Heroku进行部署。
脊柱MRI模型的准确率为83.38%，曲线下面积（ROC-AUC）为0.8873。头部MRI模型的准确率为85.43%，常规脑协议的ROC-AUC为0.9463，增强脑协议的ROC-AUC为0.9284。与癌症、感染和炎症相关的关键词与增强给药有关。结构解剖异常和中风/精神状态改变分别指示常规脊柱和脑部MRI。误差分析表明，增加样本量可能会提高头部MRI协议的性能。提供了该模型的网络版本用于演示和部署。
我们开发并通过网络部署了两个NLP模型，它们能够准确预测脊柱和头部MRI协议分配，这可以提高放射学工作流程效率。
