Department of Biomedical Engineering, Virginia Tech, Blacksburg, Virginia.
Traffic Inj Prev. 2021;22(sup1):S169-S172. doi: 10.1080/15389588.2021.1982616. Epub 2021 Dec 7.
The objective of this study was to develop a system which used the BERT natural language understanding model to identify pedal misapplication (PM) crashes from their crash narratives and validate the accuracy of the system.
The training dataset used for this study was 11 cases from the NMVCCS study and 952 cases from the North Carolina state crash database. Cases for this study were selected from their respective full datasets using a keyword search algorithm containing terms indicative of a pedal-related mistake. A BERT language model was used to classify each case narrative as either no pedal misapplication, PM by vehicle 1, PM by vehicle 2, or PM by vehicle 3. After training, the language model was used to determine the incidence of pedal misapplication in a test dataset of 8,668 North Carolina and NMVCCS cases and these results were compared to a manual review of the dataset. After manual review, 2,969 cases were pedal misapplications.
The model's AUC ROC performance at detecting PM was quantified on the entire testing dataset to evaluate the power of the system to generalize to case narratives unseen at training time. The AUC ROC value was 0.9835, indicating strong generalization to all crash narratives. By choosing the optimal threshold using the ROC curve, the system correctly identified PM in 95.7% of crash narratives. When pedal misapplication was correctly identified, the correct vehicle was identified in 95.9% of cases. A total of 3,062 pedal misapplications were identified. The model labeled cases 353 times faster than a researcher.
The strong performance of the model suggests that the automated interpretation of case narratives can be used for future research studies without any manual review. This would save time and enable the use of datasets where manual review would be infeasible. The automated extraction of information from crash narratives using deep learning natural language models has not been demonstrated previously in the literature, to the best of the authors' knowledge. This technique can be applied to large, infrequently used datasets of crash narratives and extended to extract useful vehicle, occupant, or environment information to make these datasets amenable to traditional statistical analyses.
本研究旨在开发一种系统，该系统使用 BERT 自然语言理解模型从事故叙述中识别出踏板误用（PM）事故，并验证系统的准确性。
本研究的训练数据集由 NMVCCS 研究中的 11 个案例和北卡罗来纳州的 952 个案例数据库组成。本研究的案例是从各自的完整数据集中使用包含与踏板相关错误指示词的关键字搜索算法选择的。使用 BERT 语言模型对每个案例叙述进行分类，分为无踏板误用、车辆 1 的 PM、车辆 2 的 PM 或车辆 3 的 PM。经过训练后，语言模型用于确定在北卡罗来纳州和 NMVCCS 的 8668 个测试案例数据集中的踏板误用发生率，并将这些结果与对数据集的手动审查进行比较。经过手动审查，2969 个案例为踏板误用。
该模型在整个测试数据集上检测 PM 的 AUC ROC 性能，以评估系统对训练时未见过的事故叙述进行泛化的能力。AUC ROC 值为 0.9835，表明对所有事故叙述具有很强的泛化能力。通过使用 ROC 曲线选择最佳阈值，系统正确识别出 95.7%的 PM 事故叙述。当正确识别出踏板误用时，正确识别的车辆占 95.9%的案例。总共识别出 3062 个踏板误用。该模型的标记速度比研究人员快 353 倍。
模型的优异性能表明，事故叙述的自动解释可用于未来的研究，而无需进行任何手动审查。这将节省时间，并使数据集能够进行传统的统计分析，而无需手动审查。在文献中，到目前为止，还没有在深度学习自然语言模型中展示过从事故叙述中自动提取信息的技术。该技术可以应用于大型、不常用的事故叙述数据集，并扩展到提取有用的车辆、乘员或环境信息，以使这些数据集适合于传统的统计分析。
