Park Eunsoo H, Watson Hannah I, Mehendale Felicity V, O'Neil Alison Q
Edinburgh Medical School, College of Medicine and Veterinary Medicine, University of Edinburgh, Edinburgh, United Kingdom.
Canon Medical Research Europe, Edinburgh, United Kingdom.
JMIR Med Inform. 2022 Oct 26;10(10):e39616. doi: 10.2196/39616.
Information retrieval (IR) from the free text within electronic health records (EHRs) is time consuming and complex. We hypothesize that natural language processing (NLP)-enhanced search functionality for EHRs can make clinical workflows more efficient and reduce cognitive load for clinicians.
This study aimed to evaluate the efficacy of 3 levels of search functionality (no search, string search, and NLP-enhanced search) in supporting IR for clinical users from the free text of EHR documents in a simulated clinical environment.
A clinical environment was simulated by uploading 3 sets of patient notes into an EHR research software application and presenting these alongside 3 corresponding IR tasks. Tasks contained a mixture of multiple-choice and free-text questions. A prospective crossover study design was used, for which 3 groups of evaluators were recruited, which comprised doctors (n=19) and medical students (n=16). Evaluators performed the 3 tasks using each of the search functionalities in an order in accordance with their randomly assigned group. The speed and accuracy of task completion were measured and analyzed, and user perceptions of NLP-enhanced search were reviewed in a feedback survey.
NLP-enhanced search facilitated more accurate task completion than both string search (5.14%; P=.02) and no search (5.13%; P=.08). NLP-enhanced search and string search facilitated similar task speeds, both showing an increase in speed compared to the no search function, by 11.5% (P=.008) and 16.0% (P=.007) respectively. Overall, 93% of evaluators agreed that NLP-enhanced search would make clinical workflows more efficient than string search, with qualitative feedback reporting that NLP-enhanced search reduced cognitive load.
To the best of our knowledge, this study is the largest evaluation to date of different search functionalities for supporting target clinical users in realistic clinical workflows, with a 3-way prospective crossover study design. NLP-enhanced search improved both accuracy and speed of clinical EHR IR tasks compared to browsing clinical notes without search. NLP-enhanced search improved accuracy and reduced the number of searches required for clinical EHR IR tasks compared to direct search term matching.
从电子健康记录（EHR）中的自由文本进行信息检索既耗时又复杂。我们假设，针对EHR的自然语言处理（NLP）增强搜索功能可提高临床工作流程的效率，并减轻临床医生的认知负担。
本研究旨在评估3种搜索功能级别（无搜索、字符串搜索和NLP增强搜索）在模拟临床环境中支持临床用户从EHR文档自由文本中进行信息检索的效果。
通过将3组患者记录上传到EHR研究软件应用程序中，并将这些记录与3个相应的信息检索任务一起呈现，模拟临床环境。任务包含多项选择题和自由文本问题的混合。采用前瞻性交叉研究设计，招募了3组评估人员，包括医生（n = 19）和医学生（n = 16）。评估人员按照随机分配的组的顺序，使用每种搜索功能执行这3项任务。测量并分析任务完成的速度和准确性，并在反馈调查中评估用户对NLP增强搜索的看法。
与字符串搜索（5.14%；P = 0.02）和无搜索（5.13%；P = 0.08）相比，NLP增强搜索有助于更准确地完成任务。NLP增强搜索和字符串搜索的任务速度相似，与无搜索功能相比，两者的速度均有所提高，分别提高了11.5%（P = 0.008）和16.0%（P = 0.007）。总体而言，93%的评估人员同意，NLP增强搜索将比字符串搜索使临床工作流程更高效，定性反馈表明NLP增强搜索减轻了认知负担。
据我们所知，本研究是迄今为止在现实临床工作流程中支持目标临床用户的不同搜索功能的最大规模评估，采用了三向前瞻性交叉研究设计。与无搜索浏览临床记录相比，NLP增强搜索提高了临床EHR信息检索任务的准确性和速度。与直接搜索词匹配相比，NLP增强搜索提高了准确性，并减少了临床EHR信息检索任务所需的搜索次数。
