使用语义特征和响应指标的个性化医生推荐新方法：模型评估研究。

Novel Approach to Personalized Physician Recommendations Using Semantic Features and Response Metrics: Model Evaluation Study.

机构信息

Biomedical Big Data Center, The First Affiliated Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen City, China.

Meteorological Disaster Prevention Technology Center, Xiamen Meteorological Bureau, Xiamen City, China.

出版信息

JMIR Hum Factors. 2024 Aug 15;11:e57670. doi: 10.2196/57670.

DOI:10.2196/57670

PMID:39146009

原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC11362707/

Abstract

BACKGROUND

The rapid growth of web-based medical services has highlighted the significance of smart triage systems in helping patients find the most appropriate physicians. However, traditional triage methods often rely on department recommendations and are insufficient to accurately match patients' textual questions with physicians' specialties. Therefore, there is an urgent need to develop algorithms for recommending physicians.

OBJECTIVE

This study aims to develop and validate a patient-physician hybrid recommendation (PPHR) model with response metrics for better triage performance.

METHODS

A total of 646,383 web-based medical consultation records from the Internet Hospital of the First Affiliated Hospital of Xiamen University were collected. Semantic features representing patients and physicians were developed to identify the set of most similar questions and semantically expand the pool of recommended physician candidates, respectively. The physicians' response rate feature was designed to improve candidate rankings. These 3 characteristics combine to create the PPHR model. Overall, 5 physicians participated in the evaluation of the efficiency of the PPHR model through multiple metrics and questionnaires as well as the performance of Sentence Bidirectional Encoder Representations from Transformers and Doc2Vec in text embedding.

RESULTS

The PPHR model reaches the best recommendation performance when the number of recommended physicians is 14. At this point, the model has an F-score of 76.25%, a proportion of high-quality services of 41.05%, and a rating of 3.90. After removing physicians' characteristics and response rates from the PPHR model, the F-score decreased by 12.05%, the proportion of high-quality services fell by 10.87%, the average hit ratio dropped by 1.06%, and the rating declined by 11.43%. According to whether those 5 physicians were recommended by the PPHR model, Sentence Bidirectional Encoder Representations from Transformers achieved an average hit ratio of 88.6%, while Doc2Vec achieved an average hit ratio of 53.4%.

CONCLUSIONS

The PPHR model uses semantic features and response metrics to enable patients to accurately find the physician who best suits their needs.

摘要

背景

基于互联网的医疗服务迅速发展，凸显了智能分诊系统在帮助患者找到最合适医生方面的重要性。然而，传统的分诊方法往往依赖科室推荐，不足以准确匹配患者的文本问题与医生的专业特长。因此，迫切需要开发医生推荐算法。

目的

本研究旨在开发和验证一种具有响应指标的医患混合推荐（PPHR）模型，以提高分诊性能。

方法

共收集了来自厦门大学附属第一医院互联网医院的 646383 例在线医疗咨询记录。开发了表示患者和医生的语义特征，分别用于识别最相似问题集并对推荐医生候选人池进行语义扩展。设计了医生的响应率特征，以改进候选者排名。这 3 个特征结合形成了 PPHR 模型。共有 5 位医生通过多项指标和问卷，以及 Sentence Bidirectional Encoder Representations from Transformers 和 Doc2Vec 在文本嵌入中的性能，对 PPHR 模型的效率进行了评估。

结果

当推荐医生数量为 14 时，PPHR 模型达到最佳推荐性能，此时模型的 F1 得分为 76.25%，高质量服务的比例为 41.05%，评分 3.90。从 PPHR 模型中去除医生特征和响应率后，F1 得分下降了 12.05%，高质量服务的比例下降了 10.87%，平均命中率下降了 1.06%，评分下降了 11.43%。根据是否推荐了这 5 位医生，Sentence Bidirectional Encoder Representations from Transformers 的平均命中率为 88.6%，而 Doc2Vec 的平均命中率为 53.4%。