School of Public Health, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
Ren Ji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China.
J Med Internet Res. 2021 Jun 2;23(6):e25929. doi: 10.2196/25929.
Clinical decision support systems are designed to utilize medical data, knowledge, and analysis engines and to generate patient-specific assessments or recommendations to health professionals in order to assist decision making. Artificial intelligence-enabled clinical decision support systems aid the decision-making process through an intelligent component. Well-defined evaluation methods are essential to ensure the seamless integration and contribution of these systems to clinical practice.
The purpose of this study was to develop and validate a measurement instrument and test the interrelationships of evaluation variables for an artificial intelligence-enabled clinical decision support system evaluation framework.
An artificial intelligence-enabled clinical decision support system evaluation framework consisting of 6 variables was developed. A Delphi process was conducted to develop the measurement instrument items. Cognitive interviews and pretesting were performed to refine the questions. Web-based survey response data were analyzed to remove irrelevant questions from the measurement instrument, to test dimensional structure, and to assess reliability and validity. The interrelationships of relevant variables were tested and verified using path analysis, and a 28-item measurement instrument was developed. Measurement instrument survey responses were collected from 156 respondents.
The Cronbach α of the measurement instrument was 0.963, and its content validity was 0.943. Values of average variance extracted ranged from 0.582 to 0.756, and values of the heterotrait-monotrait ratio ranged from 0.376 to 0.896. The final model had a good fit (χ=36.984; P=.08; comparative fit index 0.991; goodness-of-fit index 0.957; root mean square error of approximation 0.052; standardized root mean square residual 0.028). Variables in the final model accounted for 89% of the variance in the user acceptance dimension.
User acceptance is the central dimension of artificial intelligence-enabled clinical decision support system success. Acceptance was directly influenced by perceived ease of use, information quality, service quality, and perceived benefit. Acceptance was also indirectly influenced by system quality and information quality through perceived ease of use. User acceptance and perceived benefit were interrelated.
临床决策支持系统旨在利用医疗数据、知识和分析引擎，并为卫生专业人员生成针对患者的评估或建议，以协助决策制定。人工智能支持的临床决策支持系统通过智能组件辅助决策过程。定义明确的评估方法对于确保这些系统无缝融入和有益于临床实践至关重要。
本研究旨在开发和验证一种测量工具，并测试人工智能支持的临床决策支持系统评估框架的评估变量之间的相互关系。
开发了一个由 6 个变量组成的人工智能支持的临床决策支持系统评估框架。采用德尔菲法制定测量工具项目。进行认知访谈和预测试以完善问题。对基于网络的调查响应数据进行分析，以从测量工具中删除不相关的问题，测试维度结构，并评估可靠性和有效性。使用路径分析测试和验证相关变量之间的相互关系，并开发了一个 28 项的测量工具。从 156 名受访者中收集了测量工具调查响应。
测量工具的 Cronbach α 为 0.963，内容效度为 0.943。平均方差提取值范围为 0.582 至 0.756，异质特质-同特质比范围为 0.376 至 0.896。最终模型拟合良好（χ=36.984；P=.08；比较拟合指数 0.991；拟合优度指数 0.957；均方根误差近似值 0.052；标准化均方根残差 0.028）。最终模型中的变量解释了用户接受维度 89%的方差。
用户接受是人工智能支持的临床决策支持系统成功的核心维度。接受直接受到易用性、信息质量、服务质量和感知收益的影响。接受也通过易用性间接受到系统质量和信息质量的影响。用户接受和感知收益相互关联。
