NHS Artificial Intelligence Laboratory, NHSX, London, United Kingdom.
Independent Service Design Consultant, London, United Kingdom.
J Med Internet Res. 2021 Sep 8;23(9):e28356. doi: 10.2196/28356.
Digital health interventions (DHIs) have the potential to improve public health by combining effective interventions and population reach. However, what biomedical researchers and digital developers consider an effective intervention differs, thereby creating an ongoing challenge to integrating their respective approaches when evaluating DHIs.
This study aims to report on the Public Health England (PHE) initiative set out to operationalize an evaluation framework that combines biomedical and digital approaches and demonstrates the impact, cost-effectiveness, and benefit of DHIs on public health.
We comprised a multidisciplinary project team including service designers, academics, and public health professionals and used user-centered design methods, such as qualitative research, engagement with end users and stakeholders, and iterative learning. The iterative approach enabled the team to sequentially define the problem, understand user needs, identify opportunity areas, develop concepts, test prototypes, and plan service implementation. Stakeholders, senior leaders from PHE, and a working group critiqued the outputs.
We identified 26 themes and 82 user needs from semistructured interviews (N=15), expressed as 46 Jobs To Be Done, which were then validated across the journey of evaluation design for a DHI. We identified seven essential concepts for evaluating DHIs: evaluation thinking, evaluation canvas, contract assistant, testing toolkit, development history, data hub, and publish health outcomes. Of these, three concepts were prioritized for further testing and development, and subsequently refined into the proposed PHE Evaluation Service for public health DHIs. Testing with PHE's Couch-to-5K app digital team confirmed the viability, desirability, and feasibility of both the evaluation approach and the Evaluation Service.
An iterative, user-centered design approach enabled PHE to combine the strengths of academic and biomedical disciplines with the expertise of nonacademic and digital developers for evaluating DHIs. Design-led methodologies can add value to public health settings. The subsequent service, now known as Evaluating Digital Health Products, is currently in use by health bodies in the United Kingdom and is available to others for tackling the problem of evaluating DHIs pragmatically and responsively.
数字健康干预措施（DHIs）通过结合有效干预措施和覆盖人群，有潜力改善公共卫生。然而，生物医学研究人员和数字开发者对有效干预措施的看法不同，这就给评估 DHIs 时整合各自方法带来了持续的挑战。
本研究旨在报告英国公共卫生部（PHE）的一项倡议，该倡议旨在实施一个结合生物医学和数字方法的评估框架，并展示 DHIs 对公共卫生的影响、成本效益和益处。
我们组成了一个多学科项目团队，包括服务设计师、学者和公共卫生专业人员，并使用了以用户为中心的设计方法，如定性研究、与最终用户和利益相关者的互动以及迭代学习。迭代方法使团队能够依次定义问题、了解用户需求、确定机会领域、开发概念、测试原型以及规划服务实施。利益相关者、PHE 的高级领导人和一个工作组对产出进行了评论。
我们从半结构化访谈（N=15）中确定了 26 个主题和 82 个用户需求，表达为 46 个待完成的工作，然后在 DHIs 的评估设计过程中对其进行了验证。我们确定了评估 DHIs 的七个基本概念：评估思维、评估画布、合同助手、测试工具包、开发历史、数据中心和发布健康结果。其中，三个概念被优先用于进一步测试和开发，随后被细化为拟议的 PHE 公共卫生数字干预措施评估服务。与 PHE 的“从沙发到 5K”应用程序数字团队的测试证实了评估方法和评估服务的可行性、可取性和可行性。
迭代、以用户为中心的设计方法使 PHE 能够将学术和生物医学学科的优势与非学术和数字开发者的专业知识结合起来，用于评估 DHIs。设计主导的方法可以为公共卫生环境增加价值。随后的服务，现在称为“评估数字健康产品”，目前正在英国的卫生机构中使用，并可供其他机构用于务实和响应地解决评估 DHIs 的问题。
