Division of Health Informatics and Implementation Science, Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, 368 Plantation Street, Worcester, MA, 01605, USA.
Department of Internal Medicine, Wake Forest School of Medicine, Winston-Salem, NC, USA.
BMC Public Health. 2021 Sep 26;21(1):1749. doi: 10.1186/s12889-021-11803-8.
Motivational messaging is a frequently used digital intervention to promote positive health behavior changes, including smoking cessation. Typically, motivational messaging systems have not actively sought feedback on each message, preventing a closer examination of the user-system engagement. This study assessed the granular user-system engagement around a recommender system (a new system that actively sought user feedback on each message to improve message selection) for promoting smoking cessation and the impact of engagement on cessation outcome.
We prospectively followed a cohort of current smokers enrolled to use the recommender system for 6 months. The system sent participants motivational messages to support smoking cessation every 3 days and used machine learning to incorporate user feedback (i.e., user's rating on the perceived influence of each message, collected on a 5-point Likert scale with 1 indicating strong disagreement and 5 indicating strong agreement on perceiving the influence on quitting smoking) to improve the selection of the following message. We assessed user-system engagement by various metrics, including user response rate (i.e., the percent of times a user rated the messages) and the perceived influence of messages. We compared retention rates across different levels of user-system engagement and assessed the association between engagement and the 7-day point prevalence abstinence (missing outcome = smoking) by using multiple logistic regression.
We analyzed data from 731 participants (13% Black; 73% women). The user response rate was 0.24 (SD = 0.34) and user-perceived influence was 3.76 (SD = 0.84). The retention rate positively increased with the user response rate (trend test P < 0.001). Compared with non-response, six-month cessation increased with the levels of response rates: low response rate (odds ratio [OR] = 1.86, 95% confidence interval [CI]: 1.07-3.23), moderate response rate (OR = 2.30, 95% CI: 1.36-3.88), high response rate (OR = 2.69, 95% CI: 1.58-4.58). The association between perceived message influence and the outcome showed a similar pattern.
High user-system engagement was positively associated with both high retention rate and smoking cessation, suggesting that investigation of methods to increase engagement may be crucial to increase the impact of the recommender system for smoking cessation.
Registration Identifier: NCT03224520 . Registration date: July 21, 2017.
激励信息是促进积极健康行为改变的常用数字干预手段，包括戒烟。通常，激励信息系统并未主动针对每条信息征求反馈，从而无法更仔细地检查用户-系统参与度。本研究评估了推荐系统（一种积极针对每条信息征求用户反馈以改善信息选择的新系统）促进戒烟方面的用户-系统参与度的细节，并研究了参与度对戒烟效果的影响。
我们前瞻性地随访了一组注册使用推荐系统 6 个月的当前吸烟者。该系统每 3 天向参与者发送支持戒烟的激励信息，并使用机器学习结合用户反馈（即，用户对每条信息感知影响的评分，在 5 分制李克特量表上进行，1 表示强烈不同意，5 表示强烈同意感知对戒烟的影响）来改善后续信息的选择。我们通过各种指标评估用户-系统参与度，包括用户响应率（即用户对信息进行评分的次数百分比）和信息的感知影响。我们比较了不同用户-系统参与度水平的保留率，并使用多变量逻辑回归评估了参与度与 7 天点戒烟率（缺失结局=吸烟）之间的关联。
我们分析了 731 名参与者的数据（13%为黑人；73%为女性）。用户响应率为 0.24（标准差 [SD] = 0.34），用户感知影响为 3.76（SD = 0.84）。保留率随用户响应率的增加而呈正相关（趋势检验 P<0.001）。与无响应相比，随着响应率水平的提高，6 个月的戒烟率也随之增加：低响应率（比值比 [OR] = 1.86，95%置信区间 [CI]：1.07-3.23）、中响应率（OR = 2.30，95% CI：1.36-3.88）、高响应率（OR = 2.69，95% CI：1.58-4.58）。感知信息影响与结局之间的关联也呈现出相似的模式。
高用户-系统参与度与高保留率和戒烟率均呈正相关，这表明，研究增加参与度的方法可能对提高戒烟推荐系统的效果至关重要。
注册号：NCT03224520。注册日期：2017 年 7 月 21 日。
