Informatics Institute, University of Amsterdam, 1090 GH Amsterdam, The Netherlands.
Department of Information and Computing Sciences, Utrecht University, 3584 CC Utrecht, The Netherlands.
Int J Environ Res Public Health. 2021 Jun 4;18(11):6059. doi: 10.3390/ijerph18116059.
Just-in-time adaptive intervention (JITAI) has gained attention recently and previous studies have indicated that it is an effective strategy in the field of mobile healthcare intervention. Identifying the right moment for the intervention is a crucial component. In this paper the reinforcement learning (RL) technique has been used in a smartphone exercise application to promote physical activity. This RL model determines the 'right' time to deliver a restricted number of notifications adaptively, with respect to users' temporary context information (i.e., time and calendar). A four-week trial study was conducted to examine the feasibility of our model with real target users. JITAI reminders were sent by the RL model in the fourth week of the intervention, while the participants could only access the app's other functionalities during the first 3 weeks. Eleven target users registered for this study, and the data from 7 participants using the application for 4 weeks and receiving the intervening reminders were analyzed. Not only were the reaction behaviors of users after receiving the reminders analyzed from the application data, but the user experience with the reminders was also explored in a questionnaire and exit interviews. The results show that 83.3% reminders sent at adaptive moments were able to elicit user reaction within 50 min, and 66.7% of physical activities in the intervention week were performed within 5 h of the delivery of a reminder. Our findings indicated the usability of the RL model, while the timing of the moments to deliver reminders can be further improved based on lessons learned.
即时自适应干预（JITAI）最近受到关注，先前的研究表明它是移动医疗干预领域的一种有效策略。确定干预的正确时机是一个关键组成部分。本文在智能手机锻炼应用中使用强化学习（RL）技术来促进身体活动。该 RL 模型根据用户的临时上下文信息（即时间和日历）自适应地确定发送有限数量通知的“正确”时间。进行了为期四周的试验研究，以使用真实目标用户检验我们模型的可行性。RL 模型在干预的第四周发送 JITAI 提醒，而参与者只能在前 3 周访问应用程序的其他功能。有 11 名目标用户注册参加了这项研究，分析了 7 名参与者使用该应用程序 4 周并收到干预提醒的数据。不仅从应用程序数据中分析了用户收到提醒后的反应行为，还通过问卷调查和退出访谈探讨了用户对提醒的体验。结果表明，在自适应时刻发送的 83.3%提醒能在 50 分钟内引起用户反应，并且干预周内的 66.7%身体活动是在提醒发送后 5 小时内进行的。我们的研究结果表明 RL 模型是可行的，但是根据经验教训可以进一步改进发送提醒的时机。
