Flaherty Leah Russell, Oliver Kendra H
Allegheny Health Network, Psychiatry and Behavioral Health Instiute, Pittsburgh, PA, USA.
Responsible Data Science, Office of the Provost, University of Pittsburgh, Pittsburgh, PA, USA.
Pain Rep. 2024 Dec 24;10(1):e1219. doi: 10.1097/PR9.0000000000001219. eCollection 2025 Feb.
Interpretation and utilization of qualitative feedback from participants has immense value for program evaluation. Reliance on only quantitative data runs the risk of losing the lived patient experience, forcing their outcomes to fit into our predefined objectives.
Using large language models (LLMs), program directors may begin to employ rich, qualitative feedback expediently.
This study provides an example of the feasibility of evaluating patient responses (n = 82) to Empowered Relief, a skill-based pain education class using LLMs. We utilized a dual-method analytical approach, with both LLM-assisted and supported manual thematic review.
The thematic analysis of qualitative data using ChatGPT yielded 7 major themes: (1) Use of Specific Audiofile; (2) Mindset; (3) Technique; (4) Community and Space; (5) Knowledge; (6) Tools and Approaches; and (7) Self-awareness.
Findings from the LLM-derived analysis provided rich and unexpected information, valuable to the program and the field of pain psychology by employing the set of patients' own words to guide program evaluation. Program directors may benefit from evaluating treatment outcomes on a broader scale such as this rather than focusing solely on improvements in disability. These insights would only be uncovered with open-ended data, and although potentially more insights could emerge with the help of a qualitative research team, ChatGPT offered an ergonomic solution.
对参与者的定性反馈进行解读和利用，对项目评估具有巨大价值。仅依赖定量数据存在失去患者真实体验的风险，迫使他们的结果符合我们预先设定的目标。
通过使用大语言模型（LLMs），项目负责人或许能够开始便捷地运用丰富的定性反馈。
本研究提供了一个实例，展示了使用大语言模型评估患者（n = 82）对“赋能缓解”（一种基于技能的疼痛教育课程）的反应的可行性。我们采用了双方法分析途径，包括大语言模型辅助和支持的手动主题审查。
使用ChatGPT对定性数据进行主题分析得出了7个主要主题：（1）特定音频文件的使用；（2）心态；（3）技巧；（4）社区与空间；（5）知识；（6）工具与方法；以及（7）自我意识。
大语言模型衍生分析的结果提供了丰富且出人意料的信息，通过运用患者自己的话语来指导项目评估，对该项目和疼痛心理学领域具有重要价值。项目负责人可能会从如此更广泛的规模评估治疗结果中受益，而不是仅仅关注残疾状况的改善。这些见解只有通过开放式数据才能发现，尽管在定性研究团队的帮助下可能会出现更多见解，但ChatGPT提供了一个符合人体工程学的解决方案。
