Huo Bright, Collins Gary S, Chartash David, Thirunavukarasu Arun J, Flanagin Annette, Iorio Alfonso, Cacciamani Giovanni, Chen Xi, Liu Nan, Mathur Piyush, Chan An-Wen, Laine Christine, Pacella Daniela, Berkwits Michael, Antoniou Stavros A, Camaradou Jennifer C, Canfield Carolyn, Mittelman Michael, Feeney Timothy, Loder Elizabeth W, Agha Riaz, Saha Ashirbani, Mayol Julio, Sunjaya Anthony, Harvey Hugh, Ng Jeremy Y, McKechnie Tyler, Lee Yung, Verma Nipun, Stiglic Gregor, McCradden Melissa, Ramji Karim, Boudreau Vanessa, Ortenzi Monica, Meerpohl Joerg J, Vandvik Per Olav, Agoritsas Thomas, Samuel Diana, Frankish Helen, Anderson Michael, Yao Xiaomei, Loeb Stacy, Lokker Cynthia, Liu Xiaoxuan, Guallar Eliseo, Guyatt Gordon H
Division of General Surgery, Department of Surgery, McMaster University, Hamilton, Ontario, Canada.
UK EQUATOR Centre, University of Oxford, Oxford, United Kingdom.
JAMA Netw Open. 2025 Aug 1;8(8):e2530220. doi: 10.1001/jamanetworkopen.2025.30220.
The rise in chatbot health advice (CHA) studies is accompanied by heterogeneity in reporting standards, impacting their interpretability.
To provide reporting recommendations for studies evaluating the performance of generative artificial intelligence (AI)-driven chatbots when summarizing clinical evidence and providing health advice.
DESIGN, SETTING, AND PARTICIPANTS: CHART was developed in several phases after performing a comprehensive systematic review to identify variation in the conduct, reporting, and methodology in CHA studies. Findings from the review were used to develop a draft checklist that was revised through an international, multidisciplinary modified asynchronous Delphi consensus process of 531 stakeholders, 3 synchronous panel consensus meetings of 48 stakeholders, and subsequent pilot testing of the checklist.
CHART includes 12 items and 39 subitems to promote transparent and comprehensive reporting of CHA studies. These include title (subitem 1a), abstract or summary (subitem 1b), background (subitems 2ab), model identifiers (subitem 3ab), model details (subitems 4abc), prompt engineering (subitems 5ab), query strategy (subitems 6abcd), performance evaluation (subitems 7ab), sample size (subitem 8), data analysis (subitem 9a), results (subitems 10abc), discussion (subitems 11abc), disclosures (subitem 12a), funding (subitem 12b), ethics (subitem 12c), protocol (subitem 12d), and data availability (subitem 12e).
The CHART checklist and corresponding methodological diagram were designed to support key stakeholders including clinicians, researchers, editors, peer reviewers, and readers in reporting, understanding, and interpreting the findings of CHA studies.
聊天机器人健康建议（CHA）研究数量的增加伴随着报告标准的异质性，影响了这些研究的可解释性。
为评估生成式人工智能（AI）驱动的聊天机器人在总结临床证据和提供健康建议时的性能的研究提供报告建议。
设计、背景和参与者：在对CHA研究的实施、报告和方法的差异进行全面系统评价后，分几个阶段制定了CHA研究报告规范（CHART）。该评价的结果用于制定一份清单草案，通过531名利益相关者参与的国际多学科改良异步德尔菲共识过程、48名利益相关者参与的3次同步小组共识会议以及随后对该清单的试点测试对其进行修订。
CHART包括12项和39个子项，以促进CHA研究的透明和全面报告。这些项目包括标题（子项1a）、摘要（子项1b）、背景（子项2ab）、模型标识符（子项3ab）、模型细节（子项4abc）、提示工程（子项5ab）、查询策略（子项6abcd）、性能评估（子项7ab）、样本量（子项8）、数据分析（子项9a）、结果（子项10abc）、讨论（子项11abc）、披露（子项12a）、资金（子项12b）、伦理（子项12c）、方案（子项12d）和数据可用性（子项12e）。
CHART清单和相应的方法图旨在支持包括临床医生、研究人员、编辑、同行评审人员和读者在内的关键利益相关者报告、理解和解释CHA研究的结果。
