Zhong Jingyu, Xing Yue, Hu Yangfan, Lu Junjie, Yang Jiarui, Zhang Guangcheng, Mao Shiqi, Chen Haoda, Yin Qian, Cen Qingqing, Jiang Run, Chu Jingshen, Song Yang, Lu Minda, Ding Defang, Ge Xiang, Zhang Huan, Yao Weiwu
Laboratory of Key Technology and Materials in Minimally Invasive Spine Surgery, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
Center for Spinal Minimally Invasive Research, Shanghai Jiao Tong University, Shanghai, China.
Insights Imaging. 2024 Aug 1;15(1):186. doi: 10.1186/s13244-024-01769-7.
To evaluate whether and how the radiological journals present their policies on the use of large language models (LLMs), and identify the journal characteristic variables that are associated with the presence.
In this meta-research study, we screened Journals from the Radiology, Nuclear Medicine and Medical Imaging Category, 2022 Journal Citation Reports, excluding journals in non-English languages and relevant documents unavailable. We assessed their LLM use policies: (1) whether the policy is present; (2) whether the policy for the authors, the reviewers, and the editors is present; and (3) whether the policy asks the author to report the usage of LLMs, the name of LLMs, the section that used LLMs, the role of LLMs, the verification of LLMs, and the potential influence of LLMs. The association between the presence of policies and journal characteristic variables was evaluated.
The LLM use policies were presented in 43.9% (83/189) of journals, and those for the authors, the reviewers, and the editor were presented in 43.4% (82/189), 29.6% (56/189) and 25.9% (49/189) of journals, respectively. Many journals mentioned the aspects of the usage (43.4%, 82/189), the name (34.9%, 66/189), the verification (33.3%, 63/189), and the role (31.7%, 60/189) of LLMs, while the potential influence of LLMs (4.2%, 8/189), and the section that used LLMs (1.6%, 3/189) were seldomly touched. The publisher is related to the presence of LLM use policies (p < 0.001).
The presence of LLM use policies is suboptimal in radiological journals. A reporting guideline is encouraged to facilitate reporting quality and transparency.
It may facilitate the quality and transparency of the use of LLMs in scientific writing if a shared complete reporting guideline is developed by stakeholders and then endorsed by journals.
The policies on LLM use in radiological journals are unexplored. Some of the radiological journals presented policies on LLM use. A shared complete reporting guideline for LLM use is desired.
评估放射学杂志是否以及如何呈现其关于使用大语言模型（LLM）的政策，并确定与政策存在相关的期刊特征变量。
在这项元研究中，我们从《放射学、核医学与医学成像分类》2022年期刊引证报告中筛选期刊，排除非英语语言的期刊和无法获取的相关文献。我们评估了它们的大语言模型使用政策：（1）政策是否存在；（2）作者、审稿人和编辑的政策是否存在；（3）政策是否要求作者报告大语言模型的使用情况、大语言模型的名称、使用大语言模型的部分、大语言模型的作用、大语言模型的验证以及大语言模型的潜在影响。评估了政策的存在与期刊特征变量之间的关联。
43.9%（83/189）的期刊呈现了大语言模型使用政策，43.4%（82/189）、29.6%（56/189）和25.9%（49/189）的期刊分别呈现了针对作者、审稿人和编辑的政策。许多期刊提到了大语言模型使用的方面（43.4%，82/189）、名称（34.9%，66/189）、验证（33.3%，63/189）和作用（31.7%，60/189），而大语言模型的潜在影响（4.2%，8/189）和使用大语言模型的部分（1.6%，3/189）很少被涉及。出版商与大语言模型使用政策的存在有关（p < 0.001）。
放射学杂志中大语言模型使用政策的存在情况并不理想。鼓励制定报告指南以提高报告质量和透明度。
如果利益相关者制定并随后由期刊认可一份共享的完整报告指南，可能会促进科学写作中使用大语言模型的质量和透明度。
放射学杂志中大语言模型使用政策尚未得到探索。一些放射学杂志呈现了大语言模型使用政策。需要一份共享的大语言模型使用完整报告指南。
