Yanagita Yasutaka, Yokokawa Daiki, Uchida Shun, Li Yu, Uehara Takanori, Ikusaka Masatomi
Department of General Medicine, Chiba University Hospital, Chiba, Japan.
Uchida Internal Medicine Clinic, Saitama, Japan.
J Gen Intern Med. 2024 Dec;39(16):3282-3289. doi: 10.1007/s11606-024-09031-y. Epub 2024 Sep 23.
Creating clinical vignettes requires considerable effort. Recent developments in generative artificial intelligence (AI) for natural language processing have been remarkable and may allow for the easy and immediate creation of diverse clinical vignettes.
In this study, we evaluated the medical accuracy and grammatical correctness of AI-generated clinical vignettes in Japanese and verified their usefulness.
Clinical vignettes were created using the generative AI model GPT-4-0613. The input prompts for the clinical vignettes specified the following seven elements: (1) age, (2) sex, (3) chief complaint and time course since onset, (4) physical findings, (5) examination results, (6) diagnosis, and (7) treatment course. The list of diseases integrated into the vignettes was based on 202 cases considered in the management of diseases and symptoms in Japan's Primary Care Physicians Training Program. The clinical vignettes were evaluated for medical and Japanese-language accuracy by three physicians using a five-point scale. A total score of 13 points or above was defined as "sufficiently beneficial and immediately usable with minor revisions," a score between 10 and 12 points was defined as "partly insufficient and in need of modifications," and a score of 9 points or below was defined as "insufficient."
Regarding medical accuracy, of the 202 clinical vignettes, 118 scored 13 points or above, 78 scored between 10 and 12 points, and 6 scored 9 points or below. Regarding Japanese-language accuracy, 142 vignettes scored 13 points or above, 56 scored between 10 and 12 points, and 4 scored 9 points or below. Overall, 97% (196/202) of vignettes were available with some modifications.
Overall, 97% of the clinical vignettes proved practically useful, based on confirmation and revision by Japanese medical physicians. Given the significant effort required by physicians to create vignettes without AI, using GPT is expected to greatly optimize this process.
创建临床病例需要付出相当大的努力。生成式人工智能（AI）在自然语言处理方面的最新进展显著，可能使多样化临床病例的轻松、即时创建成为可能。
在本研究中，我们评估了日语中人工智能生成的临床病例的医学准确性和语法正确性，并验证了它们的实用性。
使用生成式人工智能模型GPT - 4 - 0613创建临床病例。临床病例的输入提示指定了以下七个要素：（1）年龄，（2）性别，（3）主诉及发病后的病程，（4）体格检查结果，（5）检查结果，（6）诊断，以及（7）治疗过程。纳入病例的疾病列表基于日本初级保健医生培训计划中所考虑的202种疾病和症状。由三名医生使用五分制对临床病例的医学和日语准确性进行评估。总分13分及以上被定义为“足够有益且稍作修改即可立即使用”，10至12分被定义为“部分不足且需要修改”，9分及以下被定义为“不足”。
在医学准确性方面，202个临床病例中，118个得分13分及以上，78个得分在10至12分之间，6个得分9分及以下。在日语准确性方面，142个病例得分13分及以上，56个得分在10至12分之间，4个得分9分及以下。总体而言，97%（196/202）的病例稍作修改即可使用。
总体而言，经日本医学医生确认和修订，97%的临床病例被证明具有实际用途。鉴于医生在没有人工智能的情况下创建病例需要付出巨大努力，使用GPT有望极大地优化这一过程。
