Department of Radiology, Charité Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Berlin, Germany.
Department of Radiology, Stanford University, Stanford, CA, United States.
J Med Internet Res. 2023 Mar 16;25:e43110. doi: 10.2196/43110.
Generative models, such as DALL-E 2 (OpenAI), could represent promising future tools for image generation, augmentation, and manipulation for artificial intelligence research in radiology, provided that these models have sufficient medical domain knowledge. Herein, we show that DALL-E 2 has learned relevant representations of x-ray images, with promising capabilities in terms of zero-shot text-to-image generation of new images, the continuation of an image beyond its original boundaries, and the removal of elements; however, its capabilities for the generation of images with pathological abnormalities (eg, tumors, fractures, and inflammation) or computed tomography, magnetic resonance imaging, or ultrasound images are still limited. The use of generative models for augmenting and generating radiological data thus seems feasible, even if the further fine-tuning and adaptation of these models to their respective domains are required first.
生成模型，如 DALL-E 2（OpenAI），可以代表未来在放射学人工智能研究中用于图像生成、增强和操作的有前途的工具，前提是这些模型具有足够的医学领域知识。在这里，我们表明 DALL-E 2 已经学习了 X 射线图像的相关表示，具有在零样本文本到新图像生成、图像超出其原始边界的延续以及元素去除方面的有前途的能力；然而，它生成具有病理异常（例如肿瘤、骨折和炎症）或计算机断层扫描、磁共振成像或超声图像的能力仍然有限。因此，即使需要首先对这些模型进行进一步的微调和适应，使用生成模型来增强和生成放射学数据似乎也是可行的。
