The Australian e-Health Research Centre, CSIRO Health and Biosecurity, Brisbane, Australia.
The Australian e-Health Research Centre, CSIRO Health and Biosecurity, Brisbane, Australia.
Artif Intell Med. 2023 Oct;144:102633. doi: 10.1016/j.artmed.2023.102633. Epub 2023 Aug 19.
Automatically generating a report from a patient's Chest X-rays (CXRs) is a promising solution to reducing clinical workload and improving patient care. However, current CXR report generators-which are predominantly encoder-to-decoder models-lack the diagnostic accuracy to be deployed in a clinical setting. To improve CXR report generation, we investigate warm starting the encoder and decoder with recent open-source computer vision and natural language processing checkpoints, such as the Vision Transformer (ViT) and PubMedBERT. To this end, each checkpoint is evaluated on the MIMIC-CXR and IU X-ray datasets. Our experimental investigation demonstrates that the Convolutional vision Transformer (CvT) ImageNet-21K and the Distilled Generative Pre-trained Transformer 2 (DistilGPT2) checkpoints are best for warm starting the encoder and decoder, respectively. Compared to the state-of-the-art (M Transformer Progressive), CvT2DistilGPT2 attained an improvement of 8.3% for CE F-1, 1.8% for BLEU-4, 1.6% for ROUGE-L, and 1.0% for METEOR. The reports generated by CvT2DistilGPT2 have a higher similarity to radiologist reports than previous approaches. This indicates that leveraging warm starting improves CXR report generation. Code and checkpoints for CvT2DistilGPT2 are available at https://github.com/aehrc/cvt2distilgpt2.
从患者的胸部 X 光片（CXRs）自动生成报告是减少临床工作量和改善患者护理的有前途的解决方案。然而，目前的 CXR 报告生成器——主要是编码器-解码器模型——缺乏在临床环境中部署的诊断准确性。为了提高 CXR 报告生成的质量，我们研究了使用最近的开源计算机视觉和自然语言处理检查点（如 Vision Transformer（ViT）和 PubMedBERT）来预热编码器和解码器。为此，我们在 MIMIC-CXR 和 IU X-ray 数据集上评估了每个检查点。我们的实验研究表明，卷积视觉 Transformer（CvT）ImageNet-21K 和蒸馏生成预训练 Transformer 2（DistilGPT2）检查点分别是预热编码器和解码器的最佳选择。与最先进的（M Transformer Progressive）相比，CvT2DistilGPT2 在 CE F-1 上提高了 8.3%，在 BLEU-4 上提高了 1.8%，在 ROUGE-L 上提高了 1.6%，在 METEOR 上提高了 1.0%。CvT2DistilGPT2 生成的报告与放射科医生的报告具有更高的相似性，优于以前的方法。这表明利用预热可以提高 CXR 报告生成的质量。CvT2DistilGPT2 的代码和检查点可在 https://github.com/aehrc/cvt2distilgpt2 上获得。
