M.S. Ramaiah Medical College, M.S. Ramaiah Nagar, Bangalore, Karnataka, 560054, India.
Trauma Imaging Research and Innovation Center, Department of Radiology, Brigham and Women's Hospital, 75 Francis St, Boston, MA, 02115, USA.
Emerg Radiol. 2021 Jun;28(3):477-483. doi: 10.1007/s10140-020-01885-z. Epub 2021 Jan 18.
The purpose of this study was to develop an automated process to analyze multimedia content on Twitter during the COVID-19 outbreak and classify content for radiological significance using deep learning (DL).
Using Twitter search features, all tweets containing keywords from both "radiology" and "COVID-19" were collected for the period January 01, 2020 up to April 24, 2020. The resulting dataset comprised of 8354 tweets. Images were classified as (i) images with text (ii) radiological content (e.g., CT scan snapshots, X-ray images), and (iii) non-medical content like personal images or memes. We trained our deep learning model using Convolutional Neural Networks (CNN) on training dataset of 1040 labeled images drawn from all three classes. We then trained another DL classifier for segmenting images into categories based on human anatomy. All software used is open-source and adapted for this research. The diagnostic performance of the algorithm was assessed by comparing results on a test set of 1885 images.
Our analysis shows that in COVID-19 related tweets on radiology, nearly 32% had textual images, another 24% had radiological content, and 44% were not of radiological significance. Our results indicated a 92% accuracy in classifying images originally labeled as chest X-ray or chest CT and a nearly 99% accurate classification of images containing medically relevant text. With larger training dataset and algorithmic tweaks, the accuracy can be further improved.
Applying DL on rich textual images and other metadata in tweets we can process and classify content for radiological significance in real time.
本研究旨在开发一种自动化流程，使用深度学习（DL）分析 COVID-19 爆发期间 Twitter 上的多媒体内容，并对具有放射学意义的内容进行分类。
利用 Twitter 搜索功能，收集 2020 年 1 月 1 日至 2020 年 4 月 24 日期间包含“放射学”和“COVID-19”两个关键词的所有推文。由此产生的数据集包含 8354 条推文。将图像分为以下三类：(i)带文字的图像；(ii)放射学内容（如 CT 扫描快照、X 射线图像）；(iii)非医学内容，如个人图像或模因。我们使用卷积神经网络（CNN）在来自这三个类别的 1040 张标记图像的训练数据集上训练我们的深度学习模型。然后，我们训练了另一个基于深度学习的分类器，用于根据人体解剖结构将图像分为不同类别。所有使用的软件均为开源软件，并为这项研究进行了适配。该算法的诊断性能通过比较 1885 张测试图像的结果来评估。
我们的分析表明，在与放射学相关的 COVID-19 推文上，近 32%的推文含有文字图像，24%的推文含有放射学内容，44%的推文不具有放射学意义。我们的结果表明，对最初标记为胸部 X 光或胸部 CT 的图像进行分类的准确率为 92%，对含有医学相关文字的图像进行分类的准确率接近 99%。通过使用更大的训练数据集和算法调整，准确率可以进一步提高。
通过对推文丰富的文字图像和其他元数据应用深度学习，我们可以实时处理和分类具有放射学意义的内容。
