Hachaj Tomasz, Miazga Justyna
Institute of Computer Science, Pedagogical University of Krakow, 2 Podchorazych Ave, 30-084 Krakow, Poland.
Entropy (Basel). 2020 Nov 30;22(12):1351. doi: 10.3390/e22121351.
Hashtag-based image descriptions are a popular approach for labeling images on social media platforms. In practice, images are often described by more than one hashtag. Due the rapid development of deep neural networks specialized in image embedding and classification, it is now possible to generate those descriptions automatically. In this paper we propose a novel Voting Deep Neural Network with Associative Rules Mining (VDNN-ARM) algorithm that can be used to solve multi-label hashtag recommendation problems. VDNN-ARM is a machine learning approach that utilizes an ensemble of deep neural networks to generate image features, which are then classified to potential hashtag sets. Proposed hashtags are then filtered by a voting schema. The remaining hashtags might be included in a final recommended hashtags dataset by application of associative rules mining, which explores dependencies in certain hashtag groups. Our approach is evaluated on a HARRISON benchmark dataset as a multi-label classification problem. The highest values of our evaluation parameters, including precision, recall, and accuracy, have been obtained for VDNN-ARM with a confidence threshold 0.95. VDNN-ARM outperforms state-of-the-art algorithms, including VGG-Object + VGG-Scene precision by 17.91% as well as ensemble-FFNN (intersection) recall by 32.33% and accuracy by 27.00%. Both the dataset and all source codes we implemented for this research are available for download, and our results can be reproduced.
基于主题标签的图像描述是在社交媒体平台上标记图像的一种流行方法。在实际应用中，图像通常由多个主题标签进行描述。由于专门用于图像嵌入和分类的深度神经网络的快速发展，现在可以自动生成这些描述。在本文中，我们提出了一种新颖的带有关联规则挖掘的投票深度神经网络（VDNN-ARM）算法，该算法可用于解决多标签主题标签推荐问题。VDNN-ARM是一种机器学习方法，它利用深度神经网络的集成来生成图像特征，然后将这些特征分类到潜在的主题标签集中。然后通过投票模式对提出的主题标签进行过滤。通过应用关联规则挖掘，探索某些主题标签组中的依赖性，剩余的主题标签可能会被纳入最终推荐的主题标签数据集中。我们的方法在HARRISON基准数据集上作为多标签分类问题进行评估。对于置信阈值为0.95的VDNN-ARM，我们获得了包括精确率、召回率和准确率在内的评估参数的最高值。VDNN-ARM优于现有算法，包括VGG-Object + VGG-Scene精确率17.91%，以及集成FFNN（交集）召回率32.33%和准确率27.00%。我们为这项研究实现的数据集和所有源代码均可下载，并且我们的结果可以重现。
