Hussain Sajid, Malik Muhammad Shahid Iqbal, Masood Nayyer
Department of Computer Science, Capital university of Science and Technology, Islamabad, Pakistan.
PeerJ Comput Sci. 2022 Dec 12;8:e1169. doi: 10.7717/peerj-cs.1169. eCollection 2022.
Automatic identification of offensive/abusive language is very necessary to get rid of unwanted behavior. However, it is more challenging to generalize the solution due to the different grammatical structures and vocabulary of each language. Most of the prior work targeted western languages, however, one study targeted a low-resource language (Urdu). The prior study used basic linguistic features and a small dataset. This study designed a new dataset (collected from popular Pakistani Facebook pages) containing 7,500 posts for offensive language detection in Urdu. The proposed methodology used four types of feature engineering models: three are frequency-based and the fourth one is the embedding model. Frequency-based are either determined by the term frequency-inverse document frequency (TF-IDF) or bag-of-words or word n-gram feature vectors. The fourth is generated by the word2vec model, trained on the Urdu embeddings using a corpus of 196,226 Facebook posts. The experiments demonstrate that the stacking-based ensemble model with word2vec shows the best performance as a standalone model by achieving 88.27% accuracy. In addition, the wrapper-based feature selection method further improves performance. The hybrid combination of TF-IDF, bag-of-words, and word2vec feature models achieved 90% accuracy and 97% AUC. In addition, it outperformed the baseline with an improvement of 3.55% in accuracy, 3.68% in the recall, 3.60% in f1-measure, 3.67% in precision, and 2.71% in AUC. The findings of this research provide practical implications for commercial applications and future research.
自动识别冒犯性/辱骂性语言对于消除不良行为非常必要。然而，由于每种语言的语法结构和词汇不同，推广解决方案更具挑战性。大多数先前的工作针对西方语言，不过，有一项研究针对的是一种资源匮乏的语言（乌尔都语）。先前的研究使用了基本的语言特征和一个小数据集。本研究设计了一个新的数据集（从巴基斯坦流行的脸书页面收集），包含7500条帖子，用于乌尔都语中的冒犯性语言检测。所提出的方法使用了四种类型的特征工程模型：三种基于频率，第四种是嵌入模型。基于频率的模型要么由词频-逆文档频率（TF-IDF）、词袋模型或词n-gram特征向量确定。第四种由word2vec模型生成，该模型使用196226条脸书帖子的语料库在乌尔都语嵌入上进行训练。实验表明，基于word2vec的堆叠集成模型作为独立模型表现最佳，准确率达到88.27%。此外，基于包装器的特征选择方法进一步提高了性能。TF-IDF、词袋模型和word2vec特征模型的混合组合实现了90%的准确率和97%的AUC。此外，它优于基线，准确率提高了3.55%，召回率提高了3.68%，F1值提高了3.60%，精确率提高了3.67%，AUC提高了2.71%。本研究的结果为商业应用和未来研究提供了实际意义。
