Khan Nek Dil, Khan Javed Ali, Li Jianqiang, Ullah Tahir, Zhao Qing
Faculty of Information Technology, Beijing University of Technology, Beijing, China.
Department of Computer Science, School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield, United Kingdom.
PeerJ Comput Sci. 2024 Jul 10;10:e2115. doi: 10.7717/peerj-cs.2115. eCollection 2024.
In today's digital world, app stores have become an essential part of software distribution, providing customers with a wide range of applications and opportunities for software developers to showcase their work. This study elaborates on the importance of end-user feedback for software evolution. However, in the literature, more emphasis has been given to high-rating & popular software apps while ignoring comparatively low-rating apps. Therefore, the proposed approach focuses on end-user reviews collected from 64 low-rated apps representing 14 categories in the Amazon App Store. We critically analyze feedback from low-rating apps and developed a grounded theory to identify various concepts important for software evolution and improving its quality including user interface (UI) and user experience (UX), functionality and features, compatibility and device-specific, performance and stability, customer support and responsiveness and security and privacy issues. Then, using a grounded theory and content analysis approach, a novel research dataset is curated to evaluate the performance of baseline machine learning (ML), and state-of-the-art deep learning (DL) algorithms in automatically classifying end-user feedback into frequently occurring issues. Various natural language processing and feature engineering techniques are utilized for improving and optimizing the performance of ML and DL classifiers. Also, an experimental study comparing various ML and DL algorithms, including multinomial naive Bayes (MNB), logistic regression (LR), random forest (RF), multi-layer perception (MLP), k-nearest neighbors (KNN), AdaBoost, Voting, convolutional neural network (CNN), long short-term memory (LSTM), bidirectional long short term memory (BiLSTM), gated recurrent unit (GRU), bidirectional gated recurrent unit (BiGRU), and recurrent neural network (RNN) classifiers, achieved satisfactory results in classifying end-user feedback to commonly occurring issues. Whereas, MLP, RF, BiGRU, GRU, CNN, LSTM, and Classifiers achieved average accuracies of 94%, 94%, 92%, 91%, 90%, 89%, and 89%, respectively. We employed the SHAP approach to identify the critical features associated with each issue type to enhance the explainability of the classifiers. This research sheds light on areas needing improvement in low-rated apps and opens up new avenues for developers to improve software quality based on user feedback.
在当今的数字世界中，应用商店已成为软件分发的重要组成部分，为客户提供了广泛的应用程序，并为软件开发人员提供了展示其作品的机会。本研究阐述了终端用户反馈对软件演进的重要性。然而，在文献中，更多的重点放在了高评分和流行的软件应用程序上，而忽略了评分相对较低的应用程序。因此，所提出的方法侧重于从亚马逊应用商店中代表14个类别的64个低评分应用程序收集的终端用户评论。我们对低评分应用程序的反馈进行了批判性分析，并开发了一种扎根理论，以识别对软件演进和提高其质量重要的各种概念，包括用户界面（UI）和用户体验（UX）、功能和特性、兼容性和特定设备、性能和稳定性、客户支持和响应能力以及安全和隐私问题。然后，使用扎根理论和内容分析方法，精心策划了一个新颖的研究数据集，以评估基线机器学习（ML）和最先进的深度学习（DL）算法在将终端用户反馈自动分类为常见问题方面的性能。利用各种自然语言处理和特征工程技术来提高和优化ML和DL分类器的性能。此外，一项比较各种ML和DL算法的实验研究，包括多项式朴素贝叶斯（MNB）、逻辑回归（LR）、随机森林（RF）、多层感知器（MLP）、k近邻（KNN）、AdaBoost、投票、卷积神经网络（CNN）、长短期记忆（LSTM）、双向长短期记忆（BiLSTM）、门控循环单元（GRU）、双向门控循环单元（BiGRU）和递归神经网络（RNN）分类器。在将终端用户反馈分类为常见问题方面取得了令人满意的结果。而MLP、RF、BiGRU、GRU、CNN、LSTM和分类器的平均准确率分别达到了94%、94%、92%、91%、90%、89%和89%。我们采用SHAP方法来识别与每种问题类型相关的关键特征，以增强分类器的可解释性。这项研究揭示了低评分应用程序中需要改进的领域，并为开发人员根据用户反馈提高软件质量开辟了新途径。
