Tucudean Georgiana, Bucos Marian, Dragulescu Bogdan, Caleanu Catalin Daniel
Communications Department, Politehnica University Timișoara, Timișoara, Timiș, România.
Applied Electronics Department, Politehnica University Timișoara, Timișoara, Timiș, România.
PeerJ Comput Sci. 2024 Aug 7;10:e2222. doi: 10.7717/peerj-cs.2222. eCollection 2024.
Natural language processing (NLP) tasks can be addressed with several deep learning architectures, and many different approaches have proven to be efficient. This study aims to briefly summarize the use cases for NLP tasks along with the main architectures. This research presents transformer-based solutions for NLP tasks such as Bidirectional Encoder Representations from Transformers (BERT), and Generative Pre-Training (GPT) architectures. To achieve that, we conducted a step-by-step process in the review strategy: identify the recent studies that include Transformers, apply filters to extract the most consistent studies, identify and define inclusion and exclusion criteria, assess the strategy proposed in each study, and finally discuss the methods and architectures presented in the resulting articles. These steps facilitated the systematic summarization and comparative analysis of NLP applications based on Transformer architectures. The primary focus is the current state of the NLP domain, particularly regarding its applications, language models, and data set types. The results provide insights into the challenges encountered in this research domain.
自然语言处理（NLP）任务可以通过多种深度学习架构来解决，并且许多不同的方法已被证明是有效的。本研究旨在简要总结NLP任务的用例以及主要架构。本研究提出了用于NLP任务的基于Transformer的解决方案，如来自Transformer的双向编码器表示（BERT）和生成式预训练（GPT）架构。为了实现这一目标，我们在综述策略中进行了一个循序渐进的过程：识别最近包含Transformer的研究，应用过滤器提取最一致的研究，识别和定义纳入和排除标准，评估每项研究中提出的策略，最后讨论所得文章中提出的方法和架构。这些步骤有助于基于Transformer架构对NLP应用进行系统的总结和比较分析。主要重点是NLP领域的当前状态，特别是关于其应用、语言模型和数据集类型。研究结果为该研究领域中遇到的挑战提供了见解。
