Nie Yuzhe
School of Foreign Languages, Shanghai University, Shanghai, China.
PeerJ Comput Sci. 2025 May 28;11:e2909. doi: 10.7717/peerj-cs.2909. eCollection 2025.
Native language identification (NLI) is a critical task in computational linguistics, supporting applications such as personalized language learning, forensic analysis, and machine translation. This study investigates the use of a fine-tuned GPT-2 model to enhance NLI accuracy. Using the NLI-PT dataset, we preprocess and fine-tune GPT-2 to classify the native language of learners based on their Portuguese-written texts. Our approach leverages deep learning techniques, including tokenization, embedding extraction, and multi-layer transformer-based classification. Experimental results show that our fine-tuned GPT-2 model significantly outperforms traditional machine learning methods (., SVM, Random Forest) and other pre-trained language models (., BERT, RoBERTa, BioBERT), achieving a weighted F1 score of 0.9419 and an accuracy of 94.65%. These results show that large transformer models work well for native language identification and can help guide future research in personalized language tools and artificial intelligence (AI)-based education.
母语识别（NLI）是计算语言学中的一项关键任务，为个性化语言学习、法医分析和机器翻译等应用提供支持。本研究调查了使用微调后的GPT-2模型来提高NLI的准确性。使用NLI-PT数据集，我们对GPT-2进行预处理和微调，以便根据学习者的葡萄牙语文本对其母语进行分类。我们的方法利用了深度学习技术，包括词元化、嵌入提取和基于多层变换器的分类。实验结果表明，我们微调后的GPT-2模型显著优于传统机器学习方法（如支持向量机、随机森林）和其他预训练语言模型（如BERT、RoBERTa、BioBERT），加权F1分数达到0.9419，准确率为94.65%。这些结果表明，大型变换器模型在母语识别方面表现良好，有助于指导未来在个性化语言工具和基于人工智能（AI）的教育方面的研究。
