Piastra Marco, Catellani Patrizia
Department of Industrial, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy.
Department of Psychology, Catholic University of the Sacred Heart, Milan, Italy.
Front Artif Intell. 2025 Feb 13;8:1484260. doi: 10.3389/frai.2025.1484260. eCollection 2025.
This study investigates the potential of ChatGPT 4 in the assessment of personality traits based on written texts. Using two publicly available datasets containing both written texts and self-assessments of the authors' psychological traits based on the Big Five model, we aimed to evaluate the predictive performance of ChatGPT 4. For each sample text, we asked for numerical predictions on an eleven-point scale and compared them with the self-assessments. We also asked for ChatGPT 4 confidence scores on an eleven-point scale for each prediction. To keep the study within a manageable scope, a zero-prompt modality was chosen, although more sophisticated prompting strategies could potentially improve performance. The results show that ChatGPT 4 has moderate but significant abilities to automatically infer personality traits from written text. However, it also shows limitations in recognizing whether the input text is appropriate or representative enough to make accurate inferences, which could hinder practical applications. Furthermore, the results suggest that improved benchmarking methods could increase the efficiency and reliability of the evaluation process. These results pave the way for a more comprehensive evaluation of the capabilities of Large Language Models in assessing personality traits from written texts.
本研究探讨了ChatGPT 4基于书面文本评估人格特质的潜力。我们使用了两个公开可用的数据集，其中既包含书面文本，也包含作者基于大五模型对自身心理特质的自我评估，旨在评估ChatGPT 4的预测性能。对于每个样本文本，我们要求其在十一点量表上给出数值预测，并将这些预测与自我评估进行比较。我们还要求ChatGPT 4针对每个预测在十一点量表上给出置信度分数。为了使研究保持在可控范围内，我们选择了零提示模式，尽管更复杂的提示策略可能会提高性能。结果表明，ChatGPT 4具有一定但显著的能力，能够从书面文本中自动推断人格特质。然而，它在识别输入文本是否足够合适或具有代表性以进行准确推断方面也存在局限性，这可能会阻碍实际应用。此外，结果表明改进的基准测试方法可以提高评估过程的效率和可靠性。这些结果为更全面地评估大语言模型从书面文本中评估人格特质的能力铺平了道路。
