Taniguchi Masahiko, Lindsey Jonathan S
Department of Chemistry, North Carolina State University, Raleigh, North Carolina, USA.
Photochem Photobiol. 2024 Nov 4. doi: 10.1111/php.14037.
The advent of chatbots raises the possibility of a paradigm shift across society including the most technical of fields with regard to access to information, generation of knowledge, and dissemination of education and training. Photochemistry is a scientific endeavor with roots in chemistry and physics and branches that encompass diverse disciplines ranging from astronomy to zoology. Here, five chatbots have each been challenged with 13 photochemically relevant queries. The chatbots included ChatGPT 3.5, ChatGPT 4.0, Copilot, Gemini Advanced, and Meta AI. The queries encompassed fundamental concepts (e.g., "Why is the fluorescence spectrum typically the mirror image of the absorption spectrum?"), practical matters (e.g., "What is the inner filter effect and how to avoid it?"), philosophical matters ("Please create the most important photochemistry questions."), and specific molecular features (e.g., "Why are azo dyes non-fluorescent?"). The chatbots were moderately effective in answering queries concerning fundamental concepts in photochemistry but were glaringly deficient in specialized queries for dyes and fluorophores. In some instances, a correct response was embedded in verbose scientific nonsense whereas in others the entire response, while grammatically correct, was utterly meaningless. The unreliable accuracy makes present chatbots poorly suited for unaided educational purposes and highlights the importance of domain experts.
聊天机器人的出现引发了整个社会范式转变的可能性，包括在获取信息、知识生成以及教育和培训传播等方面的最具技术性的领域。光化学是一项源于化学和物理学的科学事业，其分支涵盖了从天文学到动物学等不同学科。在此，五个聊天机器人分别接受了13个与光化学相关的问题的挑战。这些聊天机器人包括ChatGPT 3.5、ChatGPT 4.0、Copilot、Gemini Advanced和Meta AI。问题涵盖基本概念（例如，“为什么荧光光谱通常是吸收光谱的镜像？”）、实际问题（例如，“什么是内滤光效应以及如何避免它？”）、哲学问题（“请提出最重要的光化学问题。”）以及特定分子特征（例如，“为什么偶氮染料不发荧光？”）。聊天机器人在回答有关光化学基本概念的问题时表现一般，但在关于染料和荧光团的专业问题上明显不足。在某些情况下，正确答案夹杂在冗长的科学废话中，而在其他情况下，整个回答虽然语法正确，但完全没有意义。其准确性不可靠，使得当前的聊天机器人不太适合独立的教育目的，并凸显了领域专家的重要性。
