Rampal Nakul, Wang Kaiyu, Burigana Matthew, Hou Lingxiang, Al-Johani Juri, Sackmann Anna, Murayshid Hanan S, AlSumari Walaa A, AlAbdulkarim Arwa M, Alhazmi Nahla E, Alawad Majed O, Borgs Christian, Chayes Jennifer T, Yaghi Omar M
Department of Chemistry, University of California, Berkeley, California 94720, United States.
Kavli Energy Nanoscience Institute, University of California, Berkeley, California 94720, United States.
J Chem Theory Comput. 2024 Oct 22;20(20):9128-9137. doi: 10.1021/acs.jctc.4c00805. Epub 2024 Oct 8.
The rapid advancement in artificial intelligence and natural language processing has led to the development of large-scale datasets aimed at benchmarking the performance of machine learning models. Herein, we introduce "RetChemQA", a comprehensive benchmark dataset designed to evaluate the capabilities of such models in the domain of reticular chemistry. This dataset includes both single-hop and multi-hop question-answer pairs, encompassing approximately 45,000 question and answers (Q&As) for each type. The questions have been extracted from an extensive corpus of literature containing about 2,530 research papers from publishers including NAS, ACS, RSC, Elsevier, and Nature Publishing Group, among others. The dataset has been generated using OpenAI's GPT-4 Turbo, a cutting-edge model known for its exceptional language understanding and generation capabilities. In addition to the Q&A dataset, we also release a dataset of synthesis conditions extracted from the corpus of literature used in this study. The aim of RetChemQA is to provide a robust platform for the development and evaluation of advanced machine learning algorithms, particularly for the reticular chemistry community. The dataset is structured to reflect the complexities and nuances of real-world scientific discourse, thereby enabling nuanced performance assessments across a variety of tasks.
人工智能和自然语言处理的快速发展催生了旨在对机器学习模型性能进行基准测试的大规模数据集。在此，我们介绍“RetChemQA”，这是一个全面的基准数据集，旨在评估此类模型在网状化学领域的能力。该数据集包括单跳和多跳问答对，每种类型大约有45,000个问答（Q&A）。这些问题是从大量文献语料库中提取的，该语料库包含来自美国国家科学院（NAS）、美国化学学会（ACS）、皇家化学学会（RSC）、爱思唯尔（Elsevier）和自然出版集团等出版商的约2,530篇研究论文。该数据集是使用OpenAI的GPT - 4 Turbo生成的，这是一个以其卓越的语言理解和生成能力而闻名的前沿模型。除了问答数据集，我们还发布了一个从本研究使用的文献语料库中提取的合成条件数据集。RetChemQA的目的是为先进机器学习算法的开发和评估提供一个强大的平台，特别是为网状化学领域。该数据集的结构反映了现实世界科学论述的复杂性和细微差别，从而能够对各种任务进行细致入微的性能评估。
