Zuo Chen, Yang Xiaohao, Errickson Josh, Li Jiayang, Hong Yi, Wang Runzi
School for Environment and Sustainability, University of Michigan, Ann Arbor, USA.
School of Information, University of Michigan, Ann Arbor, USA.
Environ Evid. 2025 Apr 15;14(1):5. doi: 10.1186/s13750-025-00358-5.
Systematic reviews (SRs) in environmental science is challenging due to diverse methodologies, terminologies, and study designs across disciplines. A major limitation is that inconsistent application of eligibility criteria in evidence-screening affects the reproducibility and transparency of SRs. To explore the potential role of Artificial Intelligence (AI) in applying eligibility criteria, we developed and evaluated an AI-assisted evidence-screening framework using a case study SR on the relationship between stream fecal coliform concentrations and land use and land cover (LULC). The SR incorporates publications from hydrology, ecology, public health, landscape, and urban planning, reflecting the interdisciplinary nature of environmental research. We fine-tuned ChatGPT-3.5 Turbo model with expert-reviewed training data for title, abstract, and full-text screening of 120 articles. The AI model demonstrated substantial agreement at title/abstract review and moderate agreement at full-text review with expert reviewers and maintained internal consistency, suggesting its potential for structured screening assistance. The findings provide a structured framework for applying eligibility criteria consistently, improving evidence screening efficiency, reducing labor and costs, and informing large language models (LLMs) integration in environmental SRs. Combining AI with domain knowledge provides an exploratory step to evaluate feasibility of AI-assisted evidence screening, especially for diverse, large volume, and interdisciplinary studies. Additionally, AI-assisted screening has the potential to provide a structured approach for managing disagreement among researchers with diverse domain knowledge, though further validation is needed.
由于环境科学中各学科的方法、术语和研究设计各不相同，因此进行系统综述（SRs）具有挑战性。一个主要限制是，在证据筛选中资格标准的不一致应用会影响系统综述的可重复性和透明度。为了探索人工智能（AI）在应用资格标准方面的潜在作用，我们通过一个关于河流粪便大肠菌群浓度与土地利用和土地覆盖（LULC）关系的案例研究系统综述，开发并评估了一个人工智能辅助的证据筛选框架。该系统综述纳入了来自水文、生态、公共卫生、景观和城市规划等领域的出版物，反映了环境研究的跨学科性质。我们使用经过专家评审的训练数据对ChatGPT-3.5 Turbo模型进行了微调，用于对120篇文章的标题、摘要和全文进行筛选。该人工智能模型在标题/摘要评审中与专家评审员表现出高度一致性，在全文评审中表现出中等一致性，并保持了内部一致性，表明其在结构化筛选辅助方面的潜力。研究结果提供了一个结构化框架，用于一致地应用资格标准，提高证据筛选效率，降低劳动强度和成本，并为在环境系统综述中整合大语言模型（LLMs）提供参考。将人工智能与领域知识相结合，为评估人工智能辅助证据筛选的可行性提供了一个探索性步骤，特别是对于多样化、大量和跨学科的研究。此外，人工智能辅助筛选有可能为管理具有不同领域知识的研究人员之间的分歧提供一种结构化方法，不过还需要进一步验证。
