Yu Jiahui, Ma Tianyu, Chen Feng, Zhang Jing, Xu Yingke
Department of Biomedical Engineering, MOE Key Laboratory of Biomedical Engineering, State Key Laboratory of Extreme Photonics and Instrumentation, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou, China.
Innovation Center for Smart Medical Technologies and Devices, Binjiang Institute of Zhejiang University, Hangzhou, China.
Commun Biol. 2024 Dec 4;7(1):1619. doi: 10.1038/s42003-024-07303-1.
Microscopy is an indispensable tool for collecting biomedical information in pathological diagnosis, but manual annotation, measurement and interpretation are labor-intensive and costly. Here, we propose a task-driven framework powered by large models that excel in visual analysis and real-time control, paving the way for the next generation of microscopes. We achieve proof-of-concept success on clinical tasks, specifically in adaptive analysis of H&E-stained liver tissue slides. This work demonstrates the advanced capabilities for future digital pathology, setting a new standard for intelligent, efficient, and real-time analysis in clinical applications.
显微镜检查是病理诊断中收集生物医学信息不可或缺的工具，但人工注释、测量和解读既费力又昂贵。在此，我们提出了一个由在视觉分析和实时控制方面表现出色的大模型驱动的任务驱动框架，为下一代显微镜铺平了道路。我们在临床任务上取得了概念验证的成功，特别是在对苏木精-伊红染色的肝组织切片进行自适应分析方面。这项工作展示了未来数字病理学的先进能力，为临床应用中的智能、高效和实时分析树立了新的标准。
