Thomas Lord Department of Mechanical Engineering and Materials Science, Duke University, Durham, NC 27708, USA.
Office of Biomedical Graduate Education, Duke University School of Medicine, Durham, NC 27710, USA.
Lab Chip. 2022 Aug 9;22(16):2978-2985. doi: 10.1039/d2lc00206j.
Machine learning image recognition and classification of particles and materials is a rapidly expanding field. However, nanomaterial identification and classification are dependent on the image resolution, the image field of view, and the processing time. Optical microscopes are one of the most widely utilized technologies in laboratories across the world, due to their nondestructive abilities to identify and classify critical micro-sized objects and processes, but identifying and classifying critical nano-sized objects and processes with a conventional microscope are outside of its capabilities, due to the diffraction limit of the optics and small field of view. To overcome these challenges of nanomaterial identification and classification, we developed an intelligent nanoscope that combines machine learning and microsphere array-based imaging to: (1) surpass the diffraction limit of the microscope objective with microsphere imaging to provide high-resolution images; (2) provide large field-of-view imaging without the sacrifice of resolution by utilizing a microsphere array; and (3) rapidly classify nanomaterials using a deep convolution neural network. The intelligent nanoscope delivers more than 46 magnified images from a single image frame so that we collected more than 1000 images within 2 seconds. Moreover, the intelligent nanoscope achieves a 95% nanomaterial classification accuracy using 1000 images of training sets, which is 45% more accurate than without the microsphere array. The intelligent nanoscope also achieves a 92% bacteria classification accuracy using 50 000 images of training sets, which is 35% more accurate than without the microsphere array. This platform accomplished rapid, accurate detection and classification of nanomaterials with miniscule size differences. The capabilities of this device wield the potential to further detect and classify smaller biological nanomaterial, such as viruses or extracellular vesicles.
机器学习图像识别和颗粒及材料分类是一个快速发展的领域。然而，纳米材料的识别和分类依赖于图像分辨率、图像视场和处理时间。光学显微镜是世界范围内实验室最广泛使用的技术之一，因为它具有非破坏性的能力，可以识别和分类关键的微尺度物体和过程，但由于光学的衍射极限和小视场，用传统显微镜识别和分类关键的纳米尺度物体和过程是其能力之外的。为了克服纳米材料识别和分类的这些挑战，我们开发了一种智能纳米显微镜，它将机器学习和微球阵列成像相结合，以：（1）通过微球成像超越显微镜物镜的衍射极限，提供高分辨率图像；（2）利用微球阵列提供大视场成像，而不会牺牲分辨率；（3）利用深度卷积神经网络快速分类纳米材料。智能纳米显微镜从单个图像帧提供超过 46 个放大图像，因此我们在 2 秒内收集了超过 1000 个图像。此外，智能纳米显微镜使用 1000 个训练集图像实现了 95%的纳米材料分类准确性，比没有微球阵列的情况下提高了 45%。智能纳米显微镜使用 50000 个训练集图像实现了 92%的细菌分类准确性，比没有微球阵列的情况下提高了 35%。该平台实现了对具有微小尺寸差异的纳米材料的快速、准确检测和分类。该设备的功能有可能进一步检测和分类更小的生物纳米材料，如病毒或细胞外囊泡。
