Rajan Kohulan, Brinkhaus Henning Otto, Zielesny Achim, Steinbeck Christoph
Institute for Inorganic and Analytical Chemistry, Friedrich Schiller University Jena, Lessingstr. 8, 07743, Jena, Germany.
Institute for Bioinformatics and Chemoinformatics, Westphalian University of Applied Sciences, August-Schmidt-Ring 10, 45665, Recklinghausen, Germany.
J Cheminform. 2024 Jul 5;16(1):78. doi: 10.1186/s13321-024-00872-7.
Accurate recognition of hand-drawn chemical structures is crucial for digitising hand-written chemical information in traditional laboratory notebooks or facilitating stylus-based structure entry on tablets or smartphones. However, the inherent variability in hand-drawn structures poses challenges for existing Optical Chemical Structure Recognition (OCSR) software. To address this, we present an enhanced Deep lEarning for Chemical ImagE Recognition (DECIMER) architecture that leverages a combination of Convolutional Neural Networks (CNNs) and Transformers to improve the recognition of hand-drawn chemical structures. The model incorporates an EfficientNetV2 CNN encoder that extracts features from hand-drawn images, followed by a Transformer decoder that converts the extracted features into Simplified Molecular Input Line Entry System (SMILES) strings. Our models were trained using synthetic hand-drawn images generated by RanDepict, a tool for depicting chemical structures with different style elements. A benchmark was performed using a real-world dataset of hand-drawn chemical structures to evaluate the model's performance. The results indicate that our improved DECIMER architecture exhibits a significantly enhanced recognition accuracy compared to other approaches. SCIENTIFIC CONTRIBUTION: The new DECIMER model presented here refines our previous research efforts and is currently the only open-source model tailored specifically for the recognition of hand-drawn chemical structures. The enhanced model performs better in handling variations in handwriting styles, line thicknesses, and background noise, making it suitable for real-world applications. The DECIMER hand-drawn structure recognition model and its source code have been made available as an open-source package under a permissive license.
准确识别手绘化学结构对于将传统实验室笔记本中的手写化学信息数字化，或便于在平板电脑或智能手机上基于手写笔输入结构至关重要。然而，手绘结构中固有的变异性给现有的光学化学结构识别（OCSR）软件带来了挑战。为了解决这个问题，我们提出了一种增强的化学图像识别深度学习（DECIMER）架构，该架构利用卷积神经网络（CNN）和Transformer的组合来提高对手绘化学结构的识别。该模型包含一个EfficientNetV2 CNN编码器，用于从手绘图像中提取特征，随后是一个Transformer解码器，将提取的特征转换为简化分子输入线性输入系统（SMILES）字符串。我们的模型使用RanDepict生成的合成手绘图像进行训练，RanDepict是一种用于描绘具有不同样式元素的化学结构的工具。使用手绘化学结构的真实世界数据集进行基准测试，以评估模型的性能。结果表明，与其他方法相比，我们改进的DECIMER架构具有显著提高的识别准确率。科学贡献：这里提出的新DECIMER模型改进了我们之前的研究工作，并且是目前唯一专门为识别手绘化学结构量身定制的开源模型。增强后的模型在处理笔迹风格、线条粗细和背景噪声的变化方面表现更好，使其适用于实际应用。DECIMER手绘结构识别模型及其源代码已作为开源包在宽松许可下提供。
