Bauerle Alex, van Onzenoodt Christian, Ropinski Timo
IEEE Trans Vis Comput Graph. 2021 Jun;27(6):2980-2991. doi: 10.1109/TVCG.2021.3057483. Epub 2021 May 12.
To convey neural network architectures in publications, appropriate visualizations are of great importance. While most current deep learning papers contain such visualizations, these are usually handcrafted just before publication, which results in a lack of a common visual grammar, significant time investment, errors, and ambiguities. Current automatic network visualization tools focus on debugging the network itself and are not ideal for generating publication visualizations. Therefore, we present an approach to automate this process by translating network architectures specified in Keras into visualizations that can directly be embedded into any publication. To do so, we propose a visual grammar for convolutional neural networks (CNNs), which has been derived from an analysis of such figures extracted from all ICCV and CVPR papers published between 2013 and 2019. The proposed grammar incorporates visual encoding, network layout, layer aggregation, and legend generation. We have further realized our approach in an online system available to the community, which we have evaluated through expert feedback, and a quantitative study. It not only reduces the time needed to generate network visualizations for publications, but also enables a unified and unambiguous visualization design.
为了在出版物中展示神经网络架构，合适的可视化至关重要。虽然当前大多数深度学习论文都包含此类可视化，但这些通常是在即将发表之前手工制作的，这导致缺乏通用的视觉语法、需要投入大量时间、容易出错且存在模糊性。当前的自动网络可视化工具专注于调试网络本身，并不适合生成用于出版物的可视化。因此，我们提出了一种方法，通过将Keras中指定的网络架构转换为可直接嵌入任何出版物的可视化，来实现这一过程的自动化。为此，我们提出了一种用于卷积神经网络（CNN）的视觉语法，该语法源自对2013年至2019年发表的所有ICCV和CVPR论文中此类图形的分析。所提出的语法包含视觉编码、网络布局、层聚合和图例生成。我们还在一个可供社区使用的在线系统中实现了我们的方法，并通过专家反馈和定量研究对其进行了评估。它不仅减少了为出版物生成网络可视化所需的时间，还实现了统一且明确的可视化设计。
