Zhu Yuanxin, Carragher Bridget, Glaeser Robert M, Fellmann Denis, Bajaj Chandrajit, Bern Marshall, Mouche Fabrice, de Haas Felix, Hall Richard J, Kriegman David J, Ludtke Steven J, Mallick Satya P, Penczek Pawel A, Roseman Alan M, Sigworth Fred J, Volkmann Niels, Potter Clinton S
Center for Integrative Molecular Biosciences and Department of Cell Biology, The Scripps Research Institute, La Jolla, CA 92037, USA.
J Struct Biol. 2004 Jan-Feb;145(1-2):3-14. doi: 10.1016/j.jsb.2003.09.033.
Manual selection of single particles in images acquired using cryo-electron microscopy (cryoEM) will become a significant bottleneck when datasets of a hundred thousand or even a million particles are required for structure determination at near atomic resolution. Algorithm development of fully automated particle selection is thus an important research objective in the cryoEM field. A number of research groups are making promising new advances in this area. Evaluation of algorithms using a standard set of cryoEM images is an essential aspect of this algorithm development. With this goal in mind, a particle selection "bakeoff" was included in the program of the Multidisciplinary Workshop on Automatic Particle Selection for cryoEM. Twelve groups participated by submitting the results of testing their own algorithms on a common dataset. The dataset consisted of 82 defocus pairs of high-magnification micrographs, containing keyhole limpet hemocyanin particles, acquired using cryoEM. The results of the bakeoff are presented in this paper along with a summary of the discussion from the workshop. It was agreed that establishing benchmark particles and using bakeoffs to evaluate algorithms are useful in promoting algorithm development for fully automated particle selection, and that the infrastructure set up to support the bakeoff should be maintained and extended to include larger and more varied datasets, and more criteria for future evaluations.
当需要十万甚至上百万个颗粒的数据集来进行近原子分辨率的结构测定时，在使用冷冻电子显微镜（cryoEM）采集的图像中手动选择单个颗粒将成为一个重大瓶颈。因此，全自动颗粒选择的算法开发是冷冻电子显微镜领域的一个重要研究目标。许多研究小组在这一领域都取得了有希望的新进展。使用一组标准的冷冻电子显微镜图像对算法进行评估是该算法开发的一个重要方面。出于这个目的，冷冻电子显微镜自动颗粒选择多学科研讨会的议程中包括了一次颗粒选择“烘焙赛”。十二个小组参与其中，提交了他们在一个共同数据集上测试自己算法的结果。该数据集由82对散焦的高倍显微照片组成，包含使用冷冻电子显微镜采集的钥孔戚血蓝蛋白颗粒。本文展示了烘焙赛的结果以及研讨会上的讨论总结。与会者一致认为，建立基准颗粒并使用烘焙赛来评估算法有助于推动全自动颗粒选择算法的开发，并且为支持烘焙赛而建立的基础设施应予以维护和扩展，以纳入更大、更多样化的数据集以及未来评估的更多标准。
