Luengo-Oroz Miguel Angel, Arranz Asier, Frean John
Biomedical Image Technologies group, DIE, ETSI Telecomunicación, Universidad Politécnica de Madrid, CEI Moncloa UPM-UCM, Madrid, Spain.
J Med Internet Res. 2012 Nov 29;14(6):e167. doi: 10.2196/jmir.2338.
There are 600,000 new malaria cases daily worldwide. The gold standard for estimating the parasite burden and the corresponding severity of the disease consists in manually counting the number of parasites in blood smears through a microscope, a process that can take more than 20 minutes of an expert microscopist's time.
This research tests the feasibility of a crowdsourced approach to malaria image analysis. In particular, we investigated whether anonymous volunteers with no prior experience would be able to count malaria parasites in digitized images of thick blood smears by playing a Web-based game.
The experimental system consisted of a Web-based game where online volunteers were tasked with detecting parasites in digitized blood sample images coupled with a decision algorithm that combined the analyses from several players to produce an improved collective detection outcome. Data were collected through the MalariaSpot website. Random images of thick blood films containing Plasmodium falciparum at medium to low parasitemias, acquired by conventional optical microscopy, were presented to players. In the game, players had to find and tag as many parasites as possible in 1 minute. In the event that players found all the parasites present in the image, they were presented with a new image. In order to combine the choices of different players into a single crowd decision, we implemented an image processing pipeline and a quorum algorithm that judged a parasite tagged when a group of players agreed on its position.
Over 1 month, anonymous players from 95 countries played more than 12,000 games and generated a database of more than 270,000 clicks on the test images. Results revealed that combining 22 games from nonexpert players achieved a parasite counting accuracy higher than 99%. This performance could be obtained also by combining 13 games from players trained for 1 minute. Exhaustive computations measured the parasite counting accuracy for all players as a function of the number of games considered and the experience of the players. In addition, we propose a mathematical equation that accurately models the collective parasite counting performance.
This research validates the online gaming approach for crowdsourced counting of malaria parasites in images of thick blood films. The findings support the conclusion that nonexperts are able to rapidly learn how to identify the typical features of malaria parasites in digitized thick blood samples and that combining the analyses of several users provides similar parasite counting accuracy rates as those of expert microscopists. This experiment illustrates the potential of the crowdsourced gaming approach for performing routine malaria parasite quantification, and more generally for solving biomedical image analysis problems, with future potential for telediagnosis related to global health challenges.
全球每天有60万例新发疟疾病例。估计疟原虫负荷及相应疾病严重程度的金标准是通过显微镜人工计数血涂片上的疟原虫数量，这一过程可能需要专家显微镜技师20多分钟的时间。
本研究测试众包方法用于疟疾图像分析的可行性。具体而言，我们调查了没有相关经验的匿名志愿者是否能够通过玩一款基于网络的游戏来对厚血涂片的数字化图像中的疟原虫进行计数。
实验系统由一款基于网络的游戏组成，在线志愿者的任务是在数字化血液样本图像中检测疟原虫，同时还有一个决策算法，该算法将多个参与者的分析结果结合起来，以产生改进的集体检测结果。数据通过MalariaSpot网站收集。将通过传统光学显微镜采集的含有中低疟原虫血症的恶性疟原虫的厚血膜随机图像呈现给参与者。在游戏中，参与者必须在1分钟内找到并标记尽可能多的疟原虫。如果参与者找到了图像中所有的疟原虫，就会向他们展示一张新图像。为了将不同参与者的选择整合为一个群体决策，我们实施了一个图像处理流程和一个法定人数算法，当一组参与者对疟原虫的位置达成一致时，该算法会判定该疟原虫被标记。
在1个月的时间里，来自95个国家的匿名参与者玩了超过12000场游戏，并在测试图像上产生了超过270000次点击的数据库。结果显示，将非专业参与者的22场游戏结果结合起来，疟原虫计数准确率高于99%。将接受过1分钟培训的参与者的13场游戏结果结合起来也能获得这样的表现。详尽的计算测量了所有参与者的疟原虫计数准确率与所考虑的游戏数量和参与者经验之间的函数关系。此外，我们提出了一个数学方程，该方程能准确模拟集体疟原虫计数表现。
本研究验证了用于众包厚血膜图像中疟原虫计数的在线游戏方法。研究结果支持以下结论：非专业人员能够快速学会如何在数字化厚血样本中识别疟原虫的典型特征，并且将多个用户的分析结果结合起来能提供与专家显微镜技师相似的疟原虫计数准确率。本实验说明了众包游戏方法在进行常规疟原虫定量分析方面的潜力，更广泛地说，在解决生物医学图像分析问题方面的潜力，未来在与全球健康挑战相关的远程诊断方面具有潜力。
