Della Mea Vincenzo, Maddalena Eddy, Mizzaro Stefano, Machin Piernicola, Beltrami Carlo A
Diagn Pathol. 2014;9 Suppl 1(Suppl 1):S6. doi: 10.1186/1746-1596-9-S1-S6. Epub 2014 Dec 19.
Crowdsourcing, i.e., the outsourcing of tasks typically performed by a few experts to a large crowd as an open call, has been shown to be reasonably effective in many cases, like Wikipedia, the Chess match of Kasparov against the world in 1999, and several others. The aim of the present paper is to describe the setup of an experimentation of crowdsourcing techniques applied to the quantification of immunohistochemistry.
Fourteen Images from MIB1-stained breast specimens were first manually counted by a pathologist, then submitted to a crowdsourcing platform through a specifically developed application. 10 positivity evaluations for each image have been collected and summarized using their median. The positivity values have been then compared to the gold standard provided by the pathologist by means of Spearman correlation.
Contributors were in total 28, and evaluated 4.64 images each on average. Spearman correlation between gold and crowdsourced positivity percentages is 0.946 (p < 0.001).
Aim of the experiment was to understand how to use crowdsourcing for an image analysis task that is currently time-consuming when done by human experts. Crowdsourced work can be used in various ways, in particular statistically agregating data to reduce identification errors. However, in this preliminary experimentation we just considered the most basic indicator, that is the median positivity percentage, which provided overall good results. This method might be more aimed to research than routine: when a large number of images are in need of ad-hoc evaluation, crowdsourcing may represent a quick answer to the need.
众包，即将通常由少数专家执行的任务作为公开邀请外包给大量人群，在许多情况下已被证明相当有效，如维基百科、1999年卡斯帕罗夫与全球棋手的国际象棋比赛等。本文旨在描述将众包技术应用于免疫组织化学定量的实验设置。
首先由一名病理学家对14张MIB1染色的乳腺标本图像进行手动计数，然后通过专门开发的应用程序提交到众包平台。对每张图像收集了10次阳性评估，并使用中位数进行汇总。然后通过斯皮尔曼相关性将阳性值与病理学家提供的金标准进行比较。
贡献者共有28人，平均每人评估4.64张图像。金标准与众包阳性百分比之间的斯皮尔曼相关性为0.946（p < 0.001）。
该实验的目的是了解如何将众包用于目前由人类专家完成时耗时的图像分析任务。众包工作可以通过多种方式使用，特别是通过统计聚合数据来减少识别错误。然而，在这个初步实验中，我们只考虑了最基本的指标，即中位数阳性百分比，其总体结果良好。这种方法可能更适用于研究而非常规工作：当需要对大量图像进行特殊评估时，众包可能是满足需求的快速解决方案。
