Kurvers Ralf H J M, Krause Jens, Argenziano Giuseppe, Zalaudek Iris, Wolf Max
Department of Biology and Ecology of Fishes, Leibniz-Institute of Freshwater Ecology and Inland Fisheries, Berlin, Germany2Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany.
Department of Biology and Ecology of Fishes, Leibniz-Institute of Freshwater Ecology and Inland Fisheries, Berlin, Germany3Faculty of Life Sciences, Humboldt-University of Berlin, Berlin, Germany.
JAMA Dermatol. 2015 Dec 1;151(12):1346-1353. doi: 10.1001/jamadermatol.2015.3149.
Incidence rates of skin cancer are increasing globally, and the correct classification of skin lesions (SLs) into benign and malignant tissue remains a continuous challenge. A collective intelligence approach to skin cancer detection may improve accuracy.
To evaluate the performance of 2 well-known collective intelligence rules (majority rule and quorum rule) that combine the independent conclusions of multiple decision makers into a single decision.
DESIGN, SETTING, AND PARTICIPANTS: Evaluations were obtained from 2 large and independent data sets. The first data set consisted of 40 experienced dermoscopists, each of whom independently evaluated 108 images of SLs during the Consensus Net Meeting of 2000. The second data set consisted of 82 medical professionals with varying degrees of dermatology experience, each of whom evaluated a minimum of 110 SLs. All SLs were evaluated via the Internet. Image selection of SLs was based on high image quality and the presence of histopathologic information. Data were collected from July through October 2000 for study 1 and from February 2003 through January 2004 for study 2 and evaluated from January 5 through August 7, 2015.
For both collective intelligence rules, we determined the true-positive rate (ie, the hit rate or specificity) and the false-positive rate (ie, the false-alarm rate or 1 - sensitivity) and compared these rates with the performance of single decision makers. Furthermore, we evaluated the effect of group size on true- and false-positive rates.
One hundred twenty-two medical professionals performed 16 029 evaluations. Use of either collective intelligence rule consistently outperformed single decision makers. The groups achieved an increased true-positive rate and a decreased false-positive rate. For example, individual decision makers in study 1, using the pattern analysis as diagnostic algorithm, achieved a true-positive rate of 0.83 and a false-positive rate of 0.17. Groups of 3 individuals achieved a true-positive rate of 0.91 and a false-positive rate of 0.14. These improvements increased with increasing group size.
Collective intelligence might be a viable approach to increase diagnostic accuracy in skin cancer and reduce skin cancer-related mortality.
全球皮肤癌发病率正在上升，将皮肤病变（SLs）正确分类为良性和恶性组织仍然是一项持续的挑战。采用集体智慧方法进行皮肤癌检测可能会提高准确性。
评估两种著名的集体智慧规则（多数规则和法定人数规则）的性能，这两种规则将多个决策者的独立结论合并为一个决策。
设计、设置和参与者：评估来自两个大型独立数据集。第一个数据集由40名经验丰富的皮肤镜医师组成，他们在2000年的共识网络会议期间每人独立评估了108张SLs图像。第二个数据集由82名具有不同程度皮肤科经验的医学专业人员组成，他们每人至少评估了110个SLs。所有SLs均通过互联网进行评估。SLs的图像选择基于高图像质量和组织病理学信息的存在。研究1的数据收集时间为2000年7月至10月，研究2的数据收集时间为2003年2月至2004年1月，并于2015年1月5日至8月7日进行评估。
对于这两种集体智慧规则，我们确定了真阳性率（即命中率或特异性）和假阳性率（即误报率或1 - 敏感性），并将这些率与单个决策者的性能进行了比较。此外，我们评估了组大小对真阳性率和假阳性率的影响。
122名医学专业人员进行了16029次评估。使用任何一种集体智慧规则始终优于单个决策者。这些组实现了更高的真阳性率和更低的假阳性率。例如，在研究1中，使用模式分析作为诊断算法的个体决策者的真阳性率为0.83，假阳性率为0.17。3人小组的真阳性率为0.91，假阳性率为0.14。随着组大小的增加，这些改进也增加。
集体智慧可能是提高皮肤癌诊断准确性和降低皮肤癌相关死亡率的可行方法。
