Feldman M J, Barnett G O
Laboratory of Computer Science, Massachusetts General Hospital, Boston 02114.
Comput Methods Programs Biomed. 1991 Aug;35(4):261-6. doi: 10.1016/0169-2607(91)90004-d.
DXplain is a computer-based decision support system which generates a differential diagnosis (ddx) from a given list of clinical manifestations (Barnett et al., J. Am. Med. Assoc. 258 (1987) 67-74). An approach was developed to evaluate the accuracy of the ddx's produced by DXplain. The first step involves the collection of 65 benchmark cases drawn from a variety of sources and authors. Despite their diverse origins, the cases share in common that they are all clinical cases upon which a consulting physician might be asked to produce a differential. This helps to ensure that the evaluation of the system will be done in an environment similar to that in which the system is actually used. In the second step, all cases are reviewed by five board-certified physicians (experts) as well as DXplain. For each case, the evaluators (experts and DXplain) produce a rank-ordered ddx list along with an indication of how strongly each disease was felt to be supported by the case findings. A scoring technique was devised which rewards concordance with the gold standard: a consensus of the evaluators' ddx lists. Each evaluator receives a score which is proportional to the degree of agreement achieved with the consensus on the ddx submitted. Preliminary results on a trial evaluation of 46 cases indicate that DXplain, on average, did well in agreeing with the consensus. Agreement was achieved both in regard to the specific diagnoses listed in the ddx and the degree to which the diseases were felt to be supported by the case findings. A discussion of some important issues in the evaluation of knowledge-based systems is undertaken.
DXplain是一个基于计算机的决策支持系统，它能根据给定的临床表现列表生成鉴别诊断（ddx）（巴尼特等人，《美国医学会杂志》258（1987）67 - 74）。已开发出一种方法来评估DXplain生成的鉴别诊断的准确性。第一步涉及从各种来源和作者处收集65个基准病例。尽管这些病例来源多样，但它们的共同之处在于都是临床病例，可能会要求会诊医生对其进行鉴别诊断。这有助于确保在与系统实际使用环境相似的情况下对系统进行评估。第二步，所有病例由五位经过委员会认证的医生（专家）以及DXplain进行审查。对于每个病例，评估人员（专家和DXplain）生成一个按等级排序的鉴别诊断列表，并指出病例发现对每种疾病的支持程度。设计了一种评分技术，奖励与金标准（评估人员鉴别诊断列表的共识）的一致性。每个评估人员都会得到一个分数，该分数与提交的鉴别诊断与共识的一致程度成正比。对46个病例的初步试验评估结果表明，DXplain总体上与共识的一致性较好。在鉴别诊断中列出的具体诊断以及病例发现对疾病的支持程度方面都达成了一致。文中还对基于知识的系统评估中的一些重要问题进行了讨论。
