Lewis J R
International Business Machines, Inc., Boca Raton, Florida 33429-1328.
Hum Factors. 1994 Jun;36(2):368-78. doi: 10.1177/001872089403600215.
Recently, Virzi (1992) presented data that support three claims regarding sample sizes for usability studies: (1) observing four or five participants will allow a usability practitioner to discover 80% of a product's usability problems, (2) observing additional participants will reveal fewer and fewer new usability problems, and (3) more severe usability problems are easier to detect with the first few participants. Results from an independent usability study clearly support the second claim, partially support the first, but fail to support the third. Problem discovery shows diminishing returns as a function of sample size. Observing four to five participants will uncover about 80% of a product's usability problems as long as the average likelihood of problem detection ranges between 0.32 and 0.42, as in Virzi. If the average likelihood of problem detection is lower, then a practitioner will need to observe more than five participants to discover 80% of the problems. Using behavioral categories for problem severity (or impact), these data showed no correlation between problem severity (impact) and rate of discovery. The data provided evidence that the binomial probability formula may provide a good model for predicting problem discovery curves, given an estimate of the average likelihood of problem detection. Finally, data from economic simulations that estimated return on investment (ROI) under a variety of settings showed that only the average likelihood of problem detection strongly influenced the range of sample sizes for maximum ROI.
最近，维尔齐（1992年）给出的数据支持了关于可用性研究样本量的三个观点：（1）观察四到五名参与者将使可用性从业者发现产品80%的可用性问题；（2）观察更多参与者会发现越来越少的新可用性问题；（3）最初的几名参与者更容易发现更严重的可用性问题。一项独立可用性研究的结果明确支持第二个观点，部分支持第一个观点，但不支持第三个观点。问题发现作为样本量的函数呈现出收益递减的情况。只要问题发现的平均可能性在0.32至0.42之间（如维尔齐研究中的情况），观察四到五名参与者将发现产品约80%的可用性问题。如果问题发现的平均可能性较低，那么从业者将需要观察超过五名参与者才能发现80%的问题。使用问题严重程度（或影响）的行为类别，这些数据表明问题严重程度（影响）与发现率之间没有相关性。这些数据提供了证据，表明在给出问题发现平均可能性估计值的情况下，二项式概率公式可能为预测问题发现曲线提供一个良好的模型。最后，来自经济模拟的数据（这些模拟估计了各种情况下的投资回报率（ROI））表明，只有问题发现的平均可能性对实现最大ROI的样本量范围有强烈影响。
