Rehabilitation Engineering Laboratory, Institute of Robotics and Intelligent Systems, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland.
PLoS One. 2018 Nov 28;13(11):e0207217. doi: 10.1371/journal.pone.0207217. eCollection 2018.
When estimating psychometric functions with sampling procedures, psychophysical assessments should be precise and accurate while being as efficient as possible to reduce assessment duration. The estimation performance of sampling procedures is commonly evaluated in computer simulations for single psychometric functions and reported using metrics as a function of number of trials. However, the estimation performance of a sampling procedure may vary for different psychometric functions. Therefore, the results of these type of evaluations may not be generalizable to a heterogeneous population of interest. In addition, the maximum number of trials is often imposed by time restrictions, especially in clinical applications, making trial-based metrics suboptimal. Hence, the benefit of these simulations to select and tune an ideal sampling procedure for a specific application is limited. We suggest to evaluate the estimation performance of sampling procedures in simulations covering the entire range of psychometric functions found in a population of interest, and propose a comprehensive set of performance metrics for a detailed analysis. To illustrate the information gained from these metrics in an application example, six sampling procedures were evaluated in a computer simulation based on prior knowledge on the population distribution and requirements from proprioceptive assessments. The metrics revealed limitations of the sampling procedures, such as inhomogeneous or systematically decreasing performance depending on the psychometric functions, which can inform the tuning process of a sampling procedure. More advanced metrics allowed directly comparing overall performances of different sampling procedures and select the best-suited sampling procedure for the example application. The proposed analysis metrics can be used for any sampling procedure and the estimation of any parameter of a psychometric function, independent of the shape of the psychometric function and of how such a parameter was estimated. This framework should help to accelerate the development process of psychophysical assessments.
当使用抽样程序估计心理测量函数时，心理物理评估应该既精确又准确，同时尽可能高效，以缩短评估时间。抽样程序的估计性能通常在计算机模拟中针对单个心理测量函数进行评估，并使用指标报告作为试验次数的函数。然而，抽样程序的估计性能可能因不同的心理测量函数而异。因此，这些类型的评估结果可能无法推广到感兴趣的异质人群。此外，由于时间限制，通常会对最大试验次数进行限制，尤其是在临床应用中，因此基于试验的指标并不理想。因此，这些模拟对于选择和调整特定应用的理想抽样程序的好处是有限的。我们建议在模拟中评估抽样程序的估计性能，这些模拟涵盖了感兴趣人群中的整个心理测量函数范围，并提出了一套全面的性能指标进行详细分析。为了在应用示例中说明这些指标所获得的信息，根据对人群分布的先验知识和来自本体感觉评估的要求，在计算机模拟中评估了六种抽样程序。这些指标揭示了抽样程序的局限性，例如根据心理测量函数不均匀或系统地降低性能，这可以为抽样程序的调整过程提供信息。更先进的指标允许直接比较不同抽样程序的整体性能，并为示例应用选择最合适的抽样程序。所提出的分析指标可用于任何抽样程序和心理测量函数的任何参数的估计，独立于心理测量函数的形状以及如何估计该参数。该框架应有助于加速心理物理评估的开发过程。
