Complexity Science, Meraglim Holdings Corporation, Palm Beach Gardens, FL, United States of America.
PLoS One. 2024 Sep 25;19(9):e0304675. doi: 10.1371/journal.pone.0304675. eCollection 2024.
Simultaneous testing of one hypothesis at multiple alpha levels can be performed within a conventional Neyman-Pearson framework. This is achieved by treating the hypothesis as a family of hypotheses, each member of which explicitly concerns test level as well as effect size. Such testing encourages researchers to think about error rates and strength of evidence in both the statistical design and reporting stages of a study. Here, we show that these multi-alpha level tests can deliver acceptable expected total error costs. We first present formulas for expected error costs from single alpha and multiple alpha level tests, given prior probabilities of effect sizes that have either dichotomous or continuous distributions. Error costs are tied to decisions, with different decisions assumed for each of the potential outcomes in the multi-alpha level case. Expected total costs for tests at single and multiple alpha levels are then compared with optimal costs. This comparison highlights how sensitive optimization is to estimated error costs and to assumptions about prevalence. Testing at multiple default thresholds removes the need to formally identify decisions, or to model costs and prevalence as required in optimization approaches. Although total expected error costs with this approach will not be optimal, our results suggest they may be lower, on average, than when "optimal" test levels are based on mis-specified models.
在传统的 Neyman-Pearson 框架内，可以对多个α水平下的一个假设进行同时检验。这可以通过将假设视为一个假设族来实现，其中每个成员都明确涉及检验水平和效应大小。这种检验方法鼓励研究人员在研究的统计设计和报告阶段考虑错误率和证据强度。在这里，我们表明这些多α水平检验可以提供可接受的预期总误差成本。我们首先给出了基于效应大小的先验概率具有二项分布或连续分布时，单个α和多个α水平检验的误差成本公式。误差成本与决策相关联，在多α水平情况下的每个潜在结果都假设了不同的决策。然后将单α和多α水平检验的预期总费用与最优费用进行比较。这种比较突出了优化对估计的误差成本和对流行率的假设的敏感性。在多个默认阈值下进行检验，无需正式确定决策，也无需在优化方法中对成本和流行率进行建模。尽管这种方法的总预期误差成本不会是最优的，但我们的结果表明，与基于错误指定模型的“最优”检验水平相比，它们的平均值可能更低。
