Institute of Chemistry, Loránd Eötvös University, Budapest, Hungary.
SAR QSAR Environ Res. 2021 Apr;32(4):247-268. doi: 10.1080/1062936X.2021.1890208. Epub 2021 Mar 22.
The dependence of statistical validation parameters was investigated on the size of the sample taken in fit of multivariate linear curves. We observed that and related internal parameters were misleading as they overestimated the goodness-of-fit of models at small sample size. Cross-validation metrics showed correct trends. It was possible to scale the leave-one-out and the leave-many-out results close to identical by correcting the degrees of freedom of the models. and -randomized validation parameters were calculated and the methods provided close to identical results. We suggest to use the simplest methods in both cases. The external parameters followed correct trends with respect to the sample size, but their sensitivity differed. We plotted the Roy-Ojha metrics in 2D and we coloured them with respect to other external parameters to provide an easy classification of models. The rank correlations were calculated between the performance parameters. Up to a sample size, goodness-of-fit and robustness were distinguishable, but above a certain sample size, the parameters were redundant. The external-internal pairs were weakly correlated. Our data show that all the three aspects of validation are necessary at small sample sizes, but the internal check of robustness is not informative above a given sample size.
研究了在拟合多元线性曲线时，所取样本量对统计验证参数的依赖性。我们观察到， 和相关的内部参数具有误导性，因为它们在样本量较小时高估了模型的拟合优度。交叉验证指标显示出正确的趋势。通过修正模型的自由度，可以将留一法和留多法的结果接近一致地缩放。计算了 和 -随机验证参数，所提供的方法结果几乎相同。我们建议在这两种情况下都使用最简单的方法。外部参数与样本量之间存在正确的趋势，但它们的灵敏度不同。我们在二维空间中绘制了 Roy-Ojha 指标，并根据其他外部参数对其进行了着色，以便对模型进行简单的分类。计算了性能参数之间的等级相关系数。在一定的样本量范围内，拟合优度和稳健性是可区分的，但在某个样本量以上，参数是冗余的。外部-内部对之间的相关性较弱。我们的数据表明，在小样本量下，验证的所有三个方面都是必要的，但在给定的样本量以上，稳健性的内部检查就没有信息了。
