INSERM, Paris, France.
J Pharmacokinet Pharmacodyn. 2010 Feb;37(1):49-65. doi: 10.1007/s10928-009-9143-7. Epub 2009 Dec 23.
To evaluate by simulation the statistical properties of normalized prediction distribution errors (NPDE), prediction discrepancies (pd), standardized prediction errors (SPE), numerical predictive check (NPC) and decorrelated NPC (NPC(dec)) for the external evaluation of a population pharmacokinetic analysis, and to illustrate the use of NPDE for the evaluation of covariate models. We assume that a model M(B) has been built using a building dataset B, and that a separate validation dataset, V is available. Our null hypothesis H(0) is that the data in V can be described by M(B). We use several methods to test this hypothesis: NPDE, pd, SPE, NPC and NPC(dec). First, we evaluated by simulation the type I error under H(0) of different tests applied to the four methods. We also propose and evaluate a single global test combining normality, mean and variance tests applied to NPDE, pd and SPE. We perform tests on NPC and NPC(dec), after a decorrelation. M(B) was a one compartment model with first order absorption (without covariate), previously developed from two phase II and one phase III studies of the antidiabetic drug, gliclazide. We simulated 500 external datasets according to the design of a phase III study. Second, we investigated the application of NPDE to covariate models. We propose two approaches: the first approach uses correlation tests or mean comparisons to test the relationship between NPDE and covariates; the second evaluates NPDE split by category for discrete covariates or quantiles for continuous covariates. We generated several validation datasets under H(0) and under alternative assumptions with a model without covariate, with one continuous covariate (weight), or one categorical covariate (sex). We calculated the powers of the different tests using simulations, where the covariates of the phase III study were used. The simulations under H(0) show a high type I error for the different tests applied to SPE and an increased type I error for pd. The different tests present a type I error close to 5% for the global test appied to NPDE. We find a type I error higher than 5% for the test applied to classical NPC but this test becomes close to 5% for NPC(dec). For covariate models, when model and validation dataset are consistent, type I error of the tests are close to 5% for both effects. When validation datasets and models are not consistent, the tests detect the correlation between NPDE and the covariate. We recommend to use NPDE over SPE for external model evaluation, since they do not depend on an approximation of the model and have good statistical properties. NPDE represent a better approach than NPC, since in order to perform tests on NPC, a decorrelation step must be applied before. NPDE, in this illustration, is also a good tool to evaluate model with or without covariates.
为了评估归一化预测分布误差（NPDE）、预测差异（pd）、标准化预测误差（SPE）、数值预测检验（NPC）和去相关 NPC（NPC(dec)）在群体药代动力学分析外部评估中的统计特性，并说明 NPDE 用于评估协变量模型的用途。我们假设已经使用构建数据集 B 构建了模型 M(B)，并且可以获得单独的验证数据集 V。我们的零假设 H(0) 是 V 中的数据可以由 M(B) 描述。我们使用多种方法来检验这个假设：NPDE、pd、SPE、NPC 和 NPC(dec)。首先，我们通过模拟评估了应用于这四种方法的不同测试在 H(0)下的Ⅰ型错误。我们还提出并评估了一种将 NPDE、pd 和 SPE 的正态性、均值和方差测试相结合的单一全局测试。我们对 NPC 和 NPC(dec)进行了去相关后的测试。M(B) 是一个具有一阶吸收（无协变量）的单室模型，先前是从抗糖尿病药物格列齐特的两项 II 期和一项 III 期研究中开发的。我们根据 III 期研究的设计模拟了 500 个外部数据集。其次，我们研究了 NPDE 在协变量模型中的应用。我们提出了两种方法：第一种方法使用相关测试或均值比较来检验 NPDE 与协变量之间的关系；第二种方法根据离散协变量的类别或连续协变量的分位数对 NPDE 进行划分。我们在 H(0)下和在没有协变量的替代假设下、在具有一个连续协变量（体重）或一个分类协变量（性别）的模型下生成了几个验证数据集。我们使用模拟计算了不同测试的功效，其中使用了 III 期研究的协变量。在 H(0)下的模拟显示，应用于 SPE 的不同测试的Ⅰ型错误率较高，而 pd 的Ⅰ型错误率增加。对于应用于 NPDE 的全局测试，不同的测试呈现出接近 5%的Ⅰ型错误率。我们发现，对于应用于经典 NPC 的测试，Ⅰ型错误率高于 5%，但对于 NPC(dec)，该测试接近 5%。对于协变量模型，当模型和验证数据集一致时，两种效应的测试Ⅰ型错误率接近 5%。当验证数据集和模型不一致时，测试会检测到 NPDE 与协变量之间的相关性。我们建议在外部模型评估中使用 NPDE 而不是 SPE，因为它们不依赖于模型的近似值，并且具有良好的统计特性。NPDE 比 NPC 是一种更好的方法，因为为了对 NPC 进行测试，必须在进行测试之前进行去相关步骤。在这种情况下，NPDE 也是评估有或没有协变量的模型的一个很好的工具。
