Calcagnì Antonio, Finos Livio, Altoé Gianmarco, Pastore Massimiliano
Department of Developmental and Social Psychology, University of Padova, 35131 Padova, Italy.
Entropy (Basel). 2019 Jun 15;21(6):596. doi: 10.3390/e21060596.
In this article, we provide initial findings regarding the problem of solving likelihood equations by means of a maximum entropy (ME) approach. Unlike standard procedures that require equating the score function of the maximum likelihood problem at zero, we propose an alternative strategy where the score is instead used as an external informative constraint to the maximization of the convex Shannon's entropy function. The problem involves the reparameterization of the score parameters as expected values of discrete probability distributions where probabilities need to be estimated. This leads to a simpler situation where parameters are searched in smaller (hyper) simplex space. We assessed our proposal by means of empirical case studies and a simulation study, the latter involving the most critical case of logistic regression under data separation. The results suggested that the maximum entropy reformulation of the score problem solves the likelihood equation problem. Similarly, when maximum likelihood estimation is difficult, as is the case of logistic regression under separation, the maximum entropy proposal achieved results (numerically) comparable to those obtained by the Firth's bias-corrected approach. Overall, these first findings reveal that a maximum entropy solution can be considered as an alternative technique to solve the likelihood equation.
在本文中，我们给出了关于通过最大熵（ME）方法求解似然方程问题的初步研究结果。与要求将最大似然问题的得分函数设为零的标准程序不同，我们提出了一种替代策略，即将得分用作对凸香农熵函数最大化的外部信息约束。该问题涉及将得分参数重新参数化为离散概率分布的期望值，其中概率需要进行估计。这导致了一种更简单的情况，即在较小的（超）单纯形空间中搜索参数。我们通过实证案例研究和模拟研究对我们的提议进行了评估，后者涉及数据分离情况下逻辑回归最关键的情形。结果表明，得分问题的最大熵重新表述解决了似然方程问题。同样，当最大似然估计困难时，如分离情况下的逻辑回归，最大熵提议（在数值上）取得了与费思偏差校正方法相当的结果。总体而言，这些初步研究结果表明，最大熵解可被视为求解似然方程的一种替代技术。
