Zhao Ruzhang, Hong Pengyu, Liu Jun S
Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD 21205, USA.
Department of Computer Science, Brandeis University, Waltham, MA 02453, USA.
Entropy (Basel). 2020 Mar 2;22(3):291. doi: 10.3390/e22030291.
Traditional hypothesis-margin researches focus on obtaining large margins and feature selection. In this work, we show that the robustness of margins is also critical and can be measured using entropy. In addition, our approach provides clear mathematical formulations and explanations to uncover feature interactions, which is often lack in large hypothesis-margin based approaches. We design an algorithm, termed IMMIGRATE (Iterative max-min entropy margin-maximization with interaction terms), for training the weights associated with the interaction terms. IMMIGRATE simultaneously utilizes both local and global information and can be used as a base learner in Boosting. We evaluate IMMIGRATE in a wide range of tasks, in which it demonstrates exceptional robustness and achieves the state-of-the-art results with high interpretability.
传统的假设边际研究专注于获得大的边际和特征选择。在这项工作中，我们表明边际的稳健性也很关键，并且可以使用熵来衡量。此外，我们的方法提供了清晰的数学公式和解释来揭示特征交互作用，而这在基于大假设边际的方法中常常缺乏。我们设计了一种算法，称为IMMIGRATE（带有交互项的迭代最大最小熵边际最大化），用于训练与交互项相关的权重。IMMIGRATE同时利用局部和全局信息，并且可以用作Boosting中的基础学习器。我们在广泛的任务中评估了IMMIGRATE，它在这些任务中表现出卓越的稳健性，并以高可解释性取得了最优结果。
