Bell A J, Sejnowski T J
Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, CA 92037, USA.
Neural Comput. 1995 Nov;7(6):1129-59. doi: 10.1162/neco.1995.7.6.1129.
We derive a new self-organizing learning algorithm that maximizes the information transferred in a network of nonlinear units. The algorithm does not assume any knowledge of the input distributions, and is defined here for the zero-noise limit. Under these conditions, information maximization has extra properties not found in the linear case (Linsker 1989). The nonlinearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalization of principal components analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to 10 speakers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information maximization provides a unifying framework for problems in "blind" signal processing.
我们推导了一种新的自组织学习算法，该算法能使非线性单元网络中传输的信息最大化。该算法不假定任何关于输入分布的知识，且在此针对零噪声极限进行定义。在这些条件下，信息最大化具有线性情形（林斯克，1989年）中未发现的额外特性。传递函数中的非线性能够捕捉输入分布的高阶矩，并在输出表示中执行类似于真正冗余减少的操作。这使网络能够分离输入中的统计独立成分：主成分分析的高阶推广。我们将该网络应用于源分离（或鸡尾酒会）问题，成功分离了多达10个说话者的未知混合信号。我们还表明，网络架构的一种变体能够执行盲反卷积（消除语音信号中未知的回声和混响）。最后，我们推导了信息传递对时间延迟的依赖性。我们认为信息最大化提供了一个用于“盲”信号处理问题的统一框架。
