A maximum entropy criterion of filtering and coding for stationary autoregressive signals: its physical interpretations and suggestions for its application to neural information transmission.

The operations of encoding and decoding in communication agree with filtering operations of convolution and deconvolution for Gaussian signal processing. In an analogy with power transmission in thermodynamics, an autoregressive model of information transmission is proposed for representing a continuous communication system which requires a pair of an internal noise source and a signal source to encode or decode a message. In this model transinformation (informational entropy) equals the increase in stationary nonequilibrium organization formed through the amplification of white noise by a positive feedback system. The channel capacity is finite due to the existence of inherent noise in the system. The maximum entropy criterion in information dynamics corresponds to the 2nd law of thermodynamics. If the process is stationary, the communication system is invertible, and has the maximum efficiency of transformation. The total variation in informational entropy is zero in the cycle of the invertible system, while in the noninvertible system the entropy of decoding is less than that of encoding. A noisy autoregressive coding which maximizes transinformation is optimum, but is also ideal.