A note on least-squares learning procedures and classification by neural network models.

Neural network models are considered as mathematical classifiers whose inputs comprise random variables generated according to arbitrary stationary class distributions, and the implication of learning based on minimization of sum-square classification error over a training set of these observations for which class assignments are absolutely determined is addressed. Expectations for network outputs in such cases are weighted least-squares approximations to a posteriori probabilities for the classes, which justifies interpretation of network outputs as indicating degree of confidence in class membership. The author demonstrates this with a straightforward proof in which class probability densities are regarded as primitives and which for simplicity does not rely on probability theory or statistics. The author cites more detailed results giving conditions for consistency of the estimators and discusses some issues relating to the suitability of neural network models and back-propagation training for approximation of conditional probabilities in classification tasks.