Average Top-k Aggregate Loss for Supervised Learning.

In this work, we introduce the average top- k ( AT) loss, which is the average over the k largest individual losses over a training data, as a new aggregate loss for supervised learning. We show that the AT loss is a natural generalization of the two widely used aggregate losses, namely the average loss and the maximum loss. Yet, the AT loss can better adapt to different data distributions because of the extra flexibility provided by the different choices of k. Furthermore, it remains a convex function over all individual losses and can be combined with different types of individual loss without significant increase in computation. We then provide interpretations of the AT loss from the perspective of the modification of individual loss and robustness to training data distributions. We further study the classification calibration of the AT loss and the error bounds of AT-SVM model. We demonstrate the applicability of minimum average top- k learning for supervised learning problems including binary/multi-class classification and regression, using experiments on both synthetic and real datasets.