Hierarchical Active Learning with Overlapping Regions.

Learning of classification models from real-world data often requires substantial human effort devoted to annotation. As this process can be very time-consuming and costly, finding effective ways to reduce the annotation cost becomes critical for building such models. To address this problem we explore a new type of human feedback - -based feedback. Briefly, a region is defined as a hypercubic subspace of the input data space and represents a of data instances; the region's label is a human assessment of the class of the data subpopulation. By using algorithms one can learn instance-based classifiers from such labeled regions. In general, the key challenge is that there can be infinite many regions one can define and query in a given data space. To minimize the number and complexity of region-based queries, we propose and develop a solution that aims at incrementally building a hierarchy of regions. Furthermore, to avoid building a possibly class-irrelevant region hierarchy, we further propose to grow multiple different hierarchies in parallel and expand those more informative hierarchies. Through experiments on numerous data sets, we demonstrate that methods using region-based feedback can learn very good classifiers from very few and simple queries, and hence are highly effective in reducing human annotation effort needed for building classification models.