Active Learning of Multi-Class Classification Models from Ordered Class Sets

Authors

  • Yanbing Xue University of Pittsburgh
  • Milos Hauskrecht University of Pittsburgh

DOI:

https://doi.org/10.1609/aaai.v33i01.33015589

Abstract

In this paper, we study the problem of learning multi-class classification models from a limited set of labeled examples obtained from human annotator. We propose a new machine learning framework that learns multi-class classification models from ordered class sets the annotator may use to express not only her top class choice but also other competing classes still under consideration. Such ordered sets of competing classes are common, for example, in various diagnostic tasks. In this paper, we first develop strategies for learning multi-class classification models from examples associated with ordered class set information. After that we develop an active learning strategy that considers such a feedback. We evaluate the benefit of the framework on multiple datasets. We show that class-order feedback and active learning can reduce the annotation cost both individually and jointly.

Downloads

Published

2019-07-17

How to Cite

Xue, Y., & Hauskrecht, M. (2019). Active Learning of Multi-Class Classification Models from Ordered Class Sets. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 5589-5596. https://doi.org/10.1609/aaai.v33i01.33015589

Issue

Section

AAAI Technical Track: Machine Learning