AAAI Publications, Twenty-Ninth AAAI Conference on Artificial Intelligence

Font Size: 
Finding Meaningful Gaps to Guide Data Acquisition for a Radiation Adjudication System
Nick Gisolfi, Madalina Fiterau, Artur Dubrawski

Last modified: 2015-03-04

Abstract


We consider the problem of identifying discrepancies between training and test data which are responsible for the reduced performance of a classification system. Intended for use when data acquisition is an iterative process controlled by domain experts, our method exposes insufficiencies of training data and presents them in a user-friendly manner. The system is capable of working with any classification system which admits diagnostics on test data. We illustrate the usefulness of our approach in recovering compact representations of the revealed gaps in training data and show that predictive accuracy of the resulting models is improved once the gaps are filled through collection of additional training samples.

Keywords


Dataset Shift; Decision Support Systems; Ensemble Methods; Feature Selection

Full Text: PDF