Towards Personalization of Algorithms Evaluation in Data Mining

G. Nakhaeizadeh and A. Schnabl

Like model selection in statistics, the choice of appropriate Data Mining Algorithms (DM-Algorithms) is a very important task in the process of Knowledge Discovery. Due to this fact it is necessary to have sophisticated metrics that can be used as comparators to evaluate alternative DM-algorithms. It has been shown in literature, that Data Envelopment Analysis (DEA) is an appropriate platform to develop multi-criteria evaluation metrics that can consider - in contrary to mono-criteria metrics - all positive and negative properties of DM-algorithms. We discuss different extensions of DEA that enable consideration of qualitative properties of DM-algorithms and consideration of users preferences in development of evaluation metrics. The results open new discussions in the general debate on model selection in statistics and machine learning.

This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.