Evaluation Methods for Machine Learning
Papers from the 2006 AAAI Workshop
Chris Drummond, William Elazmeh, and Nathalie Japkowicz, Program Cochairs
Technical Report WS-06-06 published by The AAAI Press, Menlo Park, California
This technical report is also available in book and CD format.
Please Note: Abstracts are linked to individual titles, and will appear in a separate browser window. Full-text versions of the papers are linked to the abstract text. Access to full text may be restricted to AAAI members. PDF file sizes may be large!
Contents
Preface / vii
Chris Drummond, William Elazmeh, and Nathalie Japkowicz
Organizing Committee / vii
Chris Drummond, William Elazmeh, and Nathalie Japkowicz
Machine Learning as an Experimental Science (Revisited) / 1
Chris Drummond
Why Question Machine Learning Evaluation Methods? An Illustrative Review of the Shortcomings of Current Methods / 6
Nathalie Japkowicz
Evaluating Model Selection Abilities of Performance Measures / 12
Jin Huang and Charles X. Ling
Evaluating Probability Estimates from Decision Trees / 18
Nitesh V. Chawla and David A. Cieslak
Beyond Accuracy, F-score, and ROC: A Family of Discriminant Measures for Performance Evaluation / 24
Marina Sokolova, Nathalie Japkowicz, and Stan Szpakowicz
Evaluation Classifiers: Practical Considerations for Security Applications / 30
Alvaro A. Cárdenas and John S. Baras
Confidence Interval for the Difference in Classification Error / 36
William Elazmeh, Nathalie Japkowicz, and Stan Matwin
Evaluating the Explanatory Value of Bayesian Network Structure Learning Algorithms / 42
Patrick Shaughnessy and Gary Livingston