Evaluation Methods for Machine Learning
Papers from the 2006 AAAI Workshop
Chris Drummond, William Elazmeh, and Nathalie Japkowicz, Program Cochairs
Technical Report WS-06-06 published by The AAAI Press, Menlo Park, California
This technical report is also available in book and CD format.
Please Note: Abstracts are linked to individual titles, and will appear in a separate browser window. Full-text versions of the papers are linked to the abstract text. Access to full text may be restricted to AAAI members. PDF file sizes may be large!
Contents
Preface / vii
Chris Drummond, William Elazmeh, and Nathalie Japkowicz
Organizing Committee / vii
Chris Drummond, William Elazmeh, and Nathalie Japkowicz
Machine Learning as an Experimental Science (Revisited) / 1
Chris Drummond
Why Question Machine Learning Evaluation Methods? An Illustrative Review of the Shortcomings of Current Methods / 6
Nathalie Japkowicz
Evaluating Model Selection Abilities of Performance Measures / 12
Jin Huang and Charles X. Ling
Evaluating Probability Estimates from Decision Trees / 18
Nitesh V. Chawla and David A. Cieslak
Beyond Accuracy, F-score, and ROC: A Family of Discriminant Measures for Performance Evaluation / 24
Marina Sokolova, Nathalie Japkowicz, and Stan Szpakowicz
Evaluation Classifiers: Practical Considerations for Security Applications / 30
Alvaro A. Cárdenas and John S. Baras
Confidence Interval for the Difference in Classification Error / 36
William Elazmeh, Nathalie Japkowicz, and Stan Matwin
Evaluating the Explanatory Value of Bayesian Network Structure Learning Algorithms / 42
Patrick Shaughnessy and Gary Livingston
AAAI Digital Library
AAAI relies on your generous support through membership and donations. If you find these resources useful, we would be grateful for your support.