David J. Musliner and David Kortenkamp
Artificial intelligence and robotics are fields notorious for developing many different systems that are never compared with each other in a quantitative way. This paper suggests that the task of vacuum-cleaning could be used as a benchmark for comparing different robot systems. Such a comparison should take place within a simulation environment, which is outlined in the paper. This paper also suggests a possible standard robot and some benchmark environments, as well as a set of quantitative evaluation metrics that could be used to compare robot systems.