On Reproducible AI: Towards Reproducible Research, Open Science, and Digital Scholarship in AI Publications

  • Odd Erik Gundersen AAAI
  • Yolanda Gil Information Sciences Institute
  • David W. Aha US Naval Research Laboratory

Abstract

Background: Science is experiencing a reproducibility crisis. Artificial intelligence research is not an exception. Objective: To give practical and pragmatic recommendations for how to document AI research so that the results are reproducible. Method: Our analysis of the literature shows that AI publications fall short of providing enough documentation to facilitate reproducibility. Our suggested best practices are based on a framework for reproducibility and recommendations given for other disciplines. Results: We have made an author checklist based on our investigation and provided examples for how every item in the checklist can be documented. Conclusion: We encourage reviewers to use the suggested best practices and author checklist when reviewing submissions for AAAI publications and future AAAI conferences.
Published
2018-09-28
How to Cite
Gundersen, O. E., Gil, Y., & Aha, D. W. (2018). On Reproducible AI: Towards Reproducible Research, Open Science, and Digital Scholarship in AI Publications. AI Magazine, 39(3), 56-68. https://doi.org/10.1609/aimag.v39i3.2816
Section
Articles