Toward AI Research Methodology: Three Case Studies in Evaluation

Paul R. Cohen, Adele E. Howe

Research output: Contribution to journalArticle

30 Scopus citations

Abstract

The roles of evaluation in empirical artificial intelligence (Al) research are described, in an idealized cyclic model and in the context of three case studies. The case studies illustrate pitfalls in evaluation and the contributions of evaluation at all stages of the research cycle. Evaluation methods are contrasted with those of the behavioral sciences, and it is concluded that AI must define and refine its own methods. To this end, several experiment “schemas” and many specific evaluation criteria are described; recommendations are offered in the hope of encouraging the development and practice of evaluation methods in AI.

Original languageEnglish (US)
Pages (from-to)634-646
Number of pages13
JournalIEEE Transactions on Systems, Man and Cybernetics
Volume19
Issue number3
DOIs
StatePublished - Jan 1 1989

ASJC Scopus subject areas

  • Engineering(all)

Fingerprint Dive into the research topics of 'Toward AI Research Methodology: Three Case Studies in Evaluation'. Together they form a unique fingerprint.

  • Cite this