Evaluating Artifacts Using Experiments as Part of the Design Science Framework

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Design science is of increasing importance in information systems (IS) with top journals and funding agencies recognizing it as an important component of IS research. One critical element of good design science is a solid evaluation of the artifacts, i.e., of individual algorithms and completed information systems. This tutorial focuses on the design and execution of experiments to evaluate such artifacts in an efficient manner. The artifact's life cycle is taken into account to ensure the evaluation is appropriate, effective and efficient through optimal use of resources. Evaluations can range from automated evaluations of algorithms using gold standards to user studies of an entire information system with representative users in situ. The tutorial will focus on the controlled experiment for artifacts, appropriate statistical analysis, and tips on common errors that can be avoided.

Original languageEnglish (US)
Title of host publicationProceedings
Subtitle of host publicationRCIS 2019 - IEEE 13th International Conference on Research Challenges in Information Science: Towards a Design Science for Information Systems
EditorsManuel Kolp, Jean Vanderdonckt, Monique Snoeck, Yves Wautelet
PublisherIEEE Computer Society
ISBN (Electronic)9781728148441
DOIs
StatePublished - May 2019
Event13th IEEE International Conference on Research Challenges in Information Science, RCIS 2019 - Brussels, Belgium
Duration: May 29 2019May 31 2019

Publication series

NameProceedings - International Conference on Research Challenges in Information Science
Volume2019-May
ISSN (Print)2151-1349
ISSN (Electronic)2151-1357

Conference

Conference13th IEEE International Conference on Research Challenges in Information Science, RCIS 2019
CountryBelgium
CityBrussels
Period5/29/195/31/19

    Fingerprint

Keywords

  • designs science
  • evaluation
  • experiment
  • user study

ASJC Scopus subject areas

  • Computer Science Applications
  • Information Systems
  • Software

Cite this

Gondy, L. (2019). Evaluating Artifacts Using Experiments as Part of the Design Science Framework. In M. Kolp, J. Vanderdonckt, M. Snoeck, & Y. Wautelet (Eds.), Proceedings: RCIS 2019 - IEEE 13th International Conference on Research Challenges in Information Science: Towards a Design Science for Information Systems [8876947] (Proceedings - International Conference on Research Challenges in Information Science; Vol. 2019-May). IEEE Computer Society. https://doi.org/10.1109/RCIS.2019.8876947