On the measurement of ideation quality

Bruce A. Reinig, Robert O. Briggs, Jay F Nunamaker

Research output: Contribution to journalArticle

107 Citations (Scopus)

Abstract

Ideation is a key step in organizational problem solving, so researchers have developed a variety of technological interventions for improving ideation quality, which we define as the degree to which an ideation activity produces ideas that are helpful in attaining a goal. In this paper, we examine the four measures typically used to evaluate ideation quality, including idea-count, sum-of-quality, average-quality, and good-idea-count, and discuss their validity and potential biases. An experimental study comparing three levels of social comparison was used to illustrate the differences in the ideation quality measures and revealed that research conclusions were dependent on the measure used. Based on our analysis of the measures and experimental results, we recommend that only good-idea-count be used as a measure to evaluate ideation treatments and call into question research that has based its findings on the other measures. Finally, we discuss implications for research and other potential approaches to evaluating ideation quality.

Original languageEnglish (US)
Pages (from-to)143-161
Number of pages19
JournalJournal of Management Information Systems
Volume23
Issue number4
DOIs
StatePublished - Mar 2007

Fingerprint

Ideation
trend
Experimental study
Problem solving
Social comparison

Keywords

  • Construct validity
  • Electronic brainstorming
  • Group support systems
  • Idea quality
  • Idea quantity
  • Ideation

ASJC Scopus subject areas

  • Information Systems
  • Management Information Systems
  • Library and Information Sciences
  • Management of Technology and Innovation
  • Strategy and Management

Cite this

On the measurement of ideation quality. / Reinig, Bruce A.; Briggs, Robert O.; Nunamaker, Jay F.

In: Journal of Management Information Systems, Vol. 23, No. 4, 03.2007, p. 143-161.

Research output: Contribution to journalArticle

Reinig, Bruce A. ; Briggs, Robert O. ; Nunamaker, Jay F. / On the measurement of ideation quality. In: Journal of Management Information Systems. 2007 ; Vol. 23, No. 4. pp. 143-161.
@article{626bf92433f34041a45a836a831b0e0d,
title = "On the measurement of ideation quality",
abstract = "Ideation is a key step in organizational problem solving, so researchers have developed a variety of technological interventions for improving ideation quality, which we define as the degree to which an ideation activity produces ideas that are helpful in attaining a goal. In this paper, we examine the four measures typically used to evaluate ideation quality, including idea-count, sum-of-quality, average-quality, and good-idea-count, and discuss their validity and potential biases. An experimental study comparing three levels of social comparison was used to illustrate the differences in the ideation quality measures and revealed that research conclusions were dependent on the measure used. Based on our analysis of the measures and experimental results, we recommend that only good-idea-count be used as a measure to evaluate ideation treatments and call into question research that has based its findings on the other measures. Finally, we discuss implications for research and other potential approaches to evaluating ideation quality.",
keywords = "Construct validity, Electronic brainstorming, Group support systems, Idea quality, Idea quantity, Ideation",
author = "Reinig, {Bruce A.} and Briggs, {Robert O.} and Nunamaker, {Jay F}",
year = "2007",
month = "3",
doi = "10.2753/MIS0742-1222230407",
language = "English (US)",
volume = "23",
pages = "143--161",
journal = "Journal of Management Information Systems",
issn = "0742-1222",
publisher = "M.E. Sharpe Inc.",
number = "4",

}

TY - JOUR

T1 - On the measurement of ideation quality

AU - Reinig, Bruce A.

AU - Briggs, Robert O.

AU - Nunamaker, Jay F

PY - 2007/3

Y1 - 2007/3

N2 - Ideation is a key step in organizational problem solving, so researchers have developed a variety of technological interventions for improving ideation quality, which we define as the degree to which an ideation activity produces ideas that are helpful in attaining a goal. In this paper, we examine the four measures typically used to evaluate ideation quality, including idea-count, sum-of-quality, average-quality, and good-idea-count, and discuss their validity and potential biases. An experimental study comparing three levels of social comparison was used to illustrate the differences in the ideation quality measures and revealed that research conclusions were dependent on the measure used. Based on our analysis of the measures and experimental results, we recommend that only good-idea-count be used as a measure to evaluate ideation treatments and call into question research that has based its findings on the other measures. Finally, we discuss implications for research and other potential approaches to evaluating ideation quality.

AB - Ideation is a key step in organizational problem solving, so researchers have developed a variety of technological interventions for improving ideation quality, which we define as the degree to which an ideation activity produces ideas that are helpful in attaining a goal. In this paper, we examine the four measures typically used to evaluate ideation quality, including idea-count, sum-of-quality, average-quality, and good-idea-count, and discuss their validity and potential biases. An experimental study comparing three levels of social comparison was used to illustrate the differences in the ideation quality measures and revealed that research conclusions were dependent on the measure used. Based on our analysis of the measures and experimental results, we recommend that only good-idea-count be used as a measure to evaluate ideation treatments and call into question research that has based its findings on the other measures. Finally, we discuss implications for research and other potential approaches to evaluating ideation quality.

KW - Construct validity

KW - Electronic brainstorming

KW - Group support systems

KW - Idea quality

KW - Idea quantity

KW - Ideation

UR - http://www.scopus.com/inward/record.url?scp=38349176773&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=38349176773&partnerID=8YFLogxK

U2 - 10.2753/MIS0742-1222230407

DO - 10.2753/MIS0742-1222230407

M3 - Article

AN - SCOPUS:38349176773

VL - 23

SP - 143

EP - 161

JO - Journal of Management Information Systems

JF - Journal of Management Information Systems

SN - 0742-1222

IS - 4

ER -