Measuring the conceptual understandings of citizen scientists participating in zooniverse projects: A first approach

Edward E Prather, Sébastien Cormier, Colin S. Wallace, Chris Lintott, M. Jordan Raddick, Arfon Smith

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

The Zooniverse projects turn everyday people into "citizen scientists" who work online with real data to assist scientists in conducting research on a variety of topics related to galaxies, exoplanets, lunar craters, and solar flares, among others. This paper describes our initial study to assess the conceptual knowledge and reasoning abilities of citizen scientists participating in two Zooniverse projects: Galaxy Zoo and Moon Zoo. In order to measure their knowledge and abilities, we developed two new assessment instruments, the Zooniverse Astronomical Concept Survey (ZACS) and the Lunar Cratering Concept Inventory (LCCI). We found that citizen scientists with the highest level of participation in the Galaxy Zoo and Moon Zoo projects also have the highest average correct scores on the items of the ZACS and LCCI. However, the limited nature of the data provided by Zooniverse participants prevents us from being able to evaluate the statistical significance of this finding, and we make no claim about whether there is a causal relationship between one's participation in Galaxy Zoo or Moon Zoo and one's level of conceptual understanding or reasoning ability on the astrophysical topics assessed by the ZACS or the LCCI. Overall, both the ZACS and the LCCI provide Zooniverse's citizen scientists with items that offer a wide range of difficulties. Using the data from the small subset of participants who responded to all items of the ZACS, we found evidence suggesting the ZACS is a reliable instrument (α=0.78), although twenty-one of its forty items appear to have point biserials less than 0.3. The work reported here provides significant insight into the strengths and limitations of various methods for administering assessments to citizen scientists. Researchers who wish to study the knowledge and abilities of citizen scientists in the future should be sure to design their research methods to avoid the pitfalls identified by our initial findings.

Original languageEnglish (US)
Article number010109
JournalAstronomy Education Review
Volume12
Issue number1
DOIs
StatePublished - 2013

Fingerprint

cratering
citizen
moon
galaxies
ability
lunar craters
extrasolar planets
solar flares
set theory
participation
astrophysics
statistical significance
conduction
research method
evidence

ASJC Scopus subject areas

  • Education
  • Astronomy and Astrophysics

Cite this

Measuring the conceptual understandings of citizen scientists participating in zooniverse projects : A first approach. / Prather, Edward E; Cormier, Sébastien; Wallace, Colin S.; Lintott, Chris; Jordan Raddick, M.; Smith, Arfon.

In: Astronomy Education Review, Vol. 12, No. 1, 010109, 2013.

Research output: Contribution to journalArticle

Prather, Edward E ; Cormier, Sébastien ; Wallace, Colin S. ; Lintott, Chris ; Jordan Raddick, M. ; Smith, Arfon. / Measuring the conceptual understandings of citizen scientists participating in zooniverse projects : A first approach. In: Astronomy Education Review. 2013 ; Vol. 12, No. 1.
@article{23fd4166db8845ba80cc2b8038bc8836,
title = "Measuring the conceptual understandings of citizen scientists participating in zooniverse projects: A first approach",
abstract = "The Zooniverse projects turn everyday people into {"}citizen scientists{"} who work online with real data to assist scientists in conducting research on a variety of topics related to galaxies, exoplanets, lunar craters, and solar flares, among others. This paper describes our initial study to assess the conceptual knowledge and reasoning abilities of citizen scientists participating in two Zooniverse projects: Galaxy Zoo and Moon Zoo. In order to measure their knowledge and abilities, we developed two new assessment instruments, the Zooniverse Astronomical Concept Survey (ZACS) and the Lunar Cratering Concept Inventory (LCCI). We found that citizen scientists with the highest level of participation in the Galaxy Zoo and Moon Zoo projects also have the highest average correct scores on the items of the ZACS and LCCI. However, the limited nature of the data provided by Zooniverse participants prevents us from being able to evaluate the statistical significance of this finding, and we make no claim about whether there is a causal relationship between one's participation in Galaxy Zoo or Moon Zoo and one's level of conceptual understanding or reasoning ability on the astrophysical topics assessed by the ZACS or the LCCI. Overall, both the ZACS and the LCCI provide Zooniverse's citizen scientists with items that offer a wide range of difficulties. Using the data from the small subset of participants who responded to all items of the ZACS, we found evidence suggesting the ZACS is a reliable instrument (α=0.78), although twenty-one of its forty items appear to have point biserials less than 0.3. The work reported here provides significant insight into the strengths and limitations of various methods for administering assessments to citizen scientists. Researchers who wish to study the knowledge and abilities of citizen scientists in the future should be sure to design their research methods to avoid the pitfalls identified by our initial findings.",
author = "Prather, {Edward E} and S{\'e}bastien Cormier and Wallace, {Colin S.} and Chris Lintott and {Jordan Raddick}, M. and Arfon Smith",
year = "2013",
doi = "10.3847/AER2013002",
language = "English (US)",
volume = "12",
journal = "Astronomy Education Review",
issn = "1539-1515",
publisher = "AIP Publishing LLC",
number = "1",

}

TY - JOUR

T1 - Measuring the conceptual understandings of citizen scientists participating in zooniverse projects

T2 - A first approach

AU - Prather, Edward E

AU - Cormier, Sébastien

AU - Wallace, Colin S.

AU - Lintott, Chris

AU - Jordan Raddick, M.

AU - Smith, Arfon

PY - 2013

Y1 - 2013

N2 - The Zooniverse projects turn everyday people into "citizen scientists" who work online with real data to assist scientists in conducting research on a variety of topics related to galaxies, exoplanets, lunar craters, and solar flares, among others. This paper describes our initial study to assess the conceptual knowledge and reasoning abilities of citizen scientists participating in two Zooniverse projects: Galaxy Zoo and Moon Zoo. In order to measure their knowledge and abilities, we developed two new assessment instruments, the Zooniverse Astronomical Concept Survey (ZACS) and the Lunar Cratering Concept Inventory (LCCI). We found that citizen scientists with the highest level of participation in the Galaxy Zoo and Moon Zoo projects also have the highest average correct scores on the items of the ZACS and LCCI. However, the limited nature of the data provided by Zooniverse participants prevents us from being able to evaluate the statistical significance of this finding, and we make no claim about whether there is a causal relationship between one's participation in Galaxy Zoo or Moon Zoo and one's level of conceptual understanding or reasoning ability on the astrophysical topics assessed by the ZACS or the LCCI. Overall, both the ZACS and the LCCI provide Zooniverse's citizen scientists with items that offer a wide range of difficulties. Using the data from the small subset of participants who responded to all items of the ZACS, we found evidence suggesting the ZACS is a reliable instrument (α=0.78), although twenty-one of its forty items appear to have point biserials less than 0.3. The work reported here provides significant insight into the strengths and limitations of various methods for administering assessments to citizen scientists. Researchers who wish to study the knowledge and abilities of citizen scientists in the future should be sure to design their research methods to avoid the pitfalls identified by our initial findings.

AB - The Zooniverse projects turn everyday people into "citizen scientists" who work online with real data to assist scientists in conducting research on a variety of topics related to galaxies, exoplanets, lunar craters, and solar flares, among others. This paper describes our initial study to assess the conceptual knowledge and reasoning abilities of citizen scientists participating in two Zooniverse projects: Galaxy Zoo and Moon Zoo. In order to measure their knowledge and abilities, we developed two new assessment instruments, the Zooniverse Astronomical Concept Survey (ZACS) and the Lunar Cratering Concept Inventory (LCCI). We found that citizen scientists with the highest level of participation in the Galaxy Zoo and Moon Zoo projects also have the highest average correct scores on the items of the ZACS and LCCI. However, the limited nature of the data provided by Zooniverse participants prevents us from being able to evaluate the statistical significance of this finding, and we make no claim about whether there is a causal relationship between one's participation in Galaxy Zoo or Moon Zoo and one's level of conceptual understanding or reasoning ability on the astrophysical topics assessed by the ZACS or the LCCI. Overall, both the ZACS and the LCCI provide Zooniverse's citizen scientists with items that offer a wide range of difficulties. Using the data from the small subset of participants who responded to all items of the ZACS, we found evidence suggesting the ZACS is a reliable instrument (α=0.78), although twenty-one of its forty items appear to have point biserials less than 0.3. The work reported here provides significant insight into the strengths and limitations of various methods for administering assessments to citizen scientists. Researchers who wish to study the knowledge and abilities of citizen scientists in the future should be sure to design their research methods to avoid the pitfalls identified by our initial findings.

UR - http://www.scopus.com/inward/record.url?scp=84885903096&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84885903096&partnerID=8YFLogxK

U2 - 10.3847/AER2013002

DO - 10.3847/AER2013002

M3 - Article

AN - SCOPUS:84885903096

VL - 12

JO - Astronomy Education Review

JF - Astronomy Education Review

SN - 1539-1515

IS - 1

M1 - 010109

ER -