Evaluating the model activity systems: General methodological considerations

Margaret Gallego, Luis C Moll, Robert Rueda

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

Our highest ambition was to find ways to conduct diagnostic evaluations in situ-that is, in activity. To some extent this proved possible, but showing development in situ does not meet the criteria of causal social science analysis. Although in situ evaluations may be an effective tool locally, they are unlikely to be considered appropriate under contemporary views of what constitutes "real science" (Lyon 1999; Slavin 2003). We sought to solve these problems by adopting a multi-method approach that takes advantage of the analytic potential in the broad variety of settings provided by the project. In some cases, we approached, or even met, the standard that takes the psychological experiment of individual college sophomores as the model. In other cases, we provided alternative, quasi-experimental evidence. In still others, we used a qualitative longitu dinal approach that focused on interactions of a quality that, according to conclusions drawn from "proper" scientific research, should underpin the quantitative results. In the chapters that follow, readers must reach their own conclusions about the adequacy of the evidence that the Fifth Dimension is an effective program. Some may find our strategic mixing of methods, locations, and opportunities a useful tool for thinking about their own efforts at program evaluation and analysis of psychological development. Others may argue that the work should be dismissed because we simply did not succeed in getting our local communities to adhere to scientific criteria. We hope that most readers will come to the former conclusion, but expect at least some to draw the latter.

Original languageEnglish (US)
Title of host publicationThe Fifth Dimension: An After-School Program Built On Diversity
PublisherRussell Sage Foundation
Pages66-84
Number of pages19
ISBN (Print)0871540843, 9780871540843
StatePublished - 2006

Fingerprint

evaluation
psychological development
evidence
diagnostic
social science
experiment
interaction
science
community

ASJC Scopus subject areas

  • Social Sciences(all)

Cite this

Gallego, M., Moll, L. C., & Rueda, R. (2006). Evaluating the model activity systems: General methodological considerations. In The Fifth Dimension: An After-School Program Built On Diversity (pp. 66-84). Russell Sage Foundation.

Evaluating the model activity systems : General methodological considerations. / Gallego, Margaret; Moll, Luis C; Rueda, Robert.

The Fifth Dimension: An After-School Program Built On Diversity. Russell Sage Foundation, 2006. p. 66-84.

Research output: Chapter in Book/Report/Conference proceedingChapter

Gallego, M, Moll, LC & Rueda, R 2006, Evaluating the model activity systems: General methodological considerations. in The Fifth Dimension: An After-School Program Built On Diversity. Russell Sage Foundation, pp. 66-84.
Gallego M, Moll LC, Rueda R. Evaluating the model activity systems: General methodological considerations. In The Fifth Dimension: An After-School Program Built On Diversity. Russell Sage Foundation. 2006. p. 66-84
Gallego, Margaret ; Moll, Luis C ; Rueda, Robert. / Evaluating the model activity systems : General methodological considerations. The Fifth Dimension: An After-School Program Built On Diversity. Russell Sage Foundation, 2006. pp. 66-84
@inbook{64b72f6ed49441e7b239b5400f03dc41,
title = "Evaluating the model activity systems: General methodological considerations",
abstract = "Our highest ambition was to find ways to conduct diagnostic evaluations in situ-that is, in activity. To some extent this proved possible, but showing development in situ does not meet the criteria of causal social science analysis. Although in situ evaluations may be an effective tool locally, they are unlikely to be considered appropriate under contemporary views of what constitutes {"}real science{"} (Lyon 1999; Slavin 2003). We sought to solve these problems by adopting a multi-method approach that takes advantage of the analytic potential in the broad variety of settings provided by the project. In some cases, we approached, or even met, the standard that takes the psychological experiment of individual college sophomores as the model. In other cases, we provided alternative, quasi-experimental evidence. In still others, we used a qualitative longitu dinal approach that focused on interactions of a quality that, according to conclusions drawn from {"}proper{"} scientific research, should underpin the quantitative results. In the chapters that follow, readers must reach their own conclusions about the adequacy of the evidence that the Fifth Dimension is an effective program. Some may find our strategic mixing of methods, locations, and opportunities a useful tool for thinking about their own efforts at program evaluation and analysis of psychological development. Others may argue that the work should be dismissed because we simply did not succeed in getting our local communities to adhere to scientific criteria. We hope that most readers will come to the former conclusion, but expect at least some to draw the latter.",
author = "Margaret Gallego and Moll, {Luis C} and Robert Rueda",
year = "2006",
language = "English (US)",
isbn = "0871540843",
pages = "66--84",
booktitle = "The Fifth Dimension: An After-School Program Built On Diversity",
publisher = "Russell Sage Foundation",

}

TY - CHAP

T1 - Evaluating the model activity systems

T2 - General methodological considerations

AU - Gallego, Margaret

AU - Moll, Luis C

AU - Rueda, Robert

PY - 2006

Y1 - 2006

N2 - Our highest ambition was to find ways to conduct diagnostic evaluations in situ-that is, in activity. To some extent this proved possible, but showing development in situ does not meet the criteria of causal social science analysis. Although in situ evaluations may be an effective tool locally, they are unlikely to be considered appropriate under contemporary views of what constitutes "real science" (Lyon 1999; Slavin 2003). We sought to solve these problems by adopting a multi-method approach that takes advantage of the analytic potential in the broad variety of settings provided by the project. In some cases, we approached, or even met, the standard that takes the psychological experiment of individual college sophomores as the model. In other cases, we provided alternative, quasi-experimental evidence. In still others, we used a qualitative longitu dinal approach that focused on interactions of a quality that, according to conclusions drawn from "proper" scientific research, should underpin the quantitative results. In the chapters that follow, readers must reach their own conclusions about the adequacy of the evidence that the Fifth Dimension is an effective program. Some may find our strategic mixing of methods, locations, and opportunities a useful tool for thinking about their own efforts at program evaluation and analysis of psychological development. Others may argue that the work should be dismissed because we simply did not succeed in getting our local communities to adhere to scientific criteria. We hope that most readers will come to the former conclusion, but expect at least some to draw the latter.

AB - Our highest ambition was to find ways to conduct diagnostic evaluations in situ-that is, in activity. To some extent this proved possible, but showing development in situ does not meet the criteria of causal social science analysis. Although in situ evaluations may be an effective tool locally, they are unlikely to be considered appropriate under contemporary views of what constitutes "real science" (Lyon 1999; Slavin 2003). We sought to solve these problems by adopting a multi-method approach that takes advantage of the analytic potential in the broad variety of settings provided by the project. In some cases, we approached, or even met, the standard that takes the psychological experiment of individual college sophomores as the model. In other cases, we provided alternative, quasi-experimental evidence. In still others, we used a qualitative longitu dinal approach that focused on interactions of a quality that, according to conclusions drawn from "proper" scientific research, should underpin the quantitative results. In the chapters that follow, readers must reach their own conclusions about the adequacy of the evidence that the Fifth Dimension is an effective program. Some may find our strategic mixing of methods, locations, and opportunities a useful tool for thinking about their own efforts at program evaluation and analysis of psychological development. Others may argue that the work should be dismissed because we simply did not succeed in getting our local communities to adhere to scientific criteria. We hope that most readers will come to the former conclusion, but expect at least some to draw the latter.

UR - http://www.scopus.com/inward/record.url?scp=84902625662&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84902625662&partnerID=8YFLogxK

M3 - Chapter

AN - SCOPUS:84902625662

SN - 0871540843

SN - 9780871540843

SP - 66

EP - 84

BT - The Fifth Dimension: An After-School Program Built On Diversity

PB - Russell Sage Foundation

ER -