On Evaluating Curricular Effectiveness Judging the Quality of K-12 Mathematics Evaluations (2004) / Chapter Skim
Currently Skimming:
6 Case Studies and Synthesis Studies
6 Case Studies and Synthesis Studies
Pages 167-184
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 167... ...
Case studies typically document "what happened" differently than do comparative studies. Case studies provide insight into mechanisms at play that are hidden from a comparison of student achievement.
|
From page 168... ...
For our purposes the study also had to focus on 1 of the 13 mathematics curricula supported by the National Science Foundation (NSF) , the University of Chicago School Mathematics Project (UCSMP)
|
From page 169... ...
TABLE 6-1 Distribution of Case Studies by Curricula Number of Studies NSF-Supported Elementary Curriculum Materials Everyday Mathematics (EM) 4 Investigations in Number, Data and Space/TERC 1 Math Trailblazers 1 NSF-Supported Middle School Curriculum Materials Connected Mathematics Project (CMP)
|
From page 170... ...
Authors backed their claims when they used a methodology that in cluded data, a way to analyze data systematically, and a form of argu ment that could support a reader's reaching a similar or contrary con clusion. This criterion permitted us to distinguish a case study from an anecdotal report that told a story, but did not indicate how the data were systematically collected, linked to program theory, and analyzed and evaluated.
|
From page 171... ...
Findings Case studies in this review were found only in the NSF-supported mathematics curriculum materials. Therefore, the generalization of results must be restricted to NSF-supported curricula; case studies of commercially generated curricula would be needed to draw broader conclusions.
|
From page 172... ...
Case studies can therefore inform future evaluators about potential explanatory variables to include the conduct of future evaluations. Recurrent issues among the case studies were as follows: · Design features affect student subpopulations differentially; · Common practices, beliefs, and understandings among teachers and students interact in unanticipated ways with characteristics of these curricula; · Professional development is an essential consideration; and · Time and resource allocations must be carefully managed.
|
From page 173... ...
173 into Insight Mechanism Design Replicable Claims Backed Criterion done. well Each (3)
|
From page 174... ...
, in a comparison of Everyday Mathematics and Heath Mathematics, found that while average- and high-ability students seemed to benefit from using Everyday Mathematics in relation to the comparison groups, low-achieving students in both groups performed at comparatively the same level and showed only modest improvement over time. In a follow-up qualitative study of why low-achieving students benefited less than higher achieving students in Everyday Mathematics, Baxter et al.
|
From page 175... ...
Attending to these interactions in program evaluations may provide more precise understandings of a curriculum's differential impact among student subgroups and of its differential impact among implementation sites. It might also lead to a deeper understanding about how cognitive and conceptual accomplishments are produced through the interactions among curricular tasks and student and teacher participation patterns (Greeno and Goldman, 1998)
|
From page 176... ...
provided excerpts of classroom dialog suggesting that the degree to which teachers and students speak calculationally could be an important factor in how successfully they implement curricula that place a premium on public discourse in the service of teaching for understanding. Other studies suggested that the degree to which teachers are oriented to making sense of mathematical ideas for students can be important factors both in using public discussion productively (Kett, 1997; Smith, 1998)
|
From page 177... ...
Collins found that "students in schools whose teachers received sustained professional development designed to meet the needs of the participating teachers performed significantly higher on both the Massachusetts Comprehensive Assessment System (MCAS) and a nationally normed achievement test, TerraNova, than did those students whose teachers had not participated in consistent professional development" (p.
|
From page 178... ...
documented the difficulty teachers had in organizing their instruction into coherent chunks that had educationally appropriate beginning and ending points and yet fit into fixed time blocks in the school day. They pointed out the difficulty of parsing a curriculum organized around conceptual themes into predetermined time blocks in comparison to doing the same with a curriculum that is organized by topics, facts, and procedures that typically are presented in smaller units.
|
From page 179... ...
Prior comments notwithstanding, the case studies examined by the committee provided valuable information about variables that program evaluations should include and about the roles that case studies can play in those evaluations. The variables identified by examining case study results arose primarily because people wondered about what it meant for a particular curriculum to be effective.
|
From page 180... ...
Descriptions of how the primary constructs were identified and verified also would be helpful. SYNTHESIS STUDIES For the purposes of this study, a synthesis study summarizes several evaluation studies across a particular curriculum, discusses the results drawn from these data, and draws conclusions based on the data and the discussion.
|
From page 181... ...
. Each of the 16 synthesis studies is authored by a senior writer of the curriculum materials or by a person closely allied with the curriculum.
|
From page 182... ...
They begin with a brief introduction describing the context for the creation of the SIMMS curriculum as part of the NSF-funded State Systemic Initiative in Montana. Then they summarize the history of the curriculum as growing out of a 1989 national survey, "Integrated Mathematics Project," funded by the Exxon Education Foundation.
|
From page 183... ...
Although these syntheses provided important sources of integrated data on the programs, we found that they tended to lack critical scrutiny and thus may not convince readers that the authors had sought out and included competing interpretations. A common database of variables that all evaluation studies contained could assist researchers when doing synthesis studies and possibly provide additional reader confidence in the findings.
|
From page 184... ...
Nonetheless, the committee encourages synthesis evaluations, and funding agencies should consider supporting them, as a means to build on previous knowledge, to provide a summary of existing studies, to enhance understanding of the effectiveness of the various curricula, to build scientific consensus on certain aspects of education research, and to contribute to theory building.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.