On Evaluating Curricular Effectiveness Judging the Quality of K-12 Mathematics Evaluations (2004) / Chapter Skim
Currently Skimming:
2 Committee Procedures and Characteristics of Studies Reviewed
2 Committee Procedures and Characteristics of Studies Reviewed
Pages 18-35
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 18... ...
We were not charged with and therefore did not: · Evaluate the curriculum materials directly · Rate or rank specific curricular programs We recognize that both tasks could interest a broad constituency, but we believed that the field would profit from a careful, thorough review and summary of previous evaluations and research studies in relation to how previous work might inform and strengthen future efforts. We were aware that the mathematics education field lacks a clear consensus on what constitutes an effective curriculum and how to measure it to provide adequate, valid, and timely information to decision making bodies.
|
From page 19... ...
Recognizing the complexity of judging curricular effectiveness, this report is designed to assist future evaluators and policy makers in designing and conducting evaluations that provide accurate, comprehensive, and valid advice to decision makers and practitioners on the efficacy of curriculum materials. Our primary goal was to advise our audiences on what could be learned from these initial efforts and how lessons learned, strategic decisions, adaptations in method, errors and weaknesses, and tentative patterns of results could further future evaluation efforts and decision making on curricular policy.
|
From page 20... ...
Publishers produce numerous overlapping and sometimes competing mathematics curriculum products, especially given recent acquisitions and mergers. Thus determining market share by program was problematic.
|
From page 21... ...
3 4. Pearson (including Addison Wesley Longman, Scott Foresman, Silver Burdett Ginn, Simon and Schuster, Globe Fearon, Modern Curriculum Press, Celebration Press, Dale Seymour Publications, Prentice Hall School, Waterford Early Reading, Waterford Early Math and Science, Sing, Spell, Read and Write)
|
From page 22... ...
It eventually included five components, including an elementary component that produced Everyday Math (headed by Max Bell, with NSF support, and included in this study) , a professional development component with NSF support, and an evaluation component headed by Larry Hedges and Susan Stodolsky.
|
From page 23... ...
The second workshop on commercially generated materials asked the same general questions, with two additional requests for comments: · How do you evaluate materials in relation to the quality and effectiveness of the materials themselves, including content analysis, theories of
|
From page 24... ...
Sixteen prominent mathematicians and mathematics educators from a variety of perspectives on content analysis were identified. We sent a written request and received statements from eight: Richard Askey, University of Wisconsin, Madison; Eric Gutstein, University of Illinois, Chicago; Roger Howe, Yale University; William McCallum, University of Arizona; R
|
From page 25... ...
STUDY CHARACTERISTICS Table 2-1 shows the distribution of studies by methodology and identifies them as NSF-supported, commercially generated, or UCSMP. We identified studies that fit the categories of content analysis (n=36)
|
From page 27... ...
27 studies. evaluation categorizing for tree Decision 2-1 FIGURE
|
From page 28... ...
. Three of the five curricula with the most evaluation studies under review received additional NSF funding to conduct revisions.7 The elementary, Everyday Mathematics, and secondary components of the UCSMP materials followed.
|
From page 29... ...
5 1 2 2 10 20 Math Connections 2 0 2 2 6 12 Mathematics: Modeling Our World (MMOW/ARISE) 0 0 2 1 5 8 Commercially generated curricula Elementary Addison Wesley/Scott Foresman 0 0 2 0 1 3 Harcourt Brace 0 0 1 0 0 1 Middle school Saxon 13 0 6 0 21 40 Glencoe/McGraw-Hill 1 0 2 0 4 7 High school Prentice Hall/UCSMP 13 0 14 3 46 76 Houghton-Mifflin/ McDougal Littell 2 0 0 0 1 3 Number of evaluation studies 95 45 36 16 225 417 Number of times each curriculum is in each type of study 102 53 68 29 275 528 *
|
From page 30... ...
Approximately 28 percent of comparative studies and 31 percent of case studies were unpublished doctoral dissertations. Although dissertations are unpublished, these studies have been vigorously screened and provided valuable insight into current evaluation data on the curricula under review.
|
From page 31... ...
Where? Hours of professional development received Unit of analysis Use of supplemental materials Study of duration Mathematics certified Research question Average number of years of Outcome measures: Student level teaching experience Standardized tests School population: Sample and comparison Other measures (attitudes, absentee rates, or dropout rates)
|
From page 32... ...
Synthesis 16 16 0 0 0 NSF-supported curricula 15 15 0 0 0 Commercially generated curricula 0 0 0 0 0 UCSMP 1 1 0 0 0 Not counted in above 0 0 0 0 0 TOTAL 192 77 61 43 11 Table 2-4 shows the distribution of the author's background by type of study. Across the four types of evaluation studies reviewed, 40 percent were done by authors internal to the curriculum project studied.
|
From page 33... ...
Percent Number of Studies Only the of to Report State/ Type of Study Studies Data Urban Suburban Rural Region 1. Comparative analysis 95 88 36 46 31 37 NSF-supported curricula 66 89 29 38 24 35 Commercially generated curricula 16 81 31 44 19 38 UCSMP 11 91 40 60 40 20 Not counted in above 2 100 100 50 50 0 2.
|
From page 34... ...
. Content analysis studies are not included in Tables 2-5 and 2-6 because they do not report data on students or sample school locations.
|
From page 35... ...
However, few included any type of measure of quality of instruction, although case studies were more likely to provide insight into these factors than were comparative studies.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.