Skip to main content

Currently Skimming:

Executive Summary
Pages 1-10

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 1...
... Evaluation studies can provide that information but only if those evaluations meet standards of quality. Under the auspices of the National Research Council, this committee's charge was to evaluate the quality of the evaluations of the 13 mathematics curriculum materials supported by the National Science Foundation (NSF)
From page 2...
... For the 95 comparative studies, the committee stipulated that they had to be "at least minimally methodologically adequate," which required that a study: · Include quantifiably measurable outcomes such as test scores, responses to specified cognitive tasks of mathematical reasoning, performance evaluations, grades, and subsequent course taking; and
From page 3...
... The remaining studies concerned commercially supported curricular materials. On the basis of the committee's analysis of these 147 studies, we concluded that the corpus of evaluation studies as a whole across the 19 programs studied does not permit one to determine the effectiveness of individual programs with a high degree of certainty, due to the restricted number of studies for any particular curriculum, limitations in the array of methods used, and the uneven quality of the studies.
From page 4...
... However, curricular effectiveness cannot be established by a single scientifically valid study; instead a body of studies is needed, which is the second key aspect of determining effectiveness. Curricular effectiveness is an integrated judgment based on interpretation of a number of scientifically valid evaluations that combine social values, empirical evidence, and theoretical rationales.
From page 5...
... In this standard, the committee recommends that a curricular program be designated as scientifically established as effective only when it includes a collection of scientifically valid evaluation studies addressing its effectiveness that establish that an implemented curricular program produces valid improvements in learning for students, and when it can convincingly demonstrate that these improvements are due to the curricular intervention. The collection of studies should use a combination of methodologies that meet these specified criteria: (1)
From page 6...
... Thus, to ensure the independence and impartiality of evaluations of effectiveness, the committee also recommends that summative evaluations be conducted by independent evaluation teams with no membership by authors of the curriculum materials or persons under their supervision. In the body of this report, the committee offers additional recommended practices for evaluators, which include: Representativeness.
From page 7...
... As a result of our study of the set of 63 at least minimally methodologically adequate comparative analyses, the committee recommends that in the conduct of all comparative studies, explicit attention be given to the following criteria: · Identify comparative curricula by name; · Employ random assignment, or otherwise establish adequate comparability; · Select the appropriate unit of analysis; · Document extent of implementation fidelity; · Select outcome measures that can be disaggregated by content strand; · Conduct appropriate statistical tests and report effect size; · Disaggregate data by gender, race/ethnicity, socioeconomic status (SES) , and performance levels, and express constraints as to the generalizability of study.
From page 8...
... However, this result does provide a testable hypothesis, a starting point for others to examine, critique, and undertake further studies to confirm or disconfirm. Then, after applying the criteria listed above, we found that the comparative studies of both NSF-supported and commercially generated curricula that had used the more rigorous criteria never produced contrary conclusions about curricular effectiveness (compared with less rigorous methods)
From page 9...
... At the federal level, such actions include: · Specifying more explicit expectations in requests for proposals for evaluation of curricular initiatives and increasing sophistication in methodological choices and quality; · Denying continued funding for major curricular programs that fail to present evaluation data from well-designed, scientifically valid studies; · Charging a federal agency with responsibility for collecting and maintaining district- and school-level data on curricula; and · Providing training, in concert with state agencies, to district and local agencies on conducting and interpreting studies of curricular effectiveness. For publishers, such actions include: · Differentiating market research from scientifically valid evaluation studies; and · Making evaluation data available to potential clients who use federal funds to purchase curriculum materials.
From page 10...
... Finally, the committee believes there is a need for multidisciplinary basic empirical research studies on curricular effectiveness. The federal government and publishers should support such studies on topics including, but not limited to: · The development of outcome measures at the upper level of secondary education and at the elementary level in non-numeration topics that are valid and precise at the topic level; · The interplay among curricular implementation, professional development, and the forms of support and professional interaction among teachers and administrators at the school level; · Methods of observing and documenting the type and quality of instruction; · Methods of parent and community education and involvement, and · Targets of curricular controversy such as the appropriate uses of technology; the relative use of analytic, visual, and numeric approaches; or the integration or segregation of the treatment of subfields, such as algebra, geometry, statistics, and others.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.