Skip to main content

Currently Skimming:

10. Drawing Inferences for National Policy from Large-Scale Cross-National Education Surveys
Pages 295-318

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 295...
... 3. Are there lessons from the past that might be used to improve the quality of policy considerations stimulated by results of TIMSS-R, the 1999 repeat survey of the 1995 Third International Mathematics and Science Study (TIMSS)
From page 296...
... students in grades beyond elementary school scored less well on the tests than did students in many other countries. Second, the surveys underscored the importance of the content of the curriculum.
From page 297...
... scores had peaked almost a decade earlier, a fact that might have provided support for the cross-national findings. During the late 1970s, cross-national surveys continued to be unnoticed in policy debates at the national level.3 National policy focused on how to improve the effectiveness of schools for poor and minority students even as SAT scores continued to drop and the progressive policies initiated in the 1960s were challenged by opposition to northern school desegregation and negative evaluations of Head Start and Title I
From page 298...
... The U.S. Educational Environment "National education policy" in the United States whether the emphasis was on equal opportunity in the 1960s and 1970s or the push for academic rigor in the 1980s has generally a weaker influence on school practice than does national educational policy in many other countries.
From page 299...
... education while complicating the picture more. The past half-century has seen organized national interest groups in education expand dramatically in size and authority, and the forces of communication, transportation, and mobility have helped create schools that look, and are, extraordinarily alike from state to state across the nation.
From page 300...
... At the same time, the variation among states in the nature of their politics and their finance and governance systems and the dispersed nature of control within states mean that no one can expect a uniform national response across or even within states to any particular policy. What Levels of Government in the United States Should Be Interested in Policy Inferences from TIMSS?
From page 301...
... students achieve in math and science, as defined by the TIMSS assessments, relatively well in the fourth grade and relatively badly in the eighth grade. However, unless we are convinced that the TIMSS mathematics and science assessments substantially measure what is taught in U.S.
From page 302...
... . For example, 16 of the 21 countries that participated in the Population 3 Upper Grade Mathematics Literacy assessment were in violation of the international sampling guidelines; one of the 16 was the United States.
From page 303...
... For example, using only the TIMSS data, when analyzed with the most powerful statistical techniques, we cannot validly assert that differences in the pedagogy used by teachers in different nations result in differences in student achievement. We do not have adequate measures of differences in pedagogy nor a theory that is robust enough to account for the variation in the contexts of different nations.
From page 304...
... . A nonfinding no single-variable "magic bullets." In the original TIMSS reports, there are no outstanding examples of single variables found to have special power to explain differences among countries in level of student achievement.6 Difference in class size, the kind of governance system, and per-pupil expenditure, for example, are all viewed as operating within specific national contexts.
From page 305...
... Suppose, however, that the TIMSS reports or reanalyses reported single-variable relationships at the national level between school resources and student achievement. Imagine, then, that class size had statistically differentiated among high- and low-scoring nations in TIMSS.
From page 306...
... However, in my view the international surveys have been critical in forming a national alliance within the United States around this policy without crossnational student achievement "benchmarks," it would have been much more difficult to align forces within the United States in support of more challenging standards. A thought experiment that suggests the power of this kind of international benchmark is to ask what would have happened if the early crossnational surveys had found that U.S.
From page 307...
... after all, our NAEP scores show that 38 percent of the nation's fourth graders "cannot read." Although psychometricians know that an arbitrary cut score that has little to do with an actual capacity to read creates this number, most politicians and people in the press appear to believe it means that our schooling is failing nearly two-fifths of fourth graders. Another reason people may overlook the finding on fourth-grade achievement is that it is politically unwise to suggest that early intervention is not a priority.
From page 308...
... For example, the Second International Mathematics Study (SIMS) received national attention because of a similar conclusion.
From page 309...
... The achievement data are taken from the assessments, the curriculum data from the achievement survey questionnaire and a separate data bank and analyses of the curricula in many nations, and the teaching data from questionnaires and videotapes. Over the past few years, the inferential leap that this is what TIMSS shows has been publicly made or implied hundreds of times by many dozens of thoughtful people.
From page 310...
... Both saw the basic TIMSS data releases as well as the release of the data for the First in the World Consortium as powerful opportunities to support administration policies. The exposure for the administration was enhanced by the fact that different parts of the study were released over a six-month period, offering a number of opportunities for administration officials to comment to the press on the studies and their relationship to current policy.
From page 311...
... I argued against the possibility of drawing causal inferences and for a careful examination of context and corroborating data from other sources in drawing "weak," noncausal policy inferences. I also argued that the politically most powerful "weak" inference is a "synthetic" inference made up of a number of parts that together tell a plausible story.
From page 312...
... That is, would state and/or federal policy have been different if there had been no Third International Mathematics and Science Study? We do know that 14 districts or clusters of districts, including the First in the World Consortium, the Miami-Dade County Public Schools, and the Smart Consortium of districts in Ohio, as well as 13 states, opted to be part of TIMSS-R and took various administrative and policy steps to help students prepare for it.9 We might infer from this that TIMSS and the advent of TIMSS-R have influenced policy formulation and development in a substantial number of states and communities around the nation.
From page 313...
... Could BICSE suggest guidelines and develop useful benchmarks for state policy makers as they interpret the results of the national data and of the data on their own states? Are there ways to think about the appropriateness of various policy inferences suggested by TIMSS-R for different state environments?
From page 314...
... For example, what are the effects of jukus on the test taking of eighth graders in Japan and Korea? Does the curriculum of a juku in a particular country reinforce the curriculum in the country's schools?
From page 315...
... established in the early 1990s proposed five competencies, in addition to basic and advanced academic skills, as necessary for people to have when they enter the modern workplace.~ The five competencies include interpersonal skills, knowledge of systems, and the use of technology and information. Considerable evidence exists that motivation and the skill of working in groups have powerful effects on the quality of a person's work.
From page 316...
... 5. It should come as little surprise that there were substantial weaknesses in the inferences and policy conclusions drawn from analyses of test scores (including scores from the cross-cultural assessments)
From page 317...
... Paper prepared for Learning from TIMSS: An NRC Symposium on the Results of the Third International Mathematics and Science Study, convened by the Board on International Comparative Studies in Education, National Research Council, National Academy of Sciences, Washington, D.C. Haertel, E


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.