Methodological Advances in Cross-National Surveys of Educational Achievement (2002) / Chapter Skim
Currently Skimming:
1. Progress and Challenges for Large-Scale Studies
1. Progress and Challenges for Large-Scale Studies
Pages 1-24
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
Introduction
|
From page 3... ...
A crucial issue in judging the value of international studies is the quality of their methodologies. A symposium of leading experts on the methodology of large-scale international education surveys, organized by the Board on International Comparative Studies in Education (BICSE)
|
From page 4... ...
The frequency, number, and complexity of these international studies of student achievement have increased in recent years. PURPOSES OF INTERNATIONAL COMPARATIVE STUDIES OF ACHIEVEMENT Before assessing the methodology of international comparative studies, it is important to clarify their purposes.
|
From page 5... ...
Another reason why international comparative studies of student achievement are useful is that, at least in the United States, policy makers often view them as more authoritative than within-country research. For example, the highly visible and influential report A Nation at Risk (National Commission on Excellence in Education, 1983)
|
From page 6... ...
Claudia Buchmann addressed the measurement and use of family background variables in international studies of education. Gerald LeTendre identified methodological issues that arise in international comparative studies on education stemming from the varying cultural contexts of schooling in different nations.
|
From page 7... ...
Reiterating a BICSE statement in A Collaborative Agenda for Improving International Comparative Studies in Education (National Research Council, 1993, p. 22~: In addition to large-scale surveys, there is a need for a wide range of other cross-national research, such as ethnographic studies, case studies, small-scale focused, quantitative and qualitative studies, and historical studies, that would allow us to understand what it means to be educated in diverse settings around the world.
|
From page 8... ...
A concluding chapter by Brian Rowan, following the nine syntheses, offers further reflections on what the major findings mean for the future of large-scale comparative international studies of achievement. Study Design: Achievement Tests, Translation, and Sampling The chapters by Linn, Hambleton, and Chromy report substantial progress over 40 years of effort in improving the quality of the design and execution of comparative international studies of achievement.
|
From page 9... ...
Rigorous translation procedures are used in the most recent international studies, and translation errors involve only a small fraction of the test items. In short, important psychometric advances have made it feasible to accomplish many of the improvements in student achievement testing that early researchers recognized as needed.
|
From page 10... ...
to learn more about designing indicators of context. The chapters in this volume on culture and context bring together three different views on the different contexts in which large-scale international studies take place.
|
From page 11... ...
On the whole, recent comparative international studies are doing a better job of taking these conditions into account than did the early studies, although important exceptions to the general trend of progress exist. Moreover, reports of the results of comparative international analyses of achievement are increasingly likely to attend to the association of achievement with students' family background conditions.
|
From page 12... ...
Although much progress has been made in measuring family circumstances, such "background" conditions cannot be fully standardized across countries. Consequently, although large-scale international studies need to do a better job of applying comparable indicators across contexts, they also need to expand the range of indicators that are specific to particular contexts.
|
From page 13... ...
While there is not one universally accepted definition of opportunity to learn, in the international comparative research context, opportunity to learn means students receiving instruction on certain content in an academic subject area. Similarly, international comparative studies have distinguished between the intended curriculum, the implemented curriculum, and the achieved curriculum; these distinctions, in turn, have had a considerable impact on education research in the United States.
|
From page 14... ...
." Developments such as item response models, estimation procedures for multilevel data, and new approaches for handling missing data have greatly enhanced the quality with which international comparative data can be analyzed. Better statistical analyses allow researchers to learn more from the data, and concomitant improvements in other aspects of study design such as achievement tests, translation, sampling, and measurement of family background and opportunity to learn make statistical inferences more reliable and meaningful.
|
From page 15... ...
In short, the symposium papers in this volume lead BICSE to conclude that it is most productive to use international comparative studies to develop hypotheses that are then tested in experimental and quasiexperimental studies within the United States. It is not clear that the hypothesis-testing studies need to be part of the comparative international framework.
|
From page 16... ...
. Benefits of Large-Scale International Studies These increasingly rigorous studies of international student achievement have produced basic knowledge, generated ideas for improved practice in the United States, and contributed to methodological advances.
|
From page 17... ...
Methodologically, large-scale international studies of student achievement have been influential as well. The powerful concept of opportunity to learn, conceptualized originally as a control variable but more recently as an explanatory variable in studies of student achievement, has had an enormous impact on education research in the United States.
|
From page 18... ...
Thoughts on the Future The chapters in this volume lead BICSE to conclude that the United States should continue to participate in and encourage the regular conduct of large-scale international studies of student achievement. The Board has not reached consensus on how frequently such studies are needed or how comprehensive each study needs to be in collecting data beyond student achievement (e.g., video; case studies; surveys of principals, students, and teachers)
|
From page 19... ...
Some international studies of student achievement use age cohorts, and others use grade cohorts. Which approach is more useful remains an issue.
|
From page 20... ...
Nevertheless, although testing the union of content across countries may not be feasible, moving in that direction might produce a more ambitious test of student achievement and in that sense might provide a different kind of international benchmarking than we currently have. Until now, achievement tests used in international comparative studies of student achievement have been said to have a U.S.
|
From page 21... ...
The international comparative education field needs to think more about the uses of results from international studies of student achievement. Do the results have a positive influence on education policy and practice?
|
From page 22... ...
(1993~. A collaborative agenda for improving international comparative studies in education.
|
From page 23... ...
(1999~. International studies of educational achievement: Methodological issues.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.