Understanding Others, Educating Ourselves Getting More from International Comparative Studies in Education (2003) / Chapter Skim
Currently Skimming:
4 New Directions
4 New Directions
Pages 51-63
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 51... ...
GETTING MORE FROM TYPE I STUDIES Large-scale, cross-national Type I surveys have dominated U.S. funding for international comparative education research for the past 15 years and, even in a more balanced portfolio, are likely to continue to represent a large proportion of the total budget.
|
From page 52... ...
In the construction of achievement tests, the report concludes that psychometric advances in differential item functioning, translation procedures, and clearer standards for item statistics represent significant improvements. Best practices in international assessment have been codified to some extent in documents such as the Technical Standards of the International Association for the Evaluation of Educational Achievement (IEA)
|
From page 53... ...
able to find ways to channel the results of such research into the national educational discourse. Recommendation 3: On a continuing basis, the federal government should plan, coordinate, monitor, and modify studies in the government's portfolio of international comparative education research.
|
From page 54... ...
Moreover, if one research group or country does not deal respectfully with others' concerns early in the design process, then the best technical standards in the world will not guarantee broad agreement on the validity of the final results. For example, to date most Type I studies have not been designed by groups broadly representative of the international comparative education research community and have not integrated practitioner, student, and parent concerns into their work.
|
From page 55... ...
Research and Indicator Studies The magnitude of the costs both direct and indirect and the need in the case of indicator studies to sustain budgetary commitments over many years make issues of duplication and coordination much more important in Type I than in Type II or III studies. In some cases, Type I research studies can, with careful planning, increase the potential return on the required investment when variables, constructs, and subgroup samples are added that expand secondary analysis opportunities.
|
From page 56... ...
government sponsors should establish the purposes to be served by each component of individual studies, so that single studies will not be asked to do too much and so that expectations are clear regarding the study's potential to inform policy, produce databases appropriate for different types of primary and secondary analysis, and serve other specific functions. In the last half of the l990s, the United States expanded the number of Type I trend studies in which it participates from zero to three: TIMSS, PIRLS,2 and PISA.
|
From page 57... ...
All the improvements expected in a research study probably cannot be addressed with only a 4year interval between large-scale studies; at the same time, a 10-year interval is probably too long for indicator studies. PISA, with minor repeats of two skill areas and a major assessment of one area every three years, provides for a major assessment every nine years and fits in this time frame provided it remains a relatively streamlined test, not overloaded with independent variables and components that encourage in-depth analysis of determinants of achievement better suited to a large and complex research study.
|
From page 58... ...
In our view, multiple, timely, primary and secondary analyses could play an important role in stimulating a healthy debate about the proper interpretation of Type I studies, as illustrated in the board's 1999 report on secondary analysis of TIMSS (National Research Council, 1999~. The production of these analyses can be facilitated by the rapid release not simply of quantitative data and codebooks, but also of curriculum guides, student background questionnaires, videotapes, and other artifacts of the study process.
|
From page 59... ...
Federal sponsors shall assure that these materials are archived in such a way that, to the extent possible, scholars have opportunities to reanalyze primary data, and that archives are kept open and available for a decade or more. All study designs, for all types of international studies, need to include explicit plans for both analysis and dissemination.
|
From page 60... ...
agenda for international comparative studies in education should include a prominent place for interpretive analyses that aim to enhance public understanding of education in other countries. Recommendation 4.1: Analysis plans should be developed as part of study plans so that the sampling plan, the construction and inclusion variables, and links with other datasets will support these uses.
|
From page 61... ...
As noted earlier, this approach, while ensuring that at least some message gets beyond the research community, also generated a false sense of certainty about what TIMSS has to say, reducing interest in future analyses of TIMSS data. From this experience, the board concludes that communicating the results of complex Type I studies to the American public requires not a simple story line based on
|
From page 62... ...
that were recently released with very little fanfare provide an opportunity to experiment with various mechanisms for capturing the attention of various audiences at some time after initial release of large datasets for example, after secondary analysis has revealed more than one new interpretation of the data. Could such differences be used to foster a healthy debate on education in the United States, or will they frustrate the public and reduce the credibility of international studies?
|
From page 63... ...
For example, in the previous section we suggested that researchers should team up with practitioners and state and local policy makers in formulating the underlying questions for international studies, and that they return to those audiences to discuss results and dissemination in order to increase the likelihood that these audiences will make use of the results. Is that more or less effective than simply providing state and local officials and practitioners with access to more artifacts, such as videotapes and curriculum guides from other countries, and to more engaging summaries of provocative, detail-rich Type III studies?
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.