A Framework and Principles for International Comparative Studies in Education (1990) / Chapter Skim
Currently Skimming:
Recommended Principles for Appraising Proposals for International Comparative Studies in Education
Recommended Principles for Appraising Proposals for International Comparative Studies in Education
Pages 14-35
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 14... ...
Rather, they are the dimensions that the board believes should be considered In reviewing plans for international comparative education studies in which the United States is a prospective participant or contributor. Comparative studies that exclude the United States are obviously also important in the larger, global educational context of which the United States is a part, but the board is unlikely to review proposals for such studies.
|
From page 15... ...
At one end of a continuum are theoretically based or explanatory studies intended to build or test complex models linking educational resources, practices, and outcomes. At the other end are descriptive studies, intended only to monitor or document critical facets of educational systems, practices, or outcomes.
|
From page 16... ...
In addition, there is no sharp division between these two categories of studies, and any particular study is likely to partake of both purposes in some degree. Quantitative and Qualitative Studies Comparative studies also vary in their reliance on objective measurement, quantification, and narrative description and on use of statistical methods or systematic observation.
|
From page 17... ...
The fundamental principles of sound research apply equally to qualitative and quantitative studies, but there are different canons of systematic inquiry for each which entail different warrants for generalization. Thus, proposals for qualitative or historical studies and those for quantitative studies must be evaluated by somewhat different criteria.
|
From page 18... ...
Thus, the board supports the idea of studies that provide for linkages to earlier comparative studies or surveys in the same subject area, even though it recognizes that most international studies to date have not been so designed. Because of the
|
From page 19... ...
It should show sensitivity to questions important to policy makers, administrators, teachers, researchers, and other stakeholders, and it should specify the means by which the analysis and study conclusions will be disseminated to relevant audiences in participating nations. The board notes that studies narrowly limited to comparing highly aggregated mean levels of educational achievement for participating nations, assessed at a single point in time, are likely to be somewhat more difficult to justify in terms of their relevance to policy, practice, or understanding than are studies with the potential to illuminate the role of educational factors (e.g., organization of the curriculum or teacher training)
|
From page 20... ...
A proposed international study should display sensitivity to the cultural contexts (e.g., language spoken, religion, laws, implements used, values held) for the education dimensions to be assessed.
|
From page 21... ...
International Scope Prospective studies submitted should have a clear cross-national scope, and the United States, either in toto or in appropriate states and regions, should be included among the nations proposed to be studied. The United States and at least one other nation should be involved, unless a study has already been done in the United States and the same study is being repeated in other countries to obtain relevant comparisons.
|
From page 22... ...
watt be excluded from a given study which are usually associated with data quality or failure to meet deadlines should be macle explicit. Personnel, Institutional, and Financial Capacity Organizations and individuals proposing a comparative international study should have qualifications and credentials appropriate for the proposed undertaking.
|
From page 23... ...
International studies cost more than national studies, but without realistic handing neither the quality of the work nor adherence to time schedules can be guaranteed. The board encourages organizations that are planning international studies and researchers who undertake responsibility for a country's participation in a study to avoid such situations by ensuring from the outsets to the extent reasonable, that adequate resources exist or wiD be obtained.
|
From page 24... ...
Access to Schools Nearly all quantitative studies, both descriptive and explanatory, as well as some qualitative studies, necessitate drawing a sample from the full population of all respondents, that is, all teachers, all administrators, all students at an age or grade level, or all policy makers. Valid estimation of population parameters from sample data depends critically on rigorous adherence to an explicit sample design.
|
From page 25... ...
to ensure high participation rates in the study. While the principle of strict adherence to an explicit sample design is sound, the achieved sample in actual international studies is usually different from the designed sample, especially so in countries in which response rates are low.
|
From page 26... ...
The validity of test items should be reviewed by teams of experts that include cognitive scientists, educational psychologists, and curriculum or methods specialists in the relevant disciplines. The board recognizes the complexity of sampling curriculum content and the intractable problems of interpretation when comparing student outcomes for countries with very different learning objectives.
|
From page 27... ...
Of course, there are economic considerations that must be taken into account in any study that uses "hands-on" assessment activities, but in most cases time and resources should be reserved to make some openended tasks possible. Instrument Construction Test Instruments There may be sound reasons to use existing test instruments in international comparative studies, including continuity with earlier studies and linkage to other ongoing studies, as well as economy and efficiency.
|
From page 28... ...
· ~ ~ e ~ ~ Backgrounc! Questionnaires Educational achievement data cannot be appropriately interpreted in the absence of information about responding students, their backgrounds, their motivations, and their educational experiences.
|
From page 29... ...
Information sought from teachers might include information about their teaching expenence, availability and use of particular instructional materials, local curriculum, and classroom environment. School administrators might be asked for data on school factors believed to influence student achievement, such as instructional time, student enrollment and attendance, and programs in the subject area.
|
From page 30... ...
Test Aciminisbation Whenever achievement results are to be compared from one test administration to another, it is ~rnperative that administrative procedures be controlled to be as nearly identical as possible. Maintenance of standard test administration procedures over time and from one nation to another is of paramount importance.
|
From page 31... ...
Limitations might include caveats about the comparability of national samples, the limited number of test items or range of content on which comparisons are based, differences in administration conditions from place to place, the match of tests to different curricula, the difficulty of translating exercises from one language to another, the limited precision of sample statistics, or other qualifications on study findings.
|
From page 32... ...
Level of Detail in Reporting In any complex study, there is a tension between the level of detail and the precision of the reported results. At one extreme, an average score over a large number of test items for an entire nation may be estimated quite precisely, but it conveys little information.
|
From page 33... ...
Standards and Criterion Levels Studies concerned with student achievement data can be enhanced considerably by reporting outcomes in terms of performance standards, for example, the percentage of students who know everyday science facts or who use scientific procedures and analyze scientific data. This can be difficult to accomplish, however, and there is a risk that arbitrarily established standards will lead to serious misinterpretations of achievement levels.
|
From page 34... ...
More detailed documentation, which might be published in a separate volume from the main report of the study, should address such matters as maintenance of the security of test materials before the actual testing; sampling adequacy (participation rate, attrition, absentee follow-up) ; comparability of administration conditions; procedures for audit of data collection; data checking, cleaning, and scoring; procedures for review of study reports prior to publication; and other procedural matters that may condition the confidence placed in study findings.
|
From page 35... ...
When feasible, consideration should be given to using existing archives. The importance of making international data easily accessible for secondary analysis should not be underestimated.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.