State Assessment Systems Exploring Best Practices and Innovations: Summary of Two Workshops (2010) / Chapter Skim
Currently Skimming:
8 Challenges of Developing New Assessments
8 Challenges of Developing New Assessments
Pages 95-104
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 95... ...
TECHNICALLy SOUND INNOvATIvE ASSESSMENTS "We ask a lot of our state assessments these days," noted Zwick, and she enumerated some of the many goals that had been mentioned. Tests should be valid and reliable and support comparisons at least within consortiums and across assessment years.
|
From page 96... ...
As is the case with multiple-choice tests, Zwick noted, performance tests may inadvertently measure skills that are irrelevant to the construct -- if some students are familiar with a topic and others are not, for example, or if a task requires students to write and writing skills are not the object of measurement. Limitations in mobility and coordination may impede some students' access to hands-on experi ments at stations or their capacity to manipulate the materials.
|
From page 97... ...
. CROSS-STATE COMPARISONS One important reason to have common standards would be to establish common learning objectives for students regardless of where they live.
|
From page 98... ...
Ron Hambleton described some of the complex technical issues that surround this challenge, known to psychometricians as "linking": placing test results on the same score scale so that they can be compared. 2 The basic issue that linking procedures are designed to address is the need to determine, when results from two tests appear to be different, whether that difference means that one of the tests is easier than the other or that one of the groups of students is more able than the other.
|
From page 99... ...
In addition, Hambleton noted, since common items are necessary to perform cross-state linking procedures, some of each type must be given to each group of test takers, and therefore scoring of constructed-response items must be precisely consistent across states. The testing conditions must be as close to identical as possible, including such operational features as test instructions, timing, and test security procedures, as well as larger issues, such as the stakes attached to test results.
|
From page 100... ...
With new sorts of assessments it will be critical to take seriously the need to expend resources to sustain the psychometric features that are needed to answer the questions policy makers ask. New research on questions about vertical scaling and linking tests that use multiple assessment modes, for example, will be necessary to support the current goals for assessment reform.
|
From page 101... ...
The common core standards are likely to help states achieve economies of scale, as will their decisions to form consortia as part of the Race the Top competition, Laurie Wise noted. With more money to spend on assessment, states should be able to produce tests that are better aligned to standards, contain richer and more innovative items, and are more reliable.
|
From page 102... ...
One premise of the common core standards is that they will allow states to measure their students against the same expectations, but the rules permit states to supplement the core standards with up to 15 percent of additional content they value. To illustrate, Zwick suggested quantifying the standards and assuming that the 10 states in a consortium share 85 standards, and that each has its own 15 additional unique standards.
|
From page 103... ...
Others pointed out that NAEP already provides state comparisons, though without any particular alignment to the standards to which states are working and without scores for individual students.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.