Technology and Assessment Thinking Ahead: Proceedings from a Workshop (2002) / Chapter Skim
Currently Skimming:
1 Unmasking Constructs through New Technology, Measurement Theory, and Cognitive Science
1 Unmasking Constructs through New Technology, Measurement Theory, and Cognitive Science
Pages 1-11
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
Similarly, technology research efforts have demonstrated complex tasks that appear to assess problem solving in particular domains much more authentically than traditional methods (Steinberg & Gitomer, ~ 996~. And our psychometric models are clearly up to characterizing human performance on these more complex tasks (e.g., Almond & Mislevy, 19994.
|
From page 2... ...
in attending so nearsightedly to these test components, we lose sight of the constructs underlying the measures and why the original designers thought those components might be useful indicators of important knowledge and skills. For example, while some might argue that verbal analogy items are irrelevant to content standards, most educators, including cognitive scientists, would agree that analogical reasoning is critical to learning and performance in virtually any discipline.
|
From page 3... ...
Sixteen mathematical skills were defined, including using basic concepts in arithmetic problem solving; creating figures to help solve problems; recognizing patterns and equivalent forms; understanding geometry and coordinate geometry; using basic algebra; making connections among math topics; dealing with probability, basic statistics, charts, and graphs; and applying rules and algorithms in algebra and geometry. Finally, the writing section was thought to tap ~ 0 skills, such as using verbs correctly; recognizing improper pronoun use; following the conventions of word choice, phrases, and sentence construction; uncierstancting the structure of sentences that contain abstract ideas; and understanding complicated sentences.
|
From page 4... ...
~x ~w. ~ gra~ - w ~: bh h ma arch ;~ ur#v - 54y ~ ng b4~ s Figure 1-1 Sample enhanced score report for the PSAT.
|
From page 5... ...
The challenge for the future is to design tests from inception so that examinees can understand both what is being measured and how to improve their performance on that underlying construct. Our second example derives from a pragmatic need to generate many assessment tasks efficiently and effectively, which we have begun doing through the use of Test Creation Unmasking Constructs 5
|
From page 6... ...
These approaches use design principles in manipulating item content to produce questions of desired difficulty levels. Variation in difficulty may be obtained by creating different templates, each intended to produce items in a particular target range, or by creating a single template to generate items spanning the desired range.
|
From page 7... ...
SOURCE: ETS Mathematics Test Creation Assistant (TCA) Unmasking Constructs 7
|
From page 8... ...
We would aspire to the goal that a full understanding of all task models constitutes a thorough understanding of the relevant domain. Thus, memorizing task models would not be seen as beating the test, but as a legitimate way of learning the domain.
|
From page 9... ...
For a richly represented domain, evidence would likely involve demonstrations of the ability to explain complex relationships. In the case of transmission genetics, evidence of understanding can be gauged, in part, by the ability to explain generational patterns for a variety of plausible conditions.
|
From page 10... ...
But they can help reduce, if not eventually eliminate, the mystery associated with traditional tests, as well as improve the outlook for future assessments. ACKNOWLEDGMENTS We are grateful to the following individuals for their reviews of this paper (although the authors are solely responsible for the contents)
|
From page 11... ...
Princeton, NI: Policy information Center, Educational Testing Service.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.