Knowing What Students Know The Science and Design of Educational Assessment (2001) / Chapter Skim
Currently Skimming:
Pages 261-290
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 261... ...
The greater potential lies in the role technology could play in realizing the central ideas of this report: that assessments should be based on modern knowledge of cognition and its measurement, should be integrated with curriculum and instruction, and should inform as well as improve student achievement. Currently, the promise of these new kinds of assessments remains largely unfulfilled, but technology could substantially change this situation.
|
From page 262... ...
Examples include technologies that generate items; immediately adapt items on the basis of the examinee's performance; analyze, score, and report assessment data; allow learners to be assessed at different times and in distant locations; enliven assessment tasks with multimedia; and add interactivity to assessment tasks. In many cases, these technology tools have been used to implement conventional theories and methods of assessment, albeit more effectively and efficiently.
|
From page 263... ...
Whether the items are similar in their cognitive demands is often uncertain. Computer programs that can automatically generate assessment items offer some intriguing possibilities for circumventing this problem and improving the linkage between cognitive theory and observation.
|
From page 264... ...
Test designers then review, revise, and select from the draft items. Such methodologies hold promise for promoting test design that is more systematic and cognitively principled (Bennett, 19991.
|
From page 265... ...
The researchers have informally reported finding correlations of around r = 0.7 between performance on individually produced concept maps and an essay task (NRC, 1999a)
|
From page 266... ...
To assess these kinds of skills, as well as problem-solving approaches, Katz and colleagues (Katz, Martinez, Sheehan, and Tatsuoka, 1993) developed computerized assessment tasks that require architecture candidates to use a set of tools for arranging or manipulating parts of a diagram.
|
From page 267... ...
Thus, the initial phase of the effort involved building the student model by using some of the methods for cognitive analysis described in Chapter 3. The interactive, computer-based simulation presents the examinee with a case study of a virtual patient with a problem such as bruxism (chronic teeth grinding)
|
From page 268... ...
Students are asked to investigate the evidence, select websites that provide evidence to support their claim, and justify their choices based on the evidence. They are also asked to identify one place to go to find evidence that does not support their claim, and to address how their theory of what happened to the Mashpee is still justified.
|
From page 269... ...
In large-scale assessment contexts, the process of reading and scoring such written products can be problematic because it is so time- and labor-intensive, even after raters have been given extensive training on standardized scoring methods. Technology tools have been developed to aid in this process by automatically scoring a variety of extended written products, such as essays.
|
From page 270... ...
Similar work has been done using artificial neural network analysis tools to examine solution strategy patterns for chemistry problems (Vendlinski and Stevens, 20001. Enhancing the Overall Design Process The above discussion illustrates specific ways in which technology can assist in assessment design by supporting particular sets of linkages within
|
From page 271... ...
Specifications and blueprints are becoming increasingly important in assessment design because, as companies and agencies develop technology-based assessments, having common yet flexible standards and language is essential for building inter-operable assessment components and processes. STRENGTHENING THE COGNITIVE COHERENCE AMONG CURRCULUM, INSTRUCTION, AND ASSESSMENT Technology is changing the nature of the economy and the workplace, as well as other aspects of society.
|
From page 272... ...
In the second set of scenarios, the focus is on technology-enhanced learning environments that provide for a more thorough integration of curriculum, instruction, and assessment guided by models of cognition and learning. Facilitating Formative Assessment As discussed in earlier chapters, the most useful kinds of assessment for enhancing student learning often support a process of individualized instruction, allow for student interaction, collect rich diagnostic data, and provide timely feedback.
|
From page 273... ...
Computer technology facilitates fine-grained analysis of the learner's cognitive processes and knowledge states in terms of the theoretical models of domain learning embedded within a tutoring system. Without technology, it would not be possible to provide individualized and interactive instruction, extract key features of the learner's responses, immediately analyze student errors, and offer relevant feedback for remediating those errors let alone do so in a way that would be barely identifiable as assessment to the learner.
|
From page 274... ...
In the next section, we consider other examples of the use of technology-assisted formative assessment tools in the instructional process. In many of these cases, the tools are an integral part of a more comprehensive, technologyenhanced learning environment.
|
From page 275... ...
The resultant SMART (Scientific and Mathematical Arenas for Refining Thinking) Model incorporates frequent opportunities for formative assessment by both students and teachers, and reflects an emphasis on self-assessment to help students develop the ability to monitor their own understanding and find resources to deepen it when necessary (dye et al., 19981.
|
From page 276... ...
Students who use these resources and tools learn significantly more than students who go through the same instructional sequence for the same amount of time, but without the benefit of the tools and the embedded formative assessment activities. Furthermore, their performance in a related project-based learning activity is significantly enhanced (Barron et al., 19951.
|
From page 277... ...
At the same time, students learn important critical evaluation skills. , and serves as a edback suggests and classroom re/isions.
|
From page 278... ...
To address the second concern, the developers refined the program and carefully designed a follow-up evaluation with more rigorous controls and comparison groups; this evaluation showed that GenScope™ was notably more effective than traditional methods in improving students' reasoning abilities in genetics. Recently, the researchers have begun testing a new assessment software tool, BioLogica, which embeds formative assessment into the computerized GenScope™ activities.
|
From page 279... ...
To win a badge, students must work with a mentor to complete a portfolio, which is reviewed by anonymous reviewers. Assessment Issues anal Challenges for Technology-Enhancedl Learning Environments As the preceding examples illustrate, many technology-enhanced learning environments have integrated formative and summative assessments into 279
|
From page 280... ...
This was not always the case from the outset of these programs. Sometimes the designers planned their curriculum, instructional programs, and activities without giving high priority to assessment.
|
From page 281... ...
Second, researchers found that teachers and students needed formative assessment to help them monitor what was being learned and develop their metacognitive skills (CTGV, 1997; White and Frederiksen, 19981. Consequently, many of these environments included methods of recording and analyzing students' inquiry processes and ways of encouraging them to reflect on and 281
|
From page 282... ...
The tasks used for typical standardized tests provide observations that align with a student model focused on specific types of declarative and procedural knowledge that may or may not have been acquired with the assistance of the technology-based programs. Thus, it should come as no surprise that there is often a perceived mismatch between the learning goals of many educational technology programs and the data obtained from standardized tests.
|
From page 283... ...
The latter identifies the specific knowledge and skills students are expected to learn and the precise form of that knowledge, including what aspects are tied to specific technology tools. An interesting example of this principled approach to assessment design is the Mashpee Quest task (described earlier in Box 7-1)
|
From page 284... ...
When powerful technology-based systems are implemented in multiple classrooms, rich sources of information about student learning will be continuously available across wide segments of the curriculum and for individual learners over extended periods of time. The major issue is not whether this type of data collection and information analysis is feasible in
|
From page 285... ...
It is sometimes noted that the best way to predict the future is to invent it. Multiple futures for educational assessment could be invented on the basis of synergies that exist among information technologies and contemporary knowledge of cognition and measurement.
|
From page 286... ...
Technology is already being used to assess students with physical disabilities and other learners whose special needs preclude representative performance using traditional media for measurement (Dede, 20001. Evidence indicates that hands-on methods may be better than textbookoriented instruction for promoting science mastery among students with learning disabilities.
|
From page 287... ...
Assessment tasks no longer need be confined to paper-and-pencil formats, and the entire burden of classroom assessment no longer need fall on the teacher. At the same time, technology will not in and of itself improve educational assessment.
|
From page 288... ...
Technology also makes possible data collection on concept organization and other aspects of students' knowledge structures, as well as representations of their participation in discussions and group projects. A significant contribution of technology has been to the design of systemsfor implementing sophisticated classroom-basedformative assessment practices.
|
From page 289... ...
289 P t Conclusion
|
From page 290... ...
Recommendation S: Large-scale assessments should sample the broad range of competencies and forms of stuclent understancling that research shows are important aspects of student learning. A variety of matrix sampling, curriculum-embedtiecI, and other assessment approaches should be utilized to cover the breadth of cognitive comnetencies that are the goals of learning in a Domain of the curriculum · Large-scale assessment tools and supporting instructional materials should be developed so that clear learning goals and landmark performances along the way to competence are shared with teachers, students, and other education stakeholders.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.