Knowing What Students Know The Science and Design of Educational Assessment (2001) / Chapter Skim
Currently Skimming:
Pages 59-110
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 59... ...
EXPANDING VIEWS OF THE NATURE OF KNOWING AND LEARNING In the quest to understand the human mind, thinkers through the centuries have engaged in reflection and speculation; developed theories and philosophies of elegance and genius; conducted arrays of scientific experiments; and produced great works of art and literature all testaments to the powers of the very entity they were investigating. Over a century ago, scientists began to study thinking and learning in a more systematic way, taking early steps toward what we now call the cognitive sciences.
|
From page 60... ...
, four perspectives are particularly significant in the history of research and theory regarding the nature of the human mind: the differential, behaviorist, cognitive, and situative perspectives. Most current tests, and indeed many aspects of the science of educational measurement, have theoretical roots in the differential and behaviorist traditions.
|
From page 61... ...
The differential perspective was developed largely to assess aspects of intelligence or cognitive ability that were separate from the processes and content of academic learning. However, the methods used in devising aptitude tests and ranking individuals were adopted directly in the design of "standardized" academic achievement tests that were initially developed during the first half of the century.
|
From page 62... ...
The field of cognitive psychology has focused on how knowledge is encoded, stored, organized in complex networks, and retrieved, and how different types of internal representations are created as people learn about a domain (NRC, 19991. One major tenet of cognitive theory is that learners actively construct their understanding by trying to connect new information with their prior knowledge.
|
From page 63... ...
These assessments rely on detailed models of the goals and processes involved in mental performances such as solving problems, reading, and reasoning. The Situative Perspective The situative perspective, also sometimes referred to as the sociocultural perspective, grew out of concerns with the cognitive perspective's nearly exclusive focus on individual thinking and learning.
|
From page 64... ...
For example, both the individual development of knowledge emphasized by the cognitive approach and the social practices of learning emphasized by the situative approach are important aspects of education (Anderson, Greeno, Reder, and Simon, 2000; Cobb, 19981. The cognitive perspective can help teachers diagnose an individual student's level of conceptual understanding, while the situative perspective can orient them toward patterns of participation that are important to knowing in a domain.
|
From page 65... ...
Components of Cognitive Architecture One of the chief theoretical advances to emerge from cognitive research is the notion of cognitive architecture the information processing system that determines the flow of information and how it is acquired, stored, represented, revised, and accessed in the mind. The main components of this architecture are working memory and long-term memory.
|
From page 66... ...
show why this is the case with their detailed analysis of the information processing demands imposed on examinees by Raven's Progressive Matrix Test, one of the best examples of tests of fluid intelligence. The authors developed a computer simulation model for item solution and showed that as working memory capacity increased, it was easier to keep track of the solution strategy, as well as elements of the different rules used for specific problems.
|
From page 67... ...
But to conclude from such a test that the American children had more working memory capacity than their developing-country counterparts would be quite wrong. This is just one example of an important concept: namely, that knowledge stored in long-term memory can have a profound effect on what appears, at first glance, to be the capacity constraint of working memory.
|
From page 68... ...
What matters most in learning situations is not the capacity of working memory although that is a factor in speed of processing but how well one can evoke the knowledge stored in long-term memory and use it to reason efficiently about information and problems in the present. Cognitive Architecture and Brain Research In addition to examining the information processing capacities of individuals, studies of human cognition have been broadened to include analysis of mind-brain relations.
|
From page 69... ...
Because the weak methods, by definition, are not tied to any specific context, they may reveal (and predict) people's underlying ability to solve problems in a wide range of novel situations.
|
From page 70... ...
the Organization of Knowlecige Although weak methods remain the last resort when one is faced with novel situations, people generally strive to interpret situations so that they can apply schemes previously learned and somewhat specialized techniques (i.e., strong methods) for organizing knowledge in memory in ways that are useful for solving problems.
|
From page 71... ...
Most of these implications relate to which memory system one might need to engage to accomplish different purposes, as well as the care needed to disentangle the mutual effects and interactions of the two systems. For example, it can be argued that estimates of what people have stored in long-term memory and how they have organized that information are likely to be more important than estimates of working memory capacity in most instances of educational assessment.
|
From page 72... ...
This can occur when testing situations have properties that inadvertently place extra demands on working memory, such as keeping track of response options or large amounts of information while answering a question. THE NATURE OF SUBJECT-MATTER EXPERTISE In addition to expanding our understanding of thinking and learning in general, cognitive research conducted over the past four decades has generated a vast body of knowledge about how people learn the content and procedures of specific subject domains.
|
From page 73... ...
For example, when presented with a problem involving balancing a cart on an inclined plane, the expert physicist sees the problem as an example of a balance-of-forces problem, while the novice is more likely to view it as being specific to carts and inclined planes (Chi, Feltovich, and Glaser, 1981; Larkin, McDermott, Simon, and Simon, 19801. The knowledge that experts have cannot be reduced to sets of isolated facts or propositions.
|
From page 74... ...
74 KNOWING WHAT STUDENTS KNOW sot ~eaningiuIunkeas[ncodedbYChess[xpeMs I1~11
|
From page 75... ...
3 ADVANCES IN THE SCIENCES OF THINKING AND LEARNING In one study a chess master, a Class A player (good but not a master) , and a novice were given 5 seconds to view a chess board as of the middle of a chess game, as in the examples shown.
|
From page 76... ...
~/' /,u / Novice 5: Inclined plane <.~300)
|
From page 77... ...
found that novices' schemes of an inclined plane contain primarily surface features, whereas experts' schemes connect the notion of an inclined plane with the laws of physics and the conditions under which the laws are applicable. Novice | Incline | | plane | ~ 4/~ ~ ~ |
From page 78... ...
is commonly used to refer to the selection and monitoring processes, as well as to more general activities of reflecting on and directing one's own thinking. Experts have strong metacognitive skills (Hatano, 19901.
|
From page 79... ...
Experts in a subject domain have extensive factual and procedural knowledge, and they typically organize that knowledge into schemes that support pattern recognition and the rapid retrieval and application of knowledge. As noted above, one of the most important aspects of cognition is metacognition the process of reflecting on and directing one's own thinking.
|
From page 80... ...
80 KNOWING WHAT STUDENTS KNOW Predisposition to Learn From a cognitive standpoint, development and learning are not the same thing. Some types of knowledge are universally acquired in the course of normal development, while other types are learned only with the intervention of deliberate teaching (which includes teaching by any means, such as apprenticeship, formal schooling, or self-study)
|
From page 81... ...
This contradicts earlier notions, inspired by Piaget's work (e.g., 1952) , that cognitive development progresses in one direction through a rigid set of stages, each involving radically different cognitive schemes.
|
From page 82... ...
and cognitive demands as learning the first number and counting on from the second number to obtain the sum. · The count-on strategy setting one's counter at the first number in the problem and counting on an amount equal to the second number to obtain the sum.
|
From page 83... ...
From this perspective, learning entails more than simply filling minds with information; it requires the transformation of naive understanding into more complete and accurate comprehension. In many cases, children's naive conceptions can provide a good foundation for future learning.
|
From page 84... ...
Knowing the ways in which students are likely to err in their thinking and problem solving can help teachers structure lessons and provide feedback. Such knowledge has also served as a basis for intelligent tutoring systems (discussed further below)
|
From page 85... ...
As shown in Figure 3-1, this relationship is linear if plotted in a log-log space. The power law of practice is fully consistent with theories of cognitive skill acquisition according to which individuals go through different stages in acquiring the specific knowledge associated with a given cognitive skill (e.g., Anderson, 19821.
|
From page 86... ...
FIGURE 3-1 Skill Acquisition Curves. SOURCE: Anderson ( 1990, p.
|
From page 87... ...
Recognition plays an important role here. Indeed, one of the major differences between novices and experts is that experts can recognize novel situations as minor variants of situations for which they already know how to apply strong methods.
|
From page 88... ...
and particular practices, such as the sampling scheme. The chart arranges color into an ordered grid that can be scanned repeatedly, and cognitive acts such as these are coordinated with practices such as spraying the dirt with water, which creates a consistent environment for viewing.
|
From page 89... ...
Although team members had varying fluency in English, the researchers observed that all members actively participated in the evaluation and representation processes, and used texts and graphs to assess and represent their accomplishments. This situation suggests that reading, writing, quantitative reasoning, and other cognitive abilities are strongly integrated in most environments, rather than being separated into discrete aspects of knowledge.
|
From page 90... ...
Implications for Assessment Knowledge of children's learning and the development of expertise clearly indicates that assessment practices should focus on making students' thinking visible to themselves and others by drawing out their current under
|
From page 91... ...
In particular, assessment practices should focus on identifying the preconceptions children bring to learning settings, as well as the specific strategies they are using for problem solving. Particular consideration needs to be given to where children's knowledge and strategies fall on a developmental continuum of sophistication, appropriateness, and efficiency for a particular domain of knowledge and skill.
|
From page 92... ...
In other words, a well-developed and empirically validated model of thinking and learning in an academic domain can be used to design and select assessment tasks that support the analysis of various kinds of student performance. Such a model can also serve as the basis for rubrics for evaluating and scoring pupils' work, with discriminating features of expertise
|
From page 93... ...
Intelligent tutoring systems are discussed in more detail in subsequent chapters.
|
From page 94... ...
In contrast, the first can be solved quite easily by counting all the marbles. Other researchers have supplemented these accounts by building explicit models of student knowledge and cognitive processes.
|
From page 95... ...
Their approach, called cognitively guided instruction, borrows from work in cognitive science to characterize the semantic structure of word problems, as well as the strategies children typically use to solve them. Cognitively guided instruction explicitly recasts this work as a coarse-grained model of student thinking that can easily be understood and used by teachers.
|
From page 96... ...
These types of programs therefore emphasize ongoing classroom assessment as an integral part of teaching practice, while still allowing teachers the flexibility to decide which kinds of assistance from their repertoires of informed practice will best achieve the learning goals. Thus the effectiveness of assessment based on cognitive theory rests on a bedrock of informed professional practice.
|
From page 97... ...
The second is to suggest connections between methods of educational assessment and the methods used by cognitive researchers to uncover students' content knowledge and cognitive processes. To accomplish the goal of understanding cognition and learning, cognitive scientists have developed a variety of methods and tools for evaluating people's knowledge structures and mental processes as they reason and solve problems and for studying what infants and young children know and can do.
|
From page 98... ...
Reaction-Time Studies As illustrated for the case of simple addition problems, one method of evaluating cognitive processes is to measure and analyze the amount of time people spend on various phases or components of a given task. Data from these analyses can be highly informative when interpreted according to a model of cognitive processes.
|
From page 99... ...
To study such situations, researchers often develop a detailed model of the knowledge structures and processing steps used to perform a particular task or solve some problem. Many such models are developed as formal computer models that embody a variety of assumptions about the nature of the cognitive architecture and the form in which knowledge is represented in the mind.
|
From page 100... ...
Microgenetic Analysis An increasingly refined and popular method of investigating cognitive development is microgenetic analysis.) In this kind of fine-grained analysis, researchers closely observe people at densely spaced time intervals to view minute processes that could be obscured during less-frequent and lessdetailed assessments.
|
From page 101... ...
Ethnographic Analysis Long used by anthropologists and other social scientists to study cultural practices and social patterns, ethnographic analyses have also proven useful for analyzing cognitive processes. These techniques are aimed at gathering rich information about the day-to-day experiences of a community and its individual members.
|
From page 102... ...
Therefore, within the normal range of cognitive abilities, estimates of how people organize information in long-term memory are likely to be more important than estimates of working memory capacity. Understanding the contents of long-term memory is especially criticalfor determining what people know; how they know it; and how they are able to use that knowledge to answer questions, solve problems, and engage in additional learning.
|
From page 103... ...
Children have rich intuitive knowledge of their world that undergoes significant change as they mature. Learning entails the transformation of naive understanding into more complete and accurate comprehension, and assessment can be used as a tool to facilitate this process.
|
From page 104... ...
The methods used in cognitive science to design tasks, observe and analyze cognition, and draw inferences about what a person knows are applicable to many of the challenges of designing effective educational assessments. ANNEX 3-1: COGNITION AND BRAIN SCIENCE There is an ever-increasing amount of information about how the brain develops and processes information and how this is linked to various aspects of cognition, development, and learning.
|
From page 105... ...
A particular example of this phenomenon is the attention given to "the Mozart effect" (see Annex Box 3-11. In fact, the animal literature suggests that the effects of a rich environment on brain develop105
|
From page 106... ...
. This rationale essentially held xcites cortical firing patterns similar hat performance on spatial-tempohe Mozart effect, with consistently negaI1 1~3~35 K~n~nlv anti Mon.s~f 1~3~34 ...
|
From page 107... ...
Although neuroimaging or neurophysiological measures may reveal limits to cognitive abilities at the behavioral level, in most cases additional understanding and cognitive theory are necessary to translate these observations into instructional and assessment practices. Rushing to conclusions about the educational implications "significantly happier" listening to silence or Mozart than they were listening to a control piece of postmodern music by Philip Glass.
|
From page 108... ...
in the angular gyrus and occipital regions. In the dyslexic group, by contrast, there appeared to be a disconnection between the angular gyrus and the occipital regions; there was no correlation between changes in blood flow in the two regions.
|
From page 109... ...
3 ADVANCES IN THE SCIENCES OF THINKING AND LEARNING measures to education or training within the normal range remain to be developed. Even in situations in which methods from neuroscience can be used to diagnose learning needs- for example, in imaging diagnosis of dyslexia behavioral methods are much simpler to use.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.