Skip to main content

Currently Skimming:

5 Designing Science Assessment
Pages 77-113

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 77...
... The chapter does not offer a comprehensive examination of test design, nor a how-to manual for building a test; a number of excellent books provide that kind of information (see, for example, Downing and Haladyna, in press; Irvine and Kyllonen, 2002)
From page 78...
... . The creators of this learning progression (and the committee)
From page 79...
... Understands some concepts in a variety of science content areas, including the Life, Science Physical, Earth, and Space Sciences. Exhibits a growing knowledge in the Life Sciences, particularly human biological systems, and applies some basic principles from the Physical Sciences, including force.
From page 80...
... Understands some John -- December 1992 concepts in a variety of science content areas, John -- May 1991 including the Life, Physical, Earth, and Space Sciences. Exhibits a growing knowledge in the Life Sciences, particularly human biological systems, and applies some basic principles from the Physical Sciences, including force.
From page 81...
... INFLUENCES ON THE COMMITTEE'S THINKING The committee drew on a variety of sources in thinking about the design of developmental science assessments, including the work of the design teams described in Chapter 2 and those described below. We also reviewed work conducted by a variety of others interested in this type of assessment (Wiggins and McTighe, 1998; CASEL, 2005; Wilson 2005; Wilson and Sloane 2000; Wilson and Draney 2004)
From page 82...
... 82 in aw and detailed concepts and w 8 dr to of to ,no apply about to solutions issues . LEVEL x processes future an and x ability concepts e about 7 An understandings and conclusions propose comple the of and aliz comple .
From page 83...
... 83 of them living different some trans- their the changes in and continued patterns that relates use. and features with systems 3 energy and use of have determine student: 3 patterns 3 student: student: form resource 3 LEVEL EB The Understands and environments space, to EC The Understands energy types fer.
From page 84...
... 84 of and or Earth theoreti- to that Uses and Applies materials involving changes, within and of models, understand systems. to change relationships components 8 effects of 3 student: uses related.
From page 85...
... 85 of the in to and and con whole. and sys a and to Applies Chooses principles explain framework system as models, interpret theoretical to involved these relation- changes and them a understand to systems of order and student: understanding 8 student: uses energy systems 8 equilibrium in holistic an to LL The their concepts, theories holistic processes the survival tems.
From page 86...
... . The NRC's Committee on the Cognitive Foundations of Assessment portrayed this process of reasoning from evidence in the form of what it called the assessment triangle (NRC 2001b, pp.
From page 87...
... Using the triangle as a foundation, several different researchers have developed processes for assessment development that take into account the logic that underlies the assessment triangle. These approaches can be used to create any type of assessment, from a classroom assessment to a large-scale state testing program.
From page 88...
... Mislevy and colleagues argue that by breaking the specifications into smaller pieces they can be reassembled in differ BOX 5-4 The Principal Components of the Evidence-Centered Assessment Design Conceptual Assessment Framework and the Four-Process Architecture SOURCE: Almond, Steinberg, and Mislevy (2002) ; Mislevy, Steinberg, and Almond (2002)
From page 89...
... . ECD relies on specific measurement models that are associated with each task for accomplishing the summary scoring process.
From page 90...
... We use these building blocks as a framework for illustrating the assessment design process. The building blocks are: (1)
From page 91...
... among others -- provide a way of clarifying what is meant by a standard by suggesting connections between the conceptual knowledge in the standards and related abilities and understandings that can be observed and assessed. Learning performances are a way of elabo 5In a large-scale testing program the constructs would be specified in the form of a test framework.
From page 92...
... For example, learning performances might indicate that students should be able to describe phenomena, use models to explain patterns in data, construct scientific explanations, or test hypotheses. Smith, Wiser, Anderson, Krajcik, and Coppola (2004)
From page 93...
... Constructing explanations involves using scientific theories, models, and principles along with evidence to build explanations of phenomena; it also entails ruling out alternative hypotheses. · Analyzing and interpreting data.
From page 94...
... amplify this single standard as three related learning performances: · Students identify and represent mathematically the variation on a trait in a population. · Students hypothesize the function a trait may serve and explain how some variations of the trait are advantageous in the environment.
From page 95...
... A unique feature of OMC items is that they are designed in such a way that each of the possible answer choices is linked to developmental levels of student understanding, facilitating the diagnostic interpretation of student responses. OMC items provide information about the developmental understanding of students that may not be available from traditional multiple-choice items.
From page 96...
... . Possible learning performances and associated assessment tasks: The next learning performance makes use of the skill of identifying.
From page 97...
... Learning performance 1: Students identify chemical reactions. Associated Assessment Task #1: Which of the following is an example of a chemical reaction?
From page 98...
... Items that might be used to tap into students' understandings and misconceptions are included. The explanations of answer choices illustrate how assessment tasks can be made more meaningful if distractors shed light on instructional strategies that are needed to reconstruct student misconceptions.
From page 99...
... . Determining the Measurement Model Formal measurement models are statistical and psychometric tools that allow interpreters of assessment results to draw meaning from large data sets about student performance and to express the degree of uncertainty that surrounds the conclusions.
From page 100...
... 2 Student recognizes that: · the Sun appears to move across the sky every day · the observable shape of the Moon changes every 28 days Student may believe that the Sun moves around the Earth. COMMON ERROR: All motion in the sky is due to the Earth spinning on its axis.
From page 101...
... A unique feature of OMC items is that each of the possible answer choices in an OMC item is linked to developmental levels of student understanding, facilitating the diagnostic interpretation of student item responses. OMC items seek to com bine the validity advantages of open-ended items with the efficiency advantages of multiple-choice items.
From page 102...
... A range of different mathematical models are used to estimate these parameters. When the assumption of undimensionality is not met, a more complex version of the item response theory model -- multidimensional item response theory -- is more appropriate.
From page 103...
... Reprinted by permission of the Center for Assessment and Evaluation of Student Learning. In large-scale assessment programs it is typical for state personnel to decide on the measurement model that will be used, in consultation with the test development contractor.
From page 104...
... The same process can be used to analyze state standards and to compare what an assessment is measuring with a state's goals for student learning. At the heart of the process is a heuristic framework for conceptualizing the
From page 105...
... The researchers also analyzed the Delaware science content standards using the achievement framework and found that the state standards were more heavily weighted toward schematic knowledge than was the assessment -- indicating that the assessment did not adequately represent the cognitive priorities contained in the state standards. These findings led to changes in the state testing program and the development of a strong curriculum-connected assessment system for improvement of student learning to supplement the state test and provide additional information on students' science achievement (personal communication, Rachel Wood)
From page 106...
... Their first step was to organize the key concepts of atomic molecular theory around six big ideas that form two major clusters: the first two form a macroscopic level cluster and the last four form an atomic-molecular level cluster (Box 5-10 provides further detail on these concepts)
From page 107...
... The atomic-molecular theory elaborates on the macroscopic big ideas stud ied earlier and provides deeper explanatory accounts of macroscopic properties and phenomena. Six Big Ideas of Atomic Molecular Theory That Form Two Major Clusters M1.
From page 108...
... Codify the big ideas into learning performances: types of tasks or activities suitable for classroom settings through which students can demonstrate their understanding of big ideas and scientific practices.
From page 109...
... Children develop accounts of properties of matter and changes in matter that make increased use of hidden mechanisms and atomic-molecular theory. They are increasingly able to make use of all six big ideas (listed above)
From page 110...
... Evolutionary Biology11 While the importance of incorporating research findings about student learning into assessment development is widely recognized, research in many areas of science learning is incomplete. The design team that addressed evolutionary biology argued, however, that waiting for research to close all of the gaps would be unwarranted.
From page 111...
... As described earlier, learning performances describe specific cognitive processes and associated practices that are linked to achieving particular standards, and thus help to guide the selection of situations for gathering evidence of understanding as well as clues as to what the evidence means. The team illustrates its approach by developing a cartography of big ideas and associated learning performances for evolutionary biology for the first eight years of schooling.
From page 112...
... The committee therefore concludes that the goal for developing high-quality science assessments will only be achieved though the combined efforts of scientists, science educators, developmental and cognitive psychologists, experts on learning, and educational measurement specialists working collaboratively rather than separately. The experience of the design teams described in this chapter and multiple findings of other NRC committees (NRC, 1996, 2001b, 2002)
From page 113...
... DESIGNING SCIENCE ASSESSMENTS 113 Question 5-1: Have research and expert professional judgment about the ways in which students learn science been considered in the design of the state's science assessments? Question 5-2: Have the science assessments and tasks been created to shed light on how well and to what degree students are progressing over time toward more expert understanding?


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.