Developing Assessments for the Next Generation Science Standards (2014) / Chapter Skim
Currently Skimming:
5 Assessment for Monitoring
5 Assessment for Monitoring
Pages 133-192
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 133... ...
in order to align with the Next Generation Science Standards (NGSS) : they will need to address the progressive nature of learning, include multiple components that reflect three-dimensional science learning, and include an interpretive system for the evaluation of a range of student products.
|
From page 134... ...
We begin with a brief look at cur rently used assessments, considering them in light of the NGSS. We next discuss the challenges of using performance tasks in assessments intended for adminis tration on a large scale, such as a district, a state, or the national level, and we 134 Developing Assessments for the Next Generation Science Standards
|
From page 135... ...
CURRENT SCIENCE MONITORING ASSESSMENTS In the United States, the data currently used to answer monitoring-related ques tions about science learning are predominantly obtained through assessments that use two types of test administration (or data collection) strategies.
|
From page 136... ...
. 136 Developing Assessments for the Next Generation Science Standards
|
From page 137... ...
As we discuss in Chapters 3 and 4, assessing three-dimensional science learning will require exam ining the way students perform scientific and engineering practices and apply crosscutting concepts while they are engaged with disciplinary core ideas. Currently, some state science assessments include the types of questions that could be used for assessing three-dimensional learning (e.g., questions that make use of technology to present simulations or those that require extended con structed responses)
|
From page 138... ...
can help address this issue. 138 Developing Assessments for the Next Generation Science Standards
|
From page 139... ...
To date, the problem of equating assessments that rely solely on performance tasks has not yet been solved. Some assessment programs that include both performance tasks and other sorts of items use the items that are not performance based to equate different test forms, but this approach is not ideal -- the two types of tasks may actually measure somewhat different constructs, so there is a need for studies that explore when such equating would likely yield accurate results.
|
From page 140... ...
140 Developing Assessments for the Next Generation Science Standards
|
From page 141... ...
These assessment programs were ambitious, innovative responses to calls for education reform. They made use of assessment approaches that were then cutting edge for the measurement field.
|
From page 142... ...
. 142 Developing Assessments for the Next Generation Science Standards
|
From page 143... ...
With regard to the scoring process, particularly human scoring, strategies that can yield acceptable levels of interrater reliability include the following: • use of standardized tasks that are designed with a clear idea of what consti tutes poor and good performance; • clear scoring rubrics that minimize the degree to which raters must make inferences as they apply the criteria to student work and that include several samples of student responses for each score level; • involvement of raters who have significant knowledge of the skills being measured and the rating criteria being applied; and • providing raters with thorough training, combined with procedures for mon itoring their accuracy and guiding them in making corrections when inaccu racies are found. With regard to score generalizability (i.e., the extent to which the score results for one set of tasks generalize to performance on another set of tasks)
|
From page 144... ...
DESIGN OPTIONS There is no doubt that developing assessments that include performance tasks and that can be used to monitor students' performance with respect to the NGSS will be challenging, but prior research and development efforts, combined with lessons learned from prior and current operational programs, suggest some strate gies for addressing the technical challenges. New methods will be needed, drawing 144 Developing Assessments for the Next Generation Science Standards
|
From page 145... ...
Even with multiple test ing sessions, however, assessments designed for monitoring purposes alone cannot fully cover the NGSS performance expectations for a given grade within a reason able testing time and cost. Moreover, some performance expectations will be dif ficult to assess using tasks not tied directly to a school's curriculum and that can be completed in 90 minutes or less.
|
From page 146... ...
. 146 Developing Assessments for the Next Generation Science Standards
|
From page 147... ...
For each class, we provide a general description of options, illustrating the options with one or more operational assessment programs. For selective cases, we also provide examples of the types of performance tasks that might be used as part of the design option.
|
From page 148... ...
to identify two different photosynthetic pigments and to explain how the data support their iden tification. Students were then presented with a description of an experiment for investigating how the wavelength of available light affects the rate of photosynthe 148 Developing Assessments for the Next Generation Science Standards
|
From page 149... ...
Finally, students were asked to pro pose a possible evolutionary history of plants by connecting differences in resource availability with different selective pressures that drive the process of evolution through natural selection. Collectively, the multiple components in this task are designed to provide evidence relevant to the nine learning objectives, which are shown in Box 5-1.
|
From page 150... ...
150 Developing Assessments for the Next Generation Science Standards
|
From page 151... ...
Both assessments are designed to measure inquiry skills as envisioned in the science standards that predate the new science framework and the NGSS. Thus, they are not fully aligned with the NGSS performance expectations.
|
From page 152... ...
152 Developing Assessments for the Next Generation Science Standards
|
From page 153... ...
FIGURE 5-3 Two sample AP biology responses. SOURCE: College Board (2013b, pp.
|
From page 154... ...
FIGURE 5-3 Continued R02484 FIG5-3b convert.eps bitmap 154 Developing Assessments for the Next Generation Science Standards
|
From page 155... ...
FIGURE 5-3 Continued R02484 FIG5-3c convert.eps bitmap Assessment for Monitoring 155
|
From page 156... ...
FIGURE 5-3 Continued R02484 FIG5-3d convert.eps bitmap 156 Developing Assessments for the Next Generation Science Standards
|
From page 157... ...
Like the U.S. framework, the Swiss framework defined three dimensions of science education -- which they called skills, domains, and levels -- and emphasized the idea of three-dimensional science learning.
|
From page 158... ...
158 Developing Assessments for the Next Generation Science Standards
|
From page 159... ...
On the lines below, write an explanation of what happened. Scoring Rubric for Question 2 2 Points: The drawing contains the following elements: the water surface, the ship floating tilted in the water, the lowest point of the ship is the side containing the disc.
|
From page 160... ...
FIGURE 5-4 Continued R02484 FIG5-4c convert.eps 160 Developing Assessments for the Next Generation Science Standards
|
From page 161... ...
______________________________________________________________________________ ______________________________________________________________________________ ________________________________________________________________________ Scoring Rubric for Question 3 Types of Questions: a. Does the ship sink when I load it evenly with all four discs?
|
From page 162... ...
R02484 FIG5-4d convert.eps FIGURE 5-4 Continued 162 Developing Assessments for the Next Generation Science Standards
|
From page 163... ...
Figure 5-4 shows an excerpt from the portion of the task that involves the ship (see Table 1-1 for the specific disciplinary core ideas, scientific practices, and crosscutting concepts assessed)
|
From page 164... ...
164 Developing Assessments for the Next Generation Science Standards
|
From page 165... ...
For example, unless all the performance tasks are computer-based, the logistical and student-time burden of administering multiple tasks in the same classroom could be prohibitive. There are also risks associated with using all the tasks in an assessment in each classroom, such as security and memorability, which could limit the reuse of the tasks for subsequent assessments.19 The assessment strategies discussed above have varying degrees of over lap with the assessment plans that are currently in place for mathematics and language arts in the two Race to the Top Assessment Program consortia, the Partnership for Assessment of Readiness for College and Careers and the Smarter Balanced Assessment Consortium (see Chapter 1)
|
From page 166... ...
Classroom teachers could be trained to score these tasks, or student products could be submitted to the district or state 166 Developing Assessments for the Next Generation Science Standards
|
From page 167... ...
and when during the administration period to give the assessments, and they participate in the scoring process. Assessment Task Example 11, Plate Tectonics: An example of a performance task that might be used for monitoring purposes is one that was administered in a classroom after students had covered major aspects of the earth and space science standards.
|
From page 168... ...
168 Developing Assessments for the Next Generation Science Standards
|
From page 169... ...
(The developers noted that the task could also be used to generate evidence of student understanding of the crosscutting con cepts of pattern and scale, although that aspect is not covered in this rubric.) The scoring rubric addressed the middle school performance expectations, as well as the range of student responses generated from a field test of the task.
|
From page 170... ...
Classroom teachers could be trained to 170 Developing Assessments for the Next Generation Science Standards
|
From page 171... ...
. Maintaining the Quality of Classroom-Embedded Components The options described above for classroom administration as part of a monitoring assessment program introduce the possibility of local (district or school)
|
From page 172... ...
. 172 Developing Assessments for the Next Generation Science Standards
|
From page 173... ...
School assessment programs include opportunities to determine the nature of students' learning and then provide appropriate feedback or intervention. This is referred to as "authentic pedagogy." In this practice, teachers do not teach and then hand over the assessment that "counts" to external experts to judge what the students have learned: rather, authentic pedagogy occurs when the act of teaching involves placing high-stakes judgments in the hands of the teachers.
|
From page 174... ...
. 174 Developing Assessments for the Next Generation Science Standards
|
From page 175... ...
Mobile devices, computers, and other forms of technology can be used with any of the assessments we have described. Adapting assessments to technology-based enhancements opens up new possibilities for assessment tasks and for scoring and interpreting the results of tasks that assess three-dimensional science learning.
|
From page 176... ...
176 Developing Assessments for the Next Generation Science Standards
|
From page 177... ...
Example: Technology-Enhanced Version of an AP Biology Question Using the options in the above taxonomy -- or other approaches to innova tive formats -- technology-enhanced assessments can be designed to address par ticular assessment challenges. Using assessment design approaches that draw on 28Foran interactive version of the taxonomy, see http://pages.uoregon.edu/kscalise/taxonomy/ taxonomy.html [June 2013]
|
From page 178... ...
178 Developing Assessments for the Next Generation Science Standards
|
From page 179... ...
In the next three figures, we have adapted it for use in a technology-enhanced environment. The delivery environment shown here, into which the example task has been integrated, is drawn from an example presented recently for assessing hard-to-measure con structs in the Common Core State Standards (Barton and Schultz, 2012)
|
From page 180... ...
The problem with that format is that when only a few options are shown, students can "backsolve": that is, instead of directly solving the problem, they can test each of the provided solutions. Furthermore, when a limited range of answer choices is provided, student thinking may be prompted by the visual displays provided in the 180 Developing Assessments for the Next Generation Science Standards
|
From page 181... ...
Example: Technology-Enhanced Tasks on NAEP Another example of the ways in which technology enhancements can be used is provided by the 2009 NAEP Interactive Computer and Hands-On Tasks Science Assessment. This assessment, given to national samples of students in the 4th, 8th, and 12th grades, was designed to produce national results for each grade.
|
From page 182... ...
. 182 Developing Assessments for the Next Generation Science Standards
|
From page 183... ...
The task concludes by asking students to write a rec ommendation for the location of the recreation area, justify the recommendation with evidence, and discuss the environmental effects. It is important to point out that although these tasks do involve new ways of assessing science learning, they were not designed to measure the type of three dimensional science learning that is in the NGSS.
|
From page 184... ...
. 184 Developing Assessments for the Next Generation Science Standards
|
From page 185... ...
is expected to select the appropriate website to answer the question "Where do polar bears live that do not belong to any country? " In this case, the question is designed for practice to acquaint the student with the technology.
|
From page 186... ...
186 Developing Assessments for the Next Generation Science Standards
|
From page 187... ...
Task Surrounds In the context of technology-enhanced assessment, a task surround is a set of small software programs that work together to create a set of activities, such as for a research or inquiry activity, which can be readily populated with new content (Scalise, 2011, p.
|
From page 188... ...
188 Developing Assessments for the Next Generation Science Standards
|
From page 189... ...
CONCLUSIONS AND RECOMMENDATIONS Our review of various strategies for administering assessments of three-dimension al science learning in formats that will yield results to support system monitoring makes clear that there are tradeoffs with a number of competing goals. One goal is to use assessments composed principally of performance tasks, particularly those that allow students to actually demonstrate their skills using hands-on tasks.
|
From page 190... ...
CONCLUSION 5-3 Matrix sampling will be an important tool in the design of assessments for monitoring purposes to ensure that there is proper coverage of the full breadth and depth of the NGSS performance expectations. The approaches we propose for designing monitoring assessments that include performance tasks and portfolios may not yield the level of comparability 190 Developing Assessments for the Next Generation Science Standards
|
From page 191... ...
RECOMMENDATION 5-2 Assessment developers should take advantage of emerging and validated innovations in assessment design, scoring, and reporting to create and implement assessments of three-dimensional science learning. To the extent that they facilitate achieving valid and reliable outcomes, available technological approaches should be used in designing, administering, and scoring science assessments.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.