Knowing What Students Know The Science and Design of Educational Assessment (2001) / Chapter Skim
Currently Skimming:
Pages 221-260
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 221... ...
Furthermore, the information gained should be put to good use by informing decisions about curriculum and instruction and ultimately improving student learning (Falk, 2000; National Council of Teachers of Mathematics, 19951. Assessments do not function in isolation; an assessment's effectiveness in improving learning depends on its relationships to curriculum and instruction.
|
From page 222... ...
Here assessment is used by teachers and students mainly to assist learning, but also to gauge students' summative achievement over the longer term. Second is large-scale assessment, used by policy makers and educational leaders to evaluate programs and/or obtain information about whether individual students have met learning goals.
|
From page 223... ...
Trade-Offs In Assessment Design: Examples Accountability Versus Instructional Guiclance for Incliviclual Students The first example expands on the contrast between classroom and largescale assessments described above. A starting point is the desire for statewide accountability tests to be more helpful to teachers or the question of why assessment designers cannot incorporate in the tests items that are closely tied to the instructional activities in which students are engaged (i.e., assessment tasks such as those effective teachers use in their classrooms)
|
From page 224... ...
Schools can obtain useful feedback on the quality of their curriculum, but NAEP's benefits are traded off against several limitations. Measurement at the level of individual students is poor, and individuals can not be ranked, compared, or diagnosed.
|
From page 225... ...
If multiple assessments grow out of a shared knowledge base about cognition and learning in the domain, they can provide valuable multiple perspectives on student achievement while supporting a core set of learning goals. Stakeholders should not be unduly concerned if differing assessments yield different information about student achievement; in fact, in many circumstances this is exactly what should be expected.
|
From page 226... ...
The project involved working closely with 24 science and mathematics teachers to develop their formative assessment practices in everyday classroom work. During the course of the project, several aspects of the teaching and learning process were radically changed.
|
From page 227... ...
In brief, the development of good formative assessment requires radical changes in the ways students are encouraged to express their ideas and in the ways teachers give feedback to students so they can develop the ability to manage and guide their own learning. Where such innovations have been instituted, teachers have become acutely aware of the need to think more clearly about their own assumptions regarding how students learn.
|
From page 228... ...
Formative Assessment, Cumculum, and Instruction At the 2000 annual meeting of the American Educational Research Association, Shepard (2000) began her presidential address by quoting Graue's (1993, p.
|
From page 229... ...
National and state standards documents set forth learning goals, but often not at a level of detail that is useful for operationalizing those goals in instruction and assessment (American Federation of Teachers, 1999; Finn, Petrilli, and Vanourek, 19981. By dividing goal descriptions into sets appropriate for different age and grade ranges, current curriculum standards provide broad guidance about the nature of the progression to be expected in various subject domains.
|
From page 230... ...
As described in Chapter 3, cognitive research has produced a rich set of descriptions of how people develop problem-solving and reasoning competencies in various content areas, particularly for the domains of mathematics and science. These models of learning provide a fertile ground for designing formative assessments.
|
From page 231... ...
In a program such as CGI, formative assessment is woven seamlessly into the fabric of instruction (Carpenter et al., 19961. Intelligent Tutors As described in previous chapters, intelligent tutoring systems are powerful examples of the use of cognitively based classroom assessment tools blended with instruction.
|
From page 232... ...
On the other hand, some research suggests that the relationship bet~veen formative assessment and cognitive theory can be more complex. In a study of Anderson's geometry tutor with high school students and their teachers, Schofield and colleagues found that teachers provided more articulate and better-tuned feedback than did the intelligent tutor (Schofield, Eurich-Fulcer, and Britt, 19941.
|
From page 233... ...
Moreover, the assistance provided by teachers to students was less public. Hence, formative assessment and subsequent modification of instruction both highly valued by these high school students- were mediated by a triadic relationship among teacher, student, and intelligent tutor.
|
From page 234... ...
Similarly, descriptions of typical practices make little sense in the absence of careful consideration of the forms of knowledge representation and reasoning they entail (Cobb, 19981. Complex cognitively based measurement models can be embedded in intelligent tutoring systems and diagnostic assessment programs and put to good use without the teacher's having to participate in their construction.
|
From page 235... ...
Many studies that have examined gains between pre- and post-tests, comparing programs in which formative assessment was the focus of the innovation and matched control groups were used, have shown effect sizes in the range of 0.4 to 0. 7i (Black and Wiliam, 19981.
|
From page 236... ...
It is their responsibility to use the assessment information to guide their progress toward learning goals. Consider the following assessment ex
|
From page 237... ...
This can be done through techniques such as the development of scoring rubrics or criteria for evaluating student work. As emphasized in Chapter 3, metacognitive awareness and control of one's learning are crucial aspects of developing competence.
|
From page 238... ...
238 KNOWING WHAT STUDENTS KNOW BOX K-3 Impact of Reflective Inquiry on Learning White and Frederiksen (2000) carried out a controlled study comparing ThinkerTools classes in which students engaged in the reflective-assessment process with matched control classes in which they did not.
|
From page 239... ...
6 ASSESSMENT IN PRACTICE assessment. Note also that this was particularly true for the low-achieving students.
|
From page 240... ...
There is some indication from other studies that the finding of greater gains for less able students may be generalizable, and this is certainly an area to be further explored.3 For now, these initial findings suggest that effective formative assessment practices may help overcome disadvantages endured at earlier stages in education. Another possible source of bias may arise when students do not understand or accept learning goals.
|
From page 241... ...
However, fully capitalizing on the new foundations described in this report will require more substantial changes in the way large-scale assessment is approached, as well as relaxation of some of the constraints that currently drive large-scale assessment practices. As described in Chapter 5, large-scale summative assessments should focus on the most critical and central aspects of learning in a domain as identified by curriculum standards and informed by cognitive research and theory.
|
From page 242... ...
A major problem is that only limited improvements in large-scale assessments are possible under current constraints and typical standardized testing scenarios. Returning to issues of constraints and trade-offs discussed earlier in this chapter, large-scale assessments are designed to serve certain purposes under constraints that often include providing reliable and comparable scores for individuals as well as groups; sampling a broad set of curriculum standards within a limited testing time per student; and offering cost-efficiency in terms of development, scoring, and administration.
|
From page 243... ...
, a population sampling approach can be used whereby different students take different portions of a much larger assessment, and the results are combined to obtain an aggregate picture of student achievement. If individual student scores are needed, broader sampling of the domain can be achieved by extracting evidence of student performance from classroom work produced during the course of instruction (often referred to as "curriculumembedded" assessment)
|
From page 244... ...
Instead they were selected to illustrate some alternative ways of approaching large-scale assessment and the trade-offs entailed. The first two examples show how population sampling has been used for program evaluation at the national and state levels to enable coverage of a broader range of learning goals than would be possible if each student were to take the same form of a test.
|
From page 245... ...
Maryland is one of the few states in the country that has decided to optimize the use of assessment for program evaluation, forgoing individual student scores.4 A population sampling design is used, as opposed to the census testing design used by most states. MSPAP consists of criterion-referenced performance tests in reading, mathematics, writing, language usage, science, and social studies for students in grades 3, 5, and 8.
|
From page 246... ...
AP Studio Art The above two examples do not provide individual student scores. The AP Studio Art portfolio assessment is an example of an assessment that is designed to certify individual student attainment over a broad range of competencies and to be closely linked to the actual instruction students have experienced (College Board, 19941.
|
From page 247... ...
Thus by using a curriculum-embedded approach, the AP Studio Art program is able to collect rich and varied samples of student work that are tied to students' instructional experiences over the course of the year, but can also be evaluated in a standardized way for the purposes of summative assessment. It should be noted that some states attempting to implement large-scale portfolio assessment programs have encountered difficulties (Koretz and Barron, 19981.
|
From page 248... ...
To make appropriate instructional decisions, teachers should teach to the model of learning as conveyed, for example, by progress maps and rubrics for judging the quality of student work" rather than focusing on the particular items on a test. Test users must understand that any particular set of assessment tasks represents only a sample of the domain and that tasks will change from year to year.
|
From page 249... ...
. Feedback and Expectations for Learning In Chapters 4 and 5, we illustrated some of the kinds of information that could be obtained by reporting large-scale assessment results in relation to developmental progress maps or other types of learning models.
|
From page 250... ...
For example, Developmental Assessment based on progress maps is being used in the Commonwealth of Victoria to assess literacy. An evaluation of the program revealed that users were "overwhelmingly positive about the value and potential of Developmental Assessment as a means for developing shared understandings and a common language for literacy development" (Meters and Culican, 2000, p.
|
From page 251... ...
Development of the program was a collaboration between the Learning Research and Development Center of the University of Pittsuburgh and the National Center on Education and the Economy, in partnership with states and urban school districts. Together they developed challenging standards for student performance at grades 4, 8, and 10, along with large-scale assessments designed to measure attainment of those standards.6 The New Standards Project includes three interrelated components: performance standards, a portfolio assessment system,7 and an on-demand exam.
|
From page 252... ...
Before describing specific properties of such systems, we consider issues of balance and allocation of resources across classroom and large-scale assessment. Balance Between Classroom and Large-Scale Assessment The current educational assessment environment in the United States clearly reflects the considerable value and credibility accorded external, largescale assessments of individuals and programs relative to classroom assessments designed to assist learning.
|
From page 253... ...
No single test score can be considered a definitive measure of a student's competence. Multiple measures enhance the validity and fairness of the inferences drawn by giving students various ways and opportunities to demonstrate their competence.
|
From page 254... ...
. this particular physics examination is now subject to changed combining the results of external tests with classroom assessments of particular aspects of achievement for which a short formal test is not appropriate is an established feature of achievement testing systems in the United Kingdom and 8Because the whole structure of the 16-18 examinations is being changed, this examination and the curriculum on which it is based, which have been in place for 30 years, will no longer be in use after 2001.
|
From page 255... ...
Coherence For the system to support learning, it must also have a quality the committee refers to as coherence. One dimension of coherence is that the conceptual base or models of student learning underlying the various external and classroom assessments within a system should be compatible.
|
From page 256... ...
Some might argue that what we have described are the goals of current curriculum standards. But while the existing standards emphasize what students should learn, they do not describe how students learn in ways that are maximally useful for guiding instruction and assessment.
|
From page 257... ...
On the other hand, intelligent tutoring systems have a strong cognitive research base and offer opportunities for integrating formative and summative assessments, as well as measuring growth, yet their use for large-scale assessment purposes has not yet been explored. Thus, examples in this report offer a rich set of opportunities for further development toward the goal of designing assessment systems that are maximally useful for both informing and improving learning.
|
From page 258... ...
Alternatives to on-demand, census testing are available. If individual student scores are needed, broader sampling of the domain can be achieved by extracting evidence of student performance from classroom work produced during the course of instruction.
|
From page 259... ...
The current educational assessment environment in the United States assigns much greater value and credibility to external, large-scale assessments of individuals and programs than to classroom assessment designed to assist learning. The investment of money, instructional time, research, and development for large-scale testing far outweighs that for effective classroom assessment.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.