Page 38
4 Effects of High-Stakes Testing and Standards
The education reform movement has made the needs of students at risk for academic failure a key focus. Both nationally and at the state and district levels, many elements of reform have been devised with the goal of improving the performance of students who have been left behind in the past; keeping young people in school and helping all students reach high standards have been explicit goals. Standards-based reform has held out the promise of helping educators and policy makers to strengthen academic programs and to make public education more equitable than it has ever been. By making expectations for all students explicit, reforms have helped many jurisdictions understand the educational needs of the range of students they serve. Well constructed and properly used standards-based tests, by providing data about the outcomes of educational programs, can assist policy makers, administrators, and teachers in ensuring that all students are offered what they need to meet established goals and to make needed improvements in teaching, curricula, and other program elements.
However, by themselves, tests do not improve student achievement. They provide information that, together with information from other sources, can be used to improve curriculum, teaching, and learning. In some cases, however, adoption of test requirements has outpaced other reforms (Education Week, 2001). That is, testing is in many cases less expensive or simpler to adopt than are other reforms. Ensuring that curricula are aligned with standards and tests, ensuring that students have been taught
Page 39
the material and skills for which they are being held responsible, ensuring that needed resources are in place, modifying teaching strategies, and the like can all present challenges much larger than those that come with instituting new testing requirements. Meeting these challenges, however, is what will ultimately improve teaching and learning.
PURPOSES OF TESTING
Reliance on test results for a variety of purposes, including determining which students are eligible for promotion to the next grade and for graduation, has been growing in many states. Such testing can have important implications for students already experiencing academic difficulty— the students most likely to fail those tests. Understanding the effects of tests on dropout rates is complicated not only by difficulties with the available means of counting dropouts, but also by the complexity of testing itself. Tests are used for a variety of purposes, described in Box 4-1, and these purposes lead to differing consequences for students. Moreover, even
BOX 4-1 The Principal Purposes for Educational Tests
SOURCE: National Research Council, 2000:20. |
Page 40
tests used ostensibly for the same purpose can have very different effects, depending on how they are applied, constructed, and used. Tests may be closely aligned to the skills and curriculum students are required to cover and be administered to students who have had adequate opportunity to learn the required material. They may be valid and reliable for the purposes for which they are being used, and they may successfully distinguish among students who have made a good effort to learn and those who have not. Unfortunately, in some cases these conditions are not met. Test results may penalize students who are the victims of ill-prepared teachers, poorly run schools or districts, or other circumstances beyond their control.
Professional guidelines for the use of high-stakes tests have been developed by the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education (1999), and were outlined in the report of the National Research Council (1999). While these vital professional standards were well explicated, and may be widely known, a few key elements are worth highlighting here because the committee observes that they are not uniformly adhered to. Of particular importance in the context of a discussion of dropouts is the need for measures to:
- ensure that at every grade level early monitoring and intervention, remediation, and other supports are in place for students at risk for failing a test being used for a high-stakes purpose;
- ensure that necessary changes in teaching and curriculum have been made so that students have adequate opportunity to learn the material on which they are being tested before such tests are used in making promotion or graduation decisions;
- ensure that students for whom English is a second language or who have disabilities that affect their schooling are tested only in ways that comport with professional standards regarding inclusion and accommodations; and
- ensure that students are given sufficient opportunities to demonstrate mastery of required content and skills—that is, that a test is not used as the sole criterion for high-stakes decisions about students.
HIGH SCHOOL EXIT EXAMINATIONS
It is possible that exit or graduation tests play a role in students' decisions about dropping out. Such tests are currently growing in popularity,
Page 41
but the idea of using a test to determine who is eligible for a diploma is not a new one. Beginning in the 1970s, a number of states used tests of basic skills, often called minimum competency testing, as a requirement for graduation. Eighteen states now use them and 11 more are developing them (Bishop et al., 2000). 1 These tests were generally intended to ensure that graduating students had mastered the basic literacy and other skills that are considered the minimum necessary for citizenship and employment; they were not typically the sole criterion for graduation (Linn, 2000). States and districts have been making many changes to their requirements but in general seem to be moving toward more challenging exit exams. In recent years a number of states have developed new, more stringent exit examinations or have made existing tests more challenging (National Research Council 1999:164). These newer tests are intended to measure mastery of the more complex knowledge and skills that are detailed in standards documents.
The National Research Council report also addressed the question of how such tests might affect dropout rates. That report found that while a number of studies have explored the question, the results have been somewhat mixed, and further research is clearly needed. Because most of the more stringent exit exams were either recently implemented or are still in the development stages (many states have delayed the year in which students would first be required to pass the exam to graduate), most of the available research has focused on minimum competency tests. While an association between test failure and dropping out is often evident, the committee found that a clear causal connection between exit testing and dropout rates has not been conclusively established (National Research Council, 1999:174).
Bishop et al. (2000) explore the effects of policies associated with school reform, including exit exams that measure basic skills, on students' “schooling, learning, and earning.” They present a wealth of data on differences in various outcomes for students who had been exposed to minimum competency exams. They find that these tests may have the effect of increasing dropout rates for some students. They also find that such tests seem to be associated with improved college attendance and increased earnings for students who pass them. They argue that, in general, such tests seem to have
1 Change is rapid; 20 states are listed in a recent Education Week (2001) survey as requiring exit examinations.
Page 42
the effect of strengthening curricula and, in other ways, improving student learning and thus the value of a high school diploma (Bishop et al., 2000). However, there are other possible explanations for these findings. One is that dropout rates may be higher in states that administer exit exams. If fewer low-achieving students are part of the population that takes such tests by the time they are administered, then the pass rates of those who remain will be higher, even if the achievement of those who actually take the test has not improved.
The workshop discussion on this point made it clear that further research is needed before firm conclusions can be reached about the effects of testing on dropout rates. The testing programs that are being implemented around the country are intended to increase learning and to prod educators to offer and students to take more demanding coursework. The desired outcome is that fewer students will drop out, both because their academic needs will be better met and because they will be motivated to work harder. However, many observers worry that students with low economic status and other risk factors will be disproportionately likely to fail exit and promotion tests and that the result will be “differential effects on grade progression and on high-school dropout” (Hauser, 2000:9). It is important to note here that group differences in test performance do not necessarily indicate problems with a test. Test scores may reflect actual differences in achievement, which could, in turn, be the result of deficiencies in or lack of access to particular coursework or instruction (National Research Council, 1999:5). Currently available data do not provide answers to many focused questions about the relationship between explicit exit examination requirements and dropout rates and do not take account of the increasing number of alternate pathways that may account for the decrease in dropout rates, particularly among black and Hispanic students.
In considering the need for additional data about school completion, it is important to note one population of students who have not traditionally been classed among dropouts: students who complete the requirements for twelfth grade but are unable to pass an examination that is a requirement for the diploma. Many states that use exit examinations—New York, Maryland, and others—have made provisions for remedial assistance, multiple opportunities for students to take the examinations, and other supports. It seems clear that the specific ways in which an exit exam is implemented are important. Nevertheless, among the students who fail will be some, perhaps many, who would otherwise have graduated. Little information is available to indicate how many such students there are, either within juris
Page 43
dictions or across the nation. However, since pass rates on exit examinations (and other educational tests) are consistently lower for low-income students, minority students, English-language learners, and students with disabilities, the likelihood is that nongraduates who have completed all course requirements will be drawn disproportionately from among those same students. More detailed information about how many students are in this category in each of the states that rely on exit examinations is needed. Improved understanding of the possible differences between these students and students who drop out of school before the end of twelfth grade, and about the effects of failing the exam on these students' future education and employment, will be an important part of understanding the effects of exit exams.
Recommendation 5: The committee recommends that jurisdictions that administer exit exams collect data on students who complete the twelfth grade but fail exit exams and so do not graduate (disaggregated to allow monitoring of such populations as different minority groups, English-language learners, and students with disabilities).
RETENTION IN GRADE
There is another kind of high-stakes testing that can play an important role in students' progress. A significant body of research has addressed the relationship between retaining students in grade—that is, not promoting them to the next grade—and their subsequent educational progress. The National Research Council report (1999) found that the use of tests as requirements for grade promotion, both at the state and district levels, is increasing and explored data showing the rates at which students are retained in grade at various stages of schooling. Grade retention is pervasive in American schools, and it is more common among black and Hispanic youngsters than among whites. The report also documents the considerable evidence that students who are retained in grade (even as early as elementary school) perform less well in school (even when results are controlled for age and number of grades completed) and are significantly more likely to drop out of school.
According to a variety of census and other data that Hauser (1999) has assembled, there was a substantial increase between 1972 and 1996 in the numbers and percentages of students who were at least one grade behind most other children their age. Moreover, there are significant differences,
Page 44
by race and gender, in the rates at which students fall behind. These differences appear in the early primary grades and increase as students move through school. Among 15- to 17-year-olds, about 50 percent of black males and 30 percent of white females are at least one grade behind most students their age. Hauser further shows that students who are retained in any grade are significantly more likely to drop out of school than those who are not, even when factors such as sex, race and ethnicity, social background, cognitive ability, and other factors are controlled. 2
Valencia (2000) also argues that retaining students in grade is a very strong predictor of the subsequent choice to drop out of school. He notes further that African American and Mexican American children in Texas are significantly more likely to be retained in grade than are white children. Valencia's concern, based on his findings, is that as Texas and other states place increasing reliance on standardized tests in making their decisions about whether to retain students or promote them to the next grade, the result will be an increase in retention, a corresponding increase in the dropout rate as the students affected move through the system, and a disparate effect on subgroups of the population that are already vulnerable (Valencia, 2000). Other scholars have reinforced the connection between retention in grade at any level and the subsequent decision to drop out (Rumberger, 2000, 2001). Although the precise relationship between graduation testing and dropping out of school is still in dispute, there is no dispute that retention in grade is a very strong predictor of who will drop out.
Conclusion: Given the difficulty and cost of preventing students from dropping out once the process of disengagement from school has begun, it is clear that neither requiring a student to retake the grade nor promoting a failing student is, by itself, a sufficient response to his or her academic difficulty. The value and importance of addressing struggling students' difficulties directly and specifically as soon as they are apparent are paramount. Moreover, the strong association between retention in grade and dropping out suggests that retention is usually not a beneficial intervention.
2 Hauser cites one study (Temple et al., 1998) that finds an increase of 12 percentage points and another (Anderson, 1994) that finds a 70 percent increase in likeliness to drop out for students who repeat a grade.
Page 45
SUMMING UP
Distinctions among different kinds of tests are key to understanding the effects they may have on students. The effects of more rigorous, content-based tests may be very different from the effects of basic skills tests. The way in which testing programs are aligned with curricular requirements and standards, as well as other aspects of the educational system, are also central for understanding their effects. To be truly informative, evaluations of the effects of exit tests should clearly distinguish among tests of different kinds, used for different purposes, and should draw on data about each category of students who might be affected by such tests.
At this early stage in the progress of many state reforms, there is insufficient evidence to determine what effect, if any, exit examinations have on dropout rates. Indeed, the likelihood is that the effects of these tests will vary significantly, depending on the ways in which they are constructed and implemented and on the ways in which their results are used. However, there is reason to believe that both exit testing and other high-stakes testing may sometimes be used in ways that have unintended harmful effects on students at risk for academic failure because of poverty, lack of proficiency in English, disability, and membership in population subgroups that have been educationally disadvantaged.
Changes in dropout rates, or in the characteristics of the students who drop out, may be signs both of the effects—positive or negative, intended or not—of new standards or exit exams and of how well schools are helping students to meet higher standards. Any such changes should be carefully monitored and evaluated as jurisdictions progress with their reforms. However, inferences of cause and effect should be made with care. The progress of students' schooling and the influences that lead some of them to drop out are complex. Moreover, the available statistics that can reveal changes in dropout patterns are likewise complex and should be interpreted carefully. The committee hopes that the recommendations in this report will be a useful tool for those who seek to improve understanding of school completion.
