An Evaluation of the Public Schools of the District of Columbia Reform in a Changing Landscape (2015) / Chapter Skim
Currently Skimming:
6 Outcomes
6 Outcomes
Pages 161-192
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 161... ...
The only indicators of student progress that we could use were test scores and graduation rates. Although we had access to data about performance on college entrance exams, we had questions about their accuracy and validity.
|
From page 162... ...
Scaled scores were grouped into four performance levels: below basic, basic, proficient, and advanced. The cut scores for each level were determined using standard-setting procedures.4 1 PARCC tests cover mathematics and English-language arts; see Chapter 5.
|
From page 163... ...
Questions on the DC CAS mathematics test were partially aligned in 2012 and fully aligned in 2013.5 Although the city and its test development contractor, CTB/McGraw-Hill, made efforts to maintain the comparability of the tests during this transition, it is important to keep these changes in mind when comparing trend data. Since the DC CAS test scores are the data that have been used for a number of high-stakes decisions in the district (e.g., measures of adequate yearly progress,6 teacher evaluation)
|
From page 164... ...
In this situation, the tests do not meet the requirements for equating. If the designers of DC CAS had put scores onto a common overall scale so that progress could be tracked across years, it would have been done through the statistical procedure called vertical linking (Holland and Dorans, 2006, p.
|
From page 165... ...
Prior to 2012, the score for composition was a simple sum of the scores from the two rubrics, and the cut scores for the performance levels were based on this scale. In 2012, the city implemented procedures to adjust for differences in the difficulty level of the essay prompts given from one year to the next.
|
From page 166... ...
For example, if students improved from far below proficient to just below proficient, their gains would not be measured by changes in proficiency rates. There are other options for reporting that yield information about the full distribution of scores, such as the mean and standard deviation of the scale scores, the scores associated with the quintiles in the distribution (20th, 40th, 60th, 80th percentiles)
|
From page 167... ...
OSSE officials told us about new test security measures that have been introduced in D.C. schools, which are available on their website.a We note that evidence from around the country, for example, as reported by the Atlanta Journal Constitution and USA Today, suggests that cheating on high stakes tests is a widespread and a serious problem.
|
From page 168... ...
: the percentages of students scoring below basic and basic have decreased and the percentages scoring in the proficient level have increased, but there was little growth in the percentages at the advanced level. In contrast, there has been fairly steady progress in math achievement, with fewer students scoring in the basic and below basic categories each year and increasing numbers scoring in both the proficient and advanced categories: see Figure 6-3.
|
From page 169... ...
Performance Trends: By School Sector and Students' Characteristics Looking at outcomes by school sectors, Figures 6-7 and 6-8 show proficiency rates in reading and math, respectively, for DCPS and charter school students from 2007 through 2014. For that period in both sectors, data from OSSE show increases in the percentage of students scoring profi cient or above.14 In math, the proficiency rate for DCPS students increased 14 See http://osse.dc.gov/sites/default/files/dc/sites/osse/publication/attachments/2014%20 DC%20CAS%20Result%20July%2031%202014...FINAL_.pdf [February 2015]
|
From page 170... ...
. Figure 6-2 R02838 Percentage Scoring at Each Level 19 19 2009 2010 2011 2012 2013 2014 FIGURE 6-3 Percentage of students who scored at each performance level on DC CAS math: 2009-2014.
|
From page 171... ...
24 22 Percentage Scoring at Each Level 2011 2012 2013 2014 FIGURE 6-4 Percentage of students who scored at each performance level on DC CAS composition: 2011-2014. SOURCE: Data from LearnDC at http://www.learndc.org [May 2015]
|
From page 172... ...
Dis. 2014 16% 42% 37% 5% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% FIGURE 6-5 Percentage of student groups scoring at each performance level for DC CAS reading 2009 and 2014.
|
From page 173... ...
Dis. 2014 17% 35% 37% 11% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% FIGURE 6-6 Percentage of student groups scoring at each performance level for DC CAS math 2009 and 2014.
|
From page 174... ...
. = Citywide Average Citywide Average 51.4% Citywide Average 49.9% FIGURE 6-8 Percentage of charter school students who scored proficient or above on DC CAS math and reading, compared to the percentage for all public schools' students in the city: 2007-2014.
|
From page 175... ...
, but starting with a higher rate: the proficiency rate was 42.2 percent in 2007, 53 percent in 2013, and 53.4 percent in 2014. Thus, despite some evidence of improvement over time, DCPS students consistently perform below the citywide average in both subjects; public charter school students consistently perform above the citywide average in both subjects.
|
From page 176... ...
For both groups, roughly 60 percent scored below proficient in both subjects. Although there have been some small improvements over the years, these results are TABLE 6-2 Percentage of Students Who Scored Proficient or Above in Math on DC CAS by Students' Characteristics and School Sector, 2013 and 2014 State DCPS Charter Group 2013 2014 2013 2014 2013 2014 All 53.0 54.4 49.5 50.9 58.6 59.6 Female 55.5 57.4 52.2 54.3 60.4 61.8 Male 50.6 51.5 47.0 47.6 56.6 57.3 Asian 86.0 86.6 85.8 85.8 87.5 89.5 African American 47.1 48.1 40.1 40.8 56.2 57.3 Hispanic 58.6 58.3 57.4 58.0 61.0 58.7 Mixed Race 82.7 85.9 84.5 87.0 79.1 83.8 White 91.0 92.0 91 92.2 91.3 91.0 Economically Disadvantaged 46.2 47.4 40.4 40.9 54.5 56.1 English-Language Learner 50.2 49.4 48.3 49.9 53.7 48.6 Special Education 24.0 24.3 20.5 20.4 29.4 29.9 NOTE: This table excludes two groups due to small numbers: Native Hawaiian/Pacific I slander and American Indian/Native Alaskan.
|
From page 177... ...
It is important to note again that these findings cannot be interpreted causally; that is, one cannot infer from the data that these groups perform better when enrolled in charter schools because there have been demographic shifts in the enrollments for charter and DCPS schools that may also have influenced average performance. In addition to examining trends over time, we also reviewed the results from a study conducted by EdCORE researchers that examines performance after adjusting for demographic shifts in the cohorts being compared (Education Consortium for Research and Evaluation, 2014c)
|
From page 178... ...
The EdCORE report cautions that this finding does not necessarily mean that the charter schools have lower-performing students. Some students start and stay in charter schools, some move from a DCPS to a charter school, some move from a charter to a DCPS school, and some start and stay in a DCPS school.
|
From page 179... ...
However, there have been changes over time to the sampling design for the D.C.: in 2009, the city's charter schools were eliminated from the TUDA sample, although they continue to be included in state NAEP results.18 The results presented below are from D.C.'s participation in state NAEP unless otherwise noted (see discussion in National Research Council, 2011, p.
|
From page 180... ...
FIGURE 6-10 Mean scale scores on NAEP in reading for 8th-grade students for D.C. and nationally: 1992-2013.
|
From page 181... ...
. Figure 6-11 R02838 290 282 283 284 278 280 274 276 270 271 272 267 265 262 260 254 250 248 243 245 235 235 234 Score 230 231 233 DC NaƟonal 210 190 170 1990 1992 1996 2000 2000 2003 2005 2007 2009 2011 2013 FIGURE 6-12 Mean scale scores on NAEP in math, for 8th-grade students for D.C.
|
From page 182... ...
Reading Performance In 4th-grade reading, the distribution shows a trend of steadily improving performance: see Figure 6-13. On a year-to-year basis, there was a gradual shift of students from the below basic level to the basic, proficient, and advanced performance levels.
|
From page 183... ...
SOURCE: Data from U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics (2013b)
|
From page 184... ...
-- In 2013, the average for African American students was 62 points lower than for white students, and the average for His panic students was 51 points lower than for white students. Both gaps are not significantly different from those in 1992.
|
From page 185... ...
. • It is also noteworthy that mean scale scores for white students in D.C.
|
From page 186... ...
-- In 2013, the average for African American students was 54 points lower than that for white students, and the average for Hispanic students was 49 points lower than that for white students.22 -- The average for students eligible for free or reduced-price lunches was 31 points lower than for those not eligible. This gap is not significantly different from the gap in 1998.
|
From page 187... ...
-- In 2013, the average for African American students was 55 points lower than that for white students, and the average for Hispanic students was 49 points lower than that for whites. Both of these performance gaps are similar to those observed in 1992.
|
From page 188... ...
students' performance over time or any comparisons between students who attend DCPS schools and charter schools, and thus they are not included in this report. For a sound evaluation, we would have needed such data as the following: • participation rates based on the number of students who take the SAT or ACT in relation to those eligible to take it (e.g., the propor tion of juniors who take one or both of the exams)
|
From page 189... ...
Nationally, for 2012-2013, the overall rate increased from 78 to 81 percent; for blacks it increased from 66 to 68 percent, and for Hispanic students it increased from 71 to 76 percent. Large cities have also seen overall increases: for example, in Chicago the rate increased from 63 to 73 percent between 2010 and 2014; in Baltimore the rate increased from 66 to 74 percent between 2010 and 2012; and in New York City the rate increase from 69 to 70 percent between 2010 and 2012.26 Table 6-3 also shows that the overall graduation rate in D.C.
|
From page 190... ...
. Graduation rates in
|
From page 191... ...
The gap between the performance of white students and that of black and Hispanic students has not narrowed, as shown in both the DC CAS and NAEP data: for DC CAS, we could only measure the gap by comparing percentages of students who scored proficient or above; for NAEP, we could only compare scale score performance. We could not determine performance gaps on DC CAS for students of low socioeconomic status, English-language learners, and students with disabilities because the city does not report data for the relevant comparison groups (e.g., students who are not in poverty, native English speakers, and students without disabilities)
|
From page 192... ...
CONCLUSION 6-5 The committee's evaluation was limited to a few blunt measures -- proficiency rates on standardized tests and high school graduation rates -- because of lack of data. To better understand out comes for D.C.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.