An Evaluation of the Public Schools of the District of Columbia Reform in a Changing Landscape (2015) / Chapter Skim
Currently Skimming:
4 Improving Teacher Quality
4 Improving Teacher Quality
Pages 79-118
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 79... ...
placed a high priority on improving teacher quality. The chancellor and her team pursued this goal by implementing a new teacher evaluation system, IMPACT, for the District of Columbia Public Schools (DCPS)
|
From page 80... ...
IMPACT consists of a teaching and learning framework to guide teachers, multiple evaluations based on observations of teachers' practices and measures of student achievement, feedback and coaching mechanisms, a system of incentives (rewards and sanctions) , and continuing professional development opportunities.
|
From page 81... ...
The fourth section discusses measures to improve principal quality. In the final section we offer our suggestions for improvements to IMPACT and other aspects of measuring and improving teacher quality in D.C.
|
From page 82... ...
. At the same time, the need for improvements in teacher evaluation -- a key mechanism for identifying highly qualified, highly effective teachers who are likely to improve educational outcomes for students -- has received increased attention.
|
From page 83... ...
In addition, the city has developed a pool of instructional coaches, who provide support and feedback to teachers and the school leadership (District of Columbia Public Schools, 2014d)
|
From page 84... ...
The evaluations provide a means for gathering evidence to support decisions about teacher compensation and employment. Through IMPACT, various types of information are collected and used to assign teachers to one of five possible effectiveness categories: "highly effective," "effective," "developing," "minimally effective," or "ineffective." The incentives are primarily monetary, but they also include advances on the career ladder (discussed below)
|
From page 85... ...
NOTES: CP = core professionalism; CSC = commitment to the school community; IVA = individual value-added; TAS = teacher-assessed student achievement data; TLF = teaching and learning framework, which serves as a guide to observation protocols and rubrics; n.a. = not applicable because not in effect for the given year.
|
From page 86... ...
General education teachers of math and of reading and English-language arts in grades 4 through 8 receive an "individual value-added" estimate, since standardized achievement test scores are available in these subject areas and grades.5 For other teachers, an alternative measure called the "teacherassessed student achievement" estimate is calculated. Scores on the components are weighted unequally, as shown in Table 4-1, summed, and then classified into the performance categories.
|
From page 87... ...
Score distributions show that the majority of teachers receive ratings of effective or highly effective (Gitomer et al., 2014)
|
From page 88... ...
The third superintendent distinguished between principals who are primarily managers, who prioritize building operations over instruction, and those who view themselves as instructional leaders whose primary role is to help teachers improve instruction. According to this superintendent, principals of the latter type are likely to be more diligent in making teacher observations and to provide more constructive feedback because they view the observations as a tool for growth.
|
From page 89... ...
. Although classroom observation instruments and rubrics to assess other elements in teaching, such as lesson planning, have been a research focus for decades, comparably little research has examined how to assess and measure teacher involvement outside of classroom instruction (Gitomer et al., 2014)
|
From page 90... ...
The school administrator rates teachers twice annually, on a time schedule like that for the commitment measure, and ratings are based on the frequency with which certain behaviors are observed. Teachers rated as slightly or significantly below standard are subject to deductions from their total effectiveness scores, ranging from 10 to 20 points.
|
From page 91... ...
and the validity and fairness of using them to evaluate teachers. Some argue that VAM produces reliable, objective, quantified measures of a teacher's impact on student learning (e.g., Gordon et al., 2006; Glazerman et al., 2010; Hanushek and Rivkin, 2010; Bill & Melinda Gates Foundation, 2012; Kane et al., 2013; Chetty et al., 2014a)
|
From page 92... ...
However, we think it is important to note that the revised edition of the Standards for Educational and Psychological Tests (American Educational Research Asso ciation et al., 2014) lays out guidelines for using student achievement test scores to evaluate teacher effectiveness, and it specifically states that this use requires a validity argument should be set forth to justify inferences about [the value-added estimates]
|
From page 93... ...
. Classroom-level factors: • the class's average test score in the same subject from the previous year; • the standard deviation of the class's scores in the same subject from the previous year; and • the proportion of students eligible for free or reduced-price lunch.
|
From page 94... ...
Teacher-Assessed Student Achievement Scores As we note above, an individual value-added score cannot be calculated for all teachers. For the teachers for whom a value-added score is not calculated, DCPS calculates a teacher-assessed student achievement score, as is done in some other states (often called student learning objectives)
|
From page 95... ...
The teacher-assessed student achievement component in IMPACT is similar to assessments of student learning objectives used in other states when test score data are not available for groups of teachers. However, D.C.
|
From page 96... ...
. For instance, as shown in Table 4-1, there have been numerous changes in the weights assigned to the component scores.9 The changes have been substantive enough to render the overall IMPACT scores incomparable from one year to the next.
|
From page 97... ...
. The majority of teachers have received ratings of highly effective or effective.
|
From page 98... ...
, 68.8 percent received ratings of e ffective or highly effective, and 28.2 percent received ratings of minimally effective or ineffective. For group 2 (those with only a teacher-assessed student achievement score)
|
From page 99... ...
These resources include (District of Columbia Public Schools, 2012) • professionally produced lesson videos from DCPS classrooms; • curricular supports for the Common Core State Standards; • a professional development planner, an online catalog of profes sional development opportunities;
|
From page 100... ...
Annual performance determines the extent to which teachers can advance on the DCPS career ladder, and advances can lead to additional compensation as well as a reduction in the number of classroom observations required.10 Also, teachers rated as highly effective may choose to participate in additional leadership opportunities. Low performance ratings result in a range of employment sanctions.
|
From page 101... ...
Two highly effective ratings are needed to become distinguished, and then two additional highly effective ratings would be needed to become expert. According to the guide (District of Columbia Public Schools, 2014d)
|
From page 102... ...
The second component of the compensation structure is the IMPACTplus program. Teachers who receive ratings of highly effective qualify for annual bonuses that are separate from their base salary compensation.
|
From page 103... ...
would lead to increases in the overall quality of the teaching force. That is, the financial bonuses should have provided the incentive for highly effective teachers to remain in DCPS, while the mandatory dismissal of teachers with low performance ratings should have removed the low-performing teachers.
|
From page 104... ...
I As the figure shows, over 80 percent of teachers classified as highly effective or effective chose to stay with DCPS. The retention rate among teachers just in the highly effective category was 89 percent and did not change over the 3 years.
|
From page 105... ...
. Figure 4-1 R02838 100 75 Percentage Retained 25 50 0 2010-2011 2009-2010 2011-2012 IMPACT Year Ineffective Minimally effective Effective Highly effective FIGURE 4-2 Retention rates of teachers in DCPS, by effectiveness category and school year.
|
From page 106... ...
Retention and Leaving Rates by School It is also useful to examine teacher retention rates by school. There may be important differences in the rates at which schools retain the most effective teachers that could lead to inequities across schools.
|
From page 107... ...
To compensate for this problem, the researchers created a "core" group of teachers, and, for each year, the effectiveness levels of the entry and exit cohorts were compared to this core group. 100 Percentage of Schools 75 50 25 0 2009-2010 2010-2011 2011-2012 School Year 0 to 40 percent of teachers retained 40 to 60 percent of teachers retained 60 to 80 percent of teachers retained 80 to 100 percent of teachers retained FIGURE 4-4 Schoolwide retention rates of effective and highly effective DCPS teachers, by school year.
|
From page 108... ...
As a group, new hires were less effective than core teachers by a similar amount in all 4 years. Across the 4 years shown, newly hired teachers obtained IMPACT scores that were, on average, between 26 and 33 points lower than those of the core group (row 1)
|
From page 109... ...
The analysis focused on data for the years 2009-2010 through 20122013, comparing the average IMPACT scores obtained by teachers in each ward. For the analysis, the EdCORE researchers used eligibility for free and reduced-price lunch as a measure of socioeconomic status (SES)
|
From page 110... ...
. Figure 4-5 R02838 Mean IMPACT Score School Years FIGURE 4-6 Mean IMPACT scores for teachers citywide and by ward, 2009-2010 through 2012-2013.
|
From page 111... ...
In other words, the data show an association between high concentrations of poverty and low IMPACT scores, although this analysis does not permit causal inferences. One possible explanation is that high-poverty schools are not able to attract teachers with the highest IMPACT ratings (i.e., teachers Mean IMPACT Score FIGURE 4-7 Mean IMPACT scores for teachers by school-level socioeconomic status, for 2009-2010 through 2012-2013.
|
From page 112... ...
. The analysis focused on two sets of teachers, those whose IMPACT scores placed them on the cusp between minimally effective and effective and those whose scores placed them on the cusp between effective and highly effective.
|
From page 113... ...
PRINCIPAL QUALITY Along with the implementation of a program to improve teacher quality, Chancellor Rhee also sought to improve principal quality. One of her first reforms was to replace school principals who were performing poorly, as measured by their students' achievement on standardized tests.
|
From page 114... ...
But, as with estimating the value added by a teacher, estimates of the value added by the school may be confounded by factors that cannot be quantified or controlled in the valueadded approach. SUMMARY AND CONCLUSIONS Improving Teacher Quality To improve the quality of teaching that D.C.
|
From page 115... ...
CONCLUSION 4-1 DCPS officials defined a three-pronged approach to improving teacher quality: clarify performance expectations, provide quality feedback and support to teachers, and retain the most effective teachers. The design of the IMPACT teacher evaluation system and the associated implementation plan are generally consistent with current research on teacher evaluation systems.
|
From page 116... ...
The committee recognizes that systematic evaluation is difficult and somewhat uncommon, but given the novel nature and potential unintended consequences of IMPACT, DCPS could benefit from careful assessment of its effects and, more generally, of the characteristics of the educator workforce. DCPS placed a high priority on improving the quality of the teacher workforce, under the premise that improving teacher quality would lead to improved conditions and outcomes for students, particularly students who have traditionally been underserved.
|
From page 117... ...
CONCLUSION 4-5 The city needs a plan for gathering evidence to evaluate the extent to which the intended inferences from IMPACT are supported, particularly with respect to the improvement of teaching in schools serving lower-achieving students.* The city needs to undertake additional data collection and analysis to fully understand teacher quality, teacher evaluation, and teacher supports in the charter school sector as it seeks to understand and improve the quality of teaching for all public school students.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.