Assessing Accomplished Teaching: Advanced-Level Certification Programs (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

8 The Effects of the Certification Process on Practice The founders of the National Board for Professional Teaching Standards (NBPTS) anticipated that board certification would have a positive effect on the quality of teaching in this country. They envisioned that articulating the standards for accomplished teaching and recognizing teachers who meet these standards would result in large-scale improvements in the practice of teaching (Carnegie Task Force on Teaching as a Profession, 1986; National Board for Professional Teaching Standards, 1991). In these documents, the founders suggest that improvements will be realized by such mechanisms as making the standards available to teacher preparation programs and by having a growing cadre of board-certified teachers in schools throughout the country who can implement better practices and share their skills with other teachers. While the founding documents do not specifically envision that individual teachers’ practice will improve directly as a consequence of the certification process itself, more recent NBPTS publications make this claim. For instance, in “I Am a Better Teacher” (National Board for Professional Teaching Standards, 2001a, p. 1), the board states that “the certification process helps teachers improve their teaching.” Considering the time required by the assessment and the depth and complexity of the tasks involved, it seems reasonable to expect some impact on the practices of those who complete the process. While it is not typical to assume that simply taking a test would improve the skills the test intends to measure, the national board’s assessment is somewhat unique in this re- gard. Applicants for board certification are expected to analyze, dissect, and reflect on the lessons they include in their portfolio. The activities involved 182 

THE EFFECTS OF THE CERTIFICATION PROCESS ON PRACTICE 183 in preparing for the assessment and in assembling the portfolio itself may present candidates with an opportunity to develop and hone their teaching strategies. Therefore, in our evaluation framework, we investigate whether going through the NBPTS certification process could have an impact on a teacher’s practices and ultimately on student learning. The question that we take up in this chapter is whether there is any evidence that this occurs. Question 5: To what extent do teachers improve their practices and the outcomes of their students by virtue of going through the advanced-level certification process? Figure 2-1 shows where this piece of the evaluation fits within the committee’s framework, displaying our model of the ways a certification program for accomplished teachers might influence the teaching profession and the way our evaluation questions map onto this model. To respond to this aspect of the evaluation, we identified three subsidiary questions to investigate. Specifically: a. To what extent do teachers who go through the certification process improve their teaching practices and classroom climate, regardless of whether they become board certified? b. Do teachers who obtain board certification become more effective at increasing student achievement in ways that are evident in their students’ achievement scores? c. Do teachers have a greater impact on other student outcomes (e.g., higher student motivation, higher promotion rates) after they ob- tain certification than they did before they were certified? Our literature review revealed that little research has addressed these questions, and the evidence that is available is not conclusive. In this chap- ter, we discuss the available evidence on each of the subquestions and the limitations of the findings, and we propose ways to improve upon the ex- isting research base. We begin with Subquestion b because the studies that address this question were just discussed in Chapter 7. We then move to Subquestion a, for which there were two studies that objectively evaluated the impact of the process on teachers’ practices (Darling-Hammond and At- kin, 2007; Lustick and Sykes, 2006) and four survey-based studies that pro- vide self-reports from teachers about their perceptions of the impacts of the assessment on their practices (Indiana Professional Standards Board, 2002; National Board for Professional Teaching Standards, 2001a,d; Yankelovich Partners, 2001). We found no studies that addressed Subquestion c. Table 8-1 provides a summary of the studies discussed in this chapter. 

184 ASSESSING ACCOMPLISHED TEACHING TABLE 8-1  Professional Development Effects of the National Board- Certification Process Grades/ Content Study Area(s) Years Sample Size State(s) Studies of Impacts on Student Achievement Clotfelter, 3rd-5th; 1994-2004 All teachers in the NC Ladd, and reading, math state Vigdor (2006) Clotfelter, High school All teachers in the NC Ladd, and state Vigdor (2007b) Goldhaber 3rd-5th; reading, 1996-1999 All teachers in the NC and Anthony math state (2007) Harris and 3rd-10th; reading, 1999-2004 All teachers in the FL Sass (2006) math state Studies of Impact on Teachers’ Practices Darling- Middle and high 16 teachers, 9 CA Hammond and school; math and applicants and 7 Atkin (2007) science nonparticipants 

THE EFFECTS OF THE CERTIFICATION PROCESS ON PRACTICE 185 Issues Affecting Interpretations of Groups Compared Findings the Findings - NBCTs Effectiveness declined Standard errors did not - Future NBCTs during year of application. account for nesting; when - Current applicants Comparisons prior to and corrected, effect sizes may - Nonparticipants post certification showed not have been statistically mixed results. significant. - NBCTs Effectiveness increased from No concerns. - Future NBCTs precertification period to - Current applicants postcertification period. No - Nonparticipants decline was evident during the application year. - NBCTs Comparison of NBCTs at Standard errors did not - Unsuccessful various stages showed that account for nesting; when applicants future NBCTs were more corrected, effect sizes may - Nonparticipants effective than other teachers. not have been statistically Current applicants showed significant. a decline in effectiveness. During the first year of certification, NBCTs were more effective than other teachers. Results for 2+ years postcertification were inconsistent. - NBCTs Effectiveness declined Standard errors did not - Applicants during the various stages, account for nesting, but most - Nonparticipants postcertification effectiveness effects were not statistically never reached levels of significant. precertification effectiveness. Applicants and Teachers who went through Very small sample, produced nonparticipants the certification process in large part by attrition. No improved their formative information provided about assessment practices more the teachers who dropped out than the nonparticipants. of the study. Continued 

186 ASSESSING ACCOMPLISHED TEACHING TABLE 8-1  Continued Grades/ Content Study Area(s) Years Sample Size State(s) Indiana N/A 2001 71; responses from IN Professional 32 (48%) Standards Board (2002) Lustick and Science 2001-2004 188 teachers National Sykes (2006) (adolescent and young adult) NBPTS N/A September 10,700; responses National (2001a) 2001 from 5,641 (53%) NBPTS N/A Teachers 600; responses from National (2001d) board 235 (40 percent) certified between 1994 and 1999 

THE EFFECTS OF THE CERTIFICATION PROCESS ON PRACTICE 187 Issues Affecting Interpretations of Groups Compared Findings the Findings Only studied NBCTs Respondents said the process Small sample, low response made them more effective. rate. The emphasis on reflection helped them with their lesson planning. Pretest group = Teachers in the posttest Questions about the extent applicants before going group performed better than to which performance on the through the certification those in the pretest group on tasks generalizes to classroom process; Posttest group simulated NBPTS-like tasks practice. = applicants after going after going through the actual through the certification certification process. process Teachers who completed Teachers reported that the Sample included teachers who the certification process process helped them: were unsuccessful, but results - Develop stronger not reported for this group. curricular skills Sampling methodology is - Improve ways to evaluate questionable. No evaluation of learning the extent to which the sample - Improve interactions with represented the population. students Results described in an - Collaborate better with advocacy piece. Few details other teachers are provided. - Incorporate state content standards in teaching. Most reported it was a good professional development experience. Most thought it made them better teachers. Only studied NBCTs NBCTs reported that the Results reported in an process advocacy piece with few - Was better than any other details about the methodology. professional development No evaluation of the process representativeness of the - Had a greater impact than sample. receiving the credential itself - Positively affected their teaching practices - Caused them to become more reflective. Continued 

188 ASSESSING ACCOMPLISHED TEACHING TABLE 8-1  Continued Grades/ Content Study Area(s) Years Sample Size State(s) Yankelovich N/A Nov. 2000- 4,800; responses National Partners Jan. 2001 from 2,186 (45%) (2001) NOTE: N/A = Not available; NBCTs = National Board-Certified Teachers. Impact of Board Certification on Student Gains on Achievement Tests In Chapter 7, we discussed results from studies that compared achieve- ment test gains for the students of board-certified teachers who were at different stages in the certification process—before pursuing board certifi- cation, during the application process, and after becoming board certified. Goldhaber and Anthony (2005) and Clotfelter, Ladd, and Vigdor (2006, 2007b) examined these staged effects for teachers in North Carolina, and Harris and Sass (2006) examined them for teachers in Florida. Three of these studies (Clotfelter, Ladd, and Vigdor, 2006; Goldhaber and Anthony 2005; Harris and Sass, 2006) generally found that teachers who eventually earned board certification were more effective from the outset at increasing their student’s test scores than other teachers, but the studies did not provide evidence of improved effectiveness after becoming board certified. Moreover, some results implied that teachers were less ef- fective after attaining board certification than before. Results for the most recent study (Clotfelter, Ladd, and Vigdor, 2007b), which focused on high school students and teachers, were somewhat different, showing statistically significant differences (p < .05) when teachers’ effectiveness was compared before and after becoming certified. As discussed in Chapter 7, because of methodological differences among the studies, we conducted supplemental analyses to better under- stand the actual effects. The additional analyses used data for Florida and North Carolina, the only states that maintain longitudinal data that allow for such analyses, and focused on the grades, subject areas, and variables that were common to the two states. The results from our supplemental analyses were also inconclusive, some showing that teachers were slightly 

THE EFFECTS OF THE CERTIFICATION PROCESS ON PRACTICE 189 Issues Affecting Interpretations of Groups Compared Findings the Findings Only studied NBCTs Board certification increased Low response rate; their credibility in the no indication of how profession and made them representative the respondents feel more confident in their were of the sample. abilities. Board certification was associated with improved respect. more effective at increasing their students’ test scores after completing the process than before going through it and some showing no differences in pretest and posttest effectiveness (see Table 7-2). While the Florida and North Carolina data sets offer the advantage of providing longitudinal data with a wide array of variables and large sample sizes, the analyses are exploratory in nature. That is, they are based on comparisons of precertification and postcertification effectiveness, without an under­lying hypothesis about how or why improvements in effectiveness might occur and when they might become apparent. For example, it may be that teachers need a year or two to implement what they learn from the process, such that improvements in effectiveness would not be immediately apparent. Florida’s state data allowed for a preliminary exploration of this idea. In supplemental analyses with the Florida data, the researchers were able to split the postcertification stages into one year postcertification and two or more years postcertification. They conducted these analyses for reading and mathematics in three grade spans (elementary, middle, and high school). The results for mathematics showed a slight trend toward improvements in effectiveness across the two postcertification stages, but the increases were generally less than a .03 change in effect sizes (e.g., for high school mathematics the effect size was .00 during the first year post- certification and .03 afterward). The results for reading were smaller and not consistent across the three grade spans. Again, these analyses were exploratory, taking advantage of existing data systems. We think it would be useful for researchers to develop hy- potheses about what teachers might learn from the certification process, how they might implement these newly formed skills and practices, and when improvements in practice should become evident. This kind of re- 

190 ASSESSING ACCOMPLISHED TEACHING search would require a more focused, theory-based approach that begins with a conception of what teachers might learn and how they might incor- porate this information into their classroom practices, followed by the col- lection of data designed specifically to evaluate these hypotheses. We think that this approach should be considered in collecting evidence to evaluate the impact of teachers’ practices on student outcomes, as well as on other outcomes. Teachers’ practices Before and After NBPTS Participation Two studies took a different approach to evaluating the impact of the certification process on teachers by focusing directly on their performance. As with the studies just discussed, these two also do not provide definitive findings about improvements in teachers’ skills as a result of the certifica- tion process. However, we describe the approaches taken in each study because we think the methodologies have merit and should be considered in future research. Lustick and Sykes (2006) compared teachers’ performance before and after completing the assessment for the adolescent and young adult science credential. The study used simulated NBPTS-like portfolio exercises created by the researchers. To accomplish the data collection, the researchers sent each of the 118 participants, who had been randomly assigned to treatment groups, a sealed packet containing the exercises. The researchers conducted a phone interview during which each teacher opened the sealed packets and responded to the exercises. The exercises consisted of five assignments. Two of them asked teachers to describe their own experiences, much like what is required on the actual assessment. Three assignments involved reviewing materials that the researchers sent, including a videotaped lesson, a written scenario of a lesson, and a sample of student work. Their responses were recorded, transcribed, and scored by trained NBPTS assessors following the standard NBPTS rubric. For this study, teachers were assigned randomly to either a pretest or a posttest group, with the pretest group taking the simulated assessment prior to going through the actual board-certification process and the post- test group participating in the study afterward (but before receiving results from the NBPTS). Analyses revealed that the posttest group scored statisti- cally significantly (p < .05) higher than the pretest group. The authors have since accumulated information on the actual assess­ ment results for the participants (personal communication with first author, February 2, 2007). As is typical for the general pool of all NBPTS ap- plicants, about half of the teachers in each group passed. The researchers compared the performance of unsuccessful teachers in the pretest and in 

THE EFFECTS OF THE CERTIFICATION PROCESS ON PRACTICE 191 the posttest group with the performance of successful teachers in the pretest and posttest groups. Analyses indicated that both teachers who passed and teachers who failed showed gains on the simulated exercises, suggesting that even teachers who are not successful learn from the process. This study, while methodologically sound, focused on what teach- ers would do under certain hypothetical situations, but it did not evalu- ate teachers’ performance in the classroom. Darling-Hammond and Atkin (2007) examined teachers’ actual classroom practices. This study focused on the impact of the certification process on the classroom practices of middle and high school mathematics and science teachers. The authors recruited teachers interested in becoming board certified and randomly assigned them to two groups. The NBPTS group went through the certifi- cation process during the time period of the study, while the comparison group postponed their application for board certification. The researchers followed the teachers for three years and collected data at specific stages of the application process: one year prior to pursuing board certification, the year of candidacy, and the year after candidacy. Attrition became a significant issue for this study, however. About 60 teachers initially expressed interest in participating, but in the end only 16 completed the study and only nine went through the certification process. This attrition rate is so high that it calls into question the validity of the findings, but we highlight the study because the approach is one that we encourage. For this study, teachers submitted videotapes of lessons, written re- sponses to questions about the videotaped lessons, and student work sam- ples from the unit that was videotaped. They also participated in interviews and surveys about their assessment practices. The focus of the collected information was on the ways that teachers use formative assessment prac- tices in their daily teaching, including the types of assessments, the use of assessment results for planning instruction, and the feedback given to students. The researchers designed a rubric that was used to score the sub- mitted information. The researchers found that the NBPTS group used a wider range of assessment methods and questioning strategies in class discussions that elicited more complete explanations from students. They were better able to integrate their assessments with ongoing instruction. Overall, the NBPTS group showed consistent improvement in their assessment practices during the course of the study. It is not known what effect attrition had on the findings from this study. Some of the teachers dropped out of the study after the first or second year, and partial information was collected on them. The authors did not report any results for these teachers, however, so we cannot evaluate how this af- fected the results. For example, it is not known if the teachers who dropped 

192 ASSESSING ACCOMPLISHED TEACHING out tended to be systematically weaker or stronger than other teachers in the group (the first author did not respond to our queries about this). For this reason, the committee thinks that no strong conclusions can be drawn from this study. Nonetheless, the researchers’ focus on actual classroom practice and the kinds of information they collected permitted a rich set of analyses. The approaches used in these two studies are the kinds of strategies that we encourage. We think these methodologies, when carried out in a scientifically sound way with sufficient numbers of appropriately selected participants, are likely to be the most fruitful ways to investigate the im- pact of the certification process on teachers. In our judgment, these studies suggest that teachers learn from the certification process, but the evidence base at this point is simply too thin to draw any firm conclusions from these findings. What Do Teachers Report About their Experiences? As our review makes clear, there is very little empirical evidence about the impact of the certification process on teachers’ practices and effective- ness. In the absence of studies that objectively evaluate the effects of the process on teachers’ practices, we turned to other sources. One way to find out if teachers learn from the experience is simply to ask them. Our literature review identified four survey-based studies that asked teachers about their experiences after completing the certification process. Three were large-scale national surveys sponsored by the NBPTS (National Board for Professional Teaching Standards, 2001a,d; Yankelovich Partners, 2001), and one was a small-scale survey of board-certified teachers in Indiana (Indiana Professional Standards Board, 2002). The findings from these surveys were generally positive. Teachers tended to report that the certification process was a worthwhile professional development activity that improved their teaching practices and stimulated them to become more reflective, a practice that is encouraged in the NBPTS standards. However, these surveys were not conducted in a way that would best address the questions in our evaluation framework. What is needed to evaluate the impact of the certification process on teachers is to collect data on a pretest and posttest basis to compare responses before and after going through the process. These surveys collected information only after teachers had successfully completed the process. Thus, while they provide some basic information about teachers’ perceptions of the effects of the process, the results cannot be used to make inferences about changes in their practices attributable to the process itself. There were additional limitations to these studies. Results from the survey of board-certified teachers in Indiana were based on a sample of 

THE EFFECTS OF THE CERTIFICATION PROCESS ON PRACTICE 193 only 32 teachers (Indiana Professional Standards Board, 2002). Details about the NBPTS-authored studies are lacking. The results from two of the surveys (National Board for Professional Teaching Standards, 2001a,d) are presented in advocacy pieces intended to promote the program, and the discussion of the findings is neither fully detailed nor objective. The report on the third survey (Yankelovich Partners, 2001) consists of only a tally of the survey responses with little or no discussion of the methodology or findings. These shortcomings made it impossible to draw independent judg- ments about the validity of the findings. We would like to have conducted our own survey or possibly a more in-depth study that evaluated teachers’ perceptions of the process before and after going through it, but we had neither the time nor the resources for such an undertaking. Instead, we arranged for several small-scale focused conversations to follow up on findings reported in these surveys. As noted in Chapter 2, we held a structured discussion with teachers who serve on the National Research Council’s (NRC’s) Teacher Advisory Council, of which four were board-certified, and we organized a panel discussion with three board-certified teachers at one of our meetings. The teachers involved in these discussions were not intended to be representative samples of any kind but simply to provide an opportunity for committee members to hear teachers discuss the NBPTS process and to ask them questions about its effects on their teaching. The findings from these discussions generally confirmed those reported in the surveys. The board-certified teachers who spoke with us made posi- tive remarks about the process. They thought it was a worthwhile pro- fessional development experience that had significant impacts on their teaching. They said that after becoming board certified, they tended to adopt the reflective practices toward teaching that the board endorses. The remarks made by one of the panelists, Sara Eisenhardt, a board-certified elementary teacher in Cincinnati, Ohio, are typical of those made by other board-certified teachers: Sara Eisenhardt commented that one consequence of going through the board-certification process is that she learned to be more “strategic” in her teaching. She said that the focus on reflective teaching was critical and opened her eyes to new teaching methodologies. Before participating in the program, she had not thought of teaching in this way and had not learned how to discuss student work and to reflect on how effective her instruc- tion had been. As a result of the national board process, she learned to be more focused in her lesson planning, to better evaluate students’ learning and how well the lesson went, and to strategically plan for the next day’s lesson. She is now actively involved in support programs for teachers go- ing through the process and in revamping the school system’s professional development activities for teachers. 

194 ASSESSING ACCOMPLISHED TEACHING In our judgment, the survey results combined with our own conversa- tions with teachers provide some evidence, albeit weak, that teachers who have successfully earned board certification find the process to enhance their practices. Finding 8-1: Self-report information from teachers who have successfully completed the board-certification process indicates that they tend to be pos- itive about the experience. Board-certified teachers report that the process provides a professional development experience for them and has positive influences on their teaching practices, helping them become reflective of their teaching and their instructional decisions. However, no empirical re- search has yet been conducted to corroborate this self-report information. Conclusions and recommendations It is difficult to draw any firm conclusions from the research discussed in this chapter. The survey results and our own discussions with teachers suggest that teachers learn from the process, but we cannot ignore the fact that this information is entirely subjective. For one thing, it is limited to teachers who were successful in their attempt to earn board certification. It is not surprising that individuals who successfully complete a lengthy and difficult process feel positive about it, and it would be quite worrisome if these teachers reported negative perceptions of the process. In addition, the surveys were conducted only after teachers earned their credentials, making it impossible to evaluate changes in their attitudes, ideas, and perspectives as a consequence of going through the process. Also, we were particularly reluctant to draw conclusions from the four surveys discussed in this chap- ter because of methodological problems associated with each one. Nevertheless, we see an important role that survey data could play in addressing this aspect of the evaluation framework. First, surveys should be conducted both before and after the process, either by tracking the same group of teachers over time or by gathering pretest and posttest data from equivalent groups. Second, teachers who did not pass on the first attempt need to be included, and their results should be examined separately. We encourage the NBPTS and other researchers to conduct such sur- veys. The board could easily implement such data collections as a routine part of the testing process, as is done by many other testing organizations. Test-takers could be required to respond to a questionnaire at the time they register for the assessment and again after completing the assessment center exercises. This is a straightforward way to collect a wealth of information. Other data collection efforts could be focused at later stages of the process, such as shortly after receiving the assessment results, a year later, and so on, and they could be combined with data collections to address other aspects 

THE EFFECTS OF THE CERTIFICATION PROCESS ON PRACTICE 195 of our evaluation. This kind of data collection need not be on the full popu- lation of test-takers but a carefully selected random sample. In developing these questionnaires, conducting these surveys, and reporting the results, we encourage the NBPTS to consult with experts to avoid the methodological problems associated with their past surveys (see Appendix A). We also hesitate to draw firm conclusions about the effects of the certi- fication process on teachers’ practices. The findings from Lustick and Sykes and from Darling-Hammond and Atkin are suggestive, but each study has limitations. Together they represent a first step in such investigations, but more research is needed before definite conclusions can be made. We think that this line of research holds promise, however, and encourage research- ers and the NBPTS to pursue it. We recognize that such studies are difficult undertakings and recruiting participants can be a challenge. Nevertheless, studies such as these allow for more in-depth analyses of teachers’ prac- tices and a better understanding of any impacts of the board-certification process. At this stage, we cannot say whether any learning that teachers acquire from the process translates into higher achievement test scores for their students. The findings from existing studies are contradictory. Despite the fact that the most recent Clotfelter, Ladd, and Vigdor (2007b) paper found improvement in teachers’ effectiveness, we note that the improvement is not large, and it is likely that any positive effects reported in future studies will also show only small effects. There may be several reasons for this. First, it is important to remember that the national board did not set out to develop a means for raising students’ test scores. The board laid out a mechanism for professionalizing teaching, one consequence of which was to improve teachers’ practice, thereby improving student learning. The board’s discussions of accomplished practice suggest the kinds of behaviors that advanced-level teachers may demonstrate. They suggest that teachers who become board certified may be better able to engage their students. Their lessons may be more structured and better focused. They may provide more appropriately geared lessons that build on their students’ experiences and interests, and they may learn to continually adjust their lessons to meet their students’ needs. As a result of board certification, teachers may develop a new enthusiasm for their teaching and thus better stimulate their students. If these are the sorts of skills that teachers develop, however, they may not manifest themselves as higher scores on standardized achievement tests. Standardized achievement tests generally measure a fairly narrow set of skills and knowledge. They are typically paper and pencil tests that require students to demonstrate what they know through written responses. Some students are disadvantaged by these assessments and would provide more in-depth information about what they know through interviews or other, 

196 ASSESSING ACCOMPLISHED TEACHING more hands-on mechanisms. Standardized tests often do not assess higher order critical thinking skills and cannot measure enthusiasm for learning. Furthermore, as we laid out in Chapter 7, there are many methodological issues associated with conducting studies that focus on students’ standard- ized test scores that may depress potential effects. In our judgment, the existing research does not provide documenta- tion that the certification process enhances teachers’ skills at improving students’ achievement test performance. Neither does it refute this claim. At this stage, the available research focuses only on reading and mathematics achievement in two states and primarily in the elementary grades. These studies are exploratory in nature and not based on a theory of the ways in which the certification process might impact teachers’ effectiveness or when these impacts are likely to be evident. We think it is premature to conclude that these findings would generalize to all other states, content areas, and grades. We think there is a need for replication of these studies in other states, grades, and content areas, but we do not want to see all resources invested in such studies. We therefore encourage multiple avenues for research. Large-scale studies that use standardized test performance as the out- come measure are relatively easy to conduct when these tests are routinely given to all students and the data are maintained in state databases. These resources offer an efficient means for conducting such studies. However, such studies should not be the only kind conducted. We encourage re- searchers to find ways to evaluate students’ performance on other measures that are more aligned with the skills that the board emphasizes, such as assessments of critical thinking skills or evaluations of students’ attitudes toward a given subject. These studies combined with surveys and research that directly evaluates teachers’ classroom practices should provide a more complete picture of the impacts of the certification process. We therefore make the following recommendations for additional research: Recommendation 8-1: We encourage the NBPTS and other researchers to undertake research to investigate the effects of the process on the candi- dates. The studies should use pretest-posttest and longitudinal designs and should allow for comparison of responses from successful and unsuccessful candidates. Recommendation 8-2: We encourage the NBPTS and other researchers to pursue more mixed-method studies, using both quantitative and qualitative research methods, to examine the effects of board certification on teachers’ practices. These studies should examine a variety of measures and a variety 

THE EFFECTS OF THE CERTIFICATION PROCESS ON PRACTICE 197 of student outcomes. Such research should be conducted using sound meth- odologies, adequate samples, and appropriate statistical analyses. Recommendation 8-3: Researchers should work with the NBPTS to obtain the information needed to study the relationships between board certification and student achievement across the various stages of board certification. These studies should examine the impacts of the certification process on teachers’ effectiveness in increasing their students’ test scores and specifically should examine effects for the years subsequent to the receipt of board certification. To the extent that existing data sets allow, we encourage replication of studies in states besides North Carolina and Florida and in subjects beyond elementary reading and mathematics.