Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
2 Context and Motivation Doctoral education is at the heart of the U.S. system of innovation. It is the process that generates highly educated scholars and researchers, significant research results, and avenues for innovation, thereby creating the leaders needed to produce the research advances that will create new careers and economic vitality for the nation.1 Doctoral education trains the professors of the futureâit inculcates the habits of mind necessary for productive research and scholarship. Doctoral education is intimately involved in the creation of scholars whose ideas will shape both future innovations and how Americans use and understand innovation as it changes their lives. American graduate education draws students from across the United States and around the globe, particularly in the science, technology, engineering, and mathematics (STEM) fields, and has been the envy of the world since World War II. Now, however, the U.S. position is facing substantial challenges, from a growing emphasis on doctoral education in other countries to financial constraints stemming both from the economic downturn of 2008â2009 and from the continuation of declining trends in state support for higher education.2 Several reports have highlighted the threats to U.S. leadership in innovation, including recently from the National Research Council, Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future, which focuses especially on the need to improve U.S. graduate programs in STEM fields in order to improve U.S. economic competitiveness. The need for enhanced performance and accessibility is also highlighted in the recent report from the Council of Graduate Schools Graduate Education: The Backbone of American Competitiveness and Innovation.3 These and other reports lay out clear frameworks for a focused commitment to improving graduate education. As additional resources are being considered for graduate programs, it becomes increasingly important to have structures in place to continually assess these investments. In addition to international competitive forces, strong drivers in the United States are underlying efforts to improve the quality and efficiency of graduate programs. For 1 See, for example, National Research Council, Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future (Washington, D.C.: National Academies Press, 2007), chap. 1. 2 A report issued in May 2010 by the Commission on the Future of Graduate Education, The Path Forward: The Future of Graduate Education in the United States, May 2010, eloquently lays out the importance of graduate education. To read the report, visit www.fgereport.org/rsc/pdf/CFGE_report.pdf. Accessed July 9, 2010. 3 Council of Graduate Schools, Graduate Education: The Backbone of American Competitiveness and Innovation (Washington, D.C.: Council of Graduate Schools, 2007). 17
18 A DATA-BASED ASSESSMENT OF RESEARCH-DOCTORATE PROGRAMS IN THE U.S. one thing, public universities are experiencing a sustained decline in state support that is forcing institutions to increase tuition and raise funds privately, thereby mirroring some of the features of private universities. For both public and private universities, doctoral education is expensive in the commitment of time and dollars both by those engaged in the enterprise and by its funders. Thus an assessment of program effectiveness to weigh the justification of that investment is always necessary. Finally, efforts to determine whether doctoral education is living up to its promise call for an evaluation of whether it has done so by expanding domestic sources of talent, improving time to degree, and raising rates of completion. These are just some of the challenges that this study has attempted to address in view of the fact that few previous studies have been able to investigate these challenges as thoroughly. WHY ASSESS DOCTORAL PROGRAMS? The assessment of doctoral programs dates back to 1925, when Raymond M. Hughes first conducted a survey to gauge faculty opinion of âthe esteem at the present time for graduate work in your subject.â4 His survey, which appeared in a report to the Association of American Colleges, was aimed at constructing rankings of doctoral programs. The results were greeted with both interest and criticism. Since then, however, reputational measures have been repeatedly used to assess the quality of doctoral programs. In the more than 80 years since the Hughes report, doctoral education has changed tremendously in size, number of fields, and the nature of employment destinations of Ph.D. recipients. The nature of assessing doctoral programs has changed as well, from reputational rankings provided by department chairs to studies that have increasingly included objective measures of aspects of doctoral programs.5 Today, a similar kind of reappraisal, but with a different motivation, may be warranted. With the enormous importance of and investment in doctoral education comes the need for accountability, because many different sectors of the U.S. economy rely heavily on the quality of knowledge produced by the nationâs Ph.D.âs. Colleges and universities across the United States and around the world rely on American doctoral programs to educate the next generation of faculty and professional researchers. Corporations depend on highly trained doctoral students ready to bring cutting-edge technology and science to their labs and offices. Federal agencies also invest considerable sums of money to support doctoral students as fellows, trainees, and research assistants, 4 Quoted in National Research Council, Research Doctorate Programs in the United States, 10. 5 For this discussion it is important to recognize the distinction between reputational measures and those called âobjective measuresâ in this report. Reputations of program quality are derived, here and in the past NRC studies, from respondentsâ ranking of Ph.D. programs on a six-point scale from distinguished to poor, which includes one category that indicates that a respondent does not have enough knowledge of the program to rate it. These data are quantitative, objective, and measurable, just like the Likert-type scales that have been used in the social and behavioral sciences for decades. Objective measures, as used here, refer to measurements based on data derived from sources that yield faculty publication counts, citations of their work, and honorific awards, as well as measures of student support and outcomes and program diversity. These kinds of measured data may partially predict the reputational standing of a program. But reputations, as a composite subjective assessment translated into a score on a scale, may capture other elements of program quality that cannot be obtained by means of the objective measures used here.
CONTEXT AND MOTIVATION 19 as do private foundations. The provision of information for benchmarking and improvement is salient in all these sectors. For students considering a doctorate, the importance of accountability is no less striking. The decision to enroll in a doctoral program represents an enormous personal commitment. And the selection of a doctoral program is a life choice of great importance. How effective is a particular program in graduating its students in a timely way? What is the reputation of the program? What are its particular strengths and weaknesses? What kind of financial support will be available? What benefits are available for students with families? What kind of record does the program have in attracting, supporting, and graduating underrepresented students? Is the program successful in recruiting and supporting women in traditionally male-dominated fields? How do its graduates fare in the world? It is important that students considering doctoral education pose such questions and that there be places where they can find reliable answers to them. The availability of data that are comparable across similar programs can serve as a guide to areas that need action and thus the collection of such data was a goal of this study. Still in use today, the traditional measures for assessment of doctoral education have been time to degree and completion rates.6 The shift toward including student opinion in perceived learning outcomes did occur until recently.7 Additional measures used include race and gender diversity, test scores, financial support of students, percentage completing, relationship with mentor, and overall socialization. These measures were addressed in the student questionnaire. No one source, of course, can answer all questions about all doctoral programs for all prospective doctoral students, funding agencies, or university administrators. But one important purpose of this NRC study is to make a very large amount of informationâarranged in as manageable a form as possibleâavailable to those with a variety of interests: to students facing such choices and asking such questions, to agencies and government bodies and foundations that invest heavily in doctoral education, and to universities that must manage their own doctoral programs effectively. Even when the NRC study findings cannot answer all the important questions that the many constituencies of doctoral education will bring to it, the study will put them in a better position to know the questions that they then need to pose to the programs they are considering. As this committee understands, not only must an enterprise of this significance be operated effectively, but also constituencies crucial to the support of doctoral education must have access to the information that can help provide reliable assessments of its effectiveness. Likewise, using such information, policy makers must be sensitive to the changing characteristics, or evolution, of doctoral education, because such changes are 6 William G. Bowen and Neil L. Rudenstine, In Pursuit of the PhD (Princeton, N.J.: Princeton University Press, 1992); J. Gravois, In humanities, 10 years may not be enough to get a Ph.D., Chronicle of Higher Education, July 27, 2007. 7 Council of Graduate Schools, PhD Completion and Attrition: Analysis of Baseline Program Data from the PhD Completion Project (Washington, D.C.: Council of Graduate Schools, 2008); B. E. Lovitts, Leaving the Ivory Tower: The Causes and Consequences of Departure from Doctoral Study (New York: Rowman and Littlefield, 2001).
20 A DATA-BASED ASSESSMENT OF RESEARCH-DOCTORATE PROGRAMS IN THE U.S. likely to be a consequence of the prominence of doctoral education in the national system of innovation. These changes include the increasing interdisciplinarity of U.S doctoral programs. As a result, the committee went to great lengths to try to capture this complex variable. DIVERSITY OF FACULTY AND STUDENTS IN DOCTORAL PROGRAMS An area of importance in assessing doctoral programs is the demographic characteristics of doctoral students. These characteristics include their international diversity, as well as their race, ethnicity, and gender. U.S. doctoral programs have attracted students from around the world for many years. These programs are also striving to become more diverse in race, ethnicity, and gender, and to some extent they are succeeding. International Students The number of international students pursuing doctoral programs in the United States has grown significantly since the 1990s. According to the Institute for International Education, the absolute numbers of enrollments of international doctoral students increased from 100,092 in 2003â2004 (academic year) to 108,976 in 2007â2008.8 Graduate applications overall, however, moved in the opposite direction. International graduate applications for the 2003â2004 academic year dropped suddenly and sharply. Although this brief downward trend now appears to have slowed or stopped, the decline was sharp enough that graduate applications and new enrollments have not yet returned to pre-2003 levels.9 One cause of this reversal in growth was the sensitivity of the international graduate application process to perturbations in visa policy and practices. Measures put in place after the terrorists attacks of September 11, 2001, not only made entry into the United States for study more difficult, but may also have had a chilling effect on the interest of international students in pursuing graduate study in the U.S. universities. Compounding the issue was an escalation in the level of competition worldwide for the best international doctoral students. For example, the European Union nations have recognized how important the knowledge and skills developed through doctoral education are to building a twenty-first century economy, and so those nations have given high priority to strengthening the doctoral education they offer. And China, which has provided large numbers of superb doctoral students for U.S. universities for the last half- century, has introduced ambitious programs to expand and strengthen doctoral education in its own universities. Furthermore, countries that provide doctoral study in English, such as Great Britain, Canada, and Australia, capitalized on the situation in the United States by moving rapidly to recruit more international students. Students from other countries enrolled in the entire range of doctoral programs were surveyed for this study. These students come from all over the world, but the 8 Institute for International Education, Open Doors (Washington, D.C.: IIE, 2003â2007). 9 N. Bell, Findings from 2009 CGS Graduate Admissions Survey, Phase III: Final Offer of Admissions and Enrollment (Washington, D.C.: Council of Graduate Schools, 2009).
CONTEXT AND MOTIVATION 21 number of students from India, China, and South Korea are particularly high. International enrollments are especially high in doctoral programs in engineering and the physical and mathematical sciences. Indeed, it is not unusual for major Ph.D. programs in engineering to award half or more of their doctorates to students from other countries. U.S. research universities have benefited greatly from the influx of doctoral students from other nations. While enrolled, these students characteristically show higher than average completion rates and shorter than average times to degree.10 Their contribution to laboratory research in the STEM fields is enormous. In fact, the research productivity of U.S. universities is closely tied to their ability to recruit and retain talented students who come to this country to pursue doctoral study. Many successful international doctoral students stay in the United States. By becoming university faculty, by establishing start-up companies, and by contributing to the research enterprise of corporate America, these international Ph.D.âs are a powerful component of the research engine that fuels the American economy. When international graduates return to their country of origin, they take back with them an understanding of American culture and values that is important in clarifying and stabilizing the place of the United States in the global political and economic culture. Similarly, domestic students gain a more global perspective and benefit from collaborations with graduate students from other countries and are thus better prepared for the global workplace they will encounter after graduation. Overall, international recipients of American doctorates play important roles in the educational, social, political, economic, and cultural infrastructures of many countries. In the increasingly global arena of high-level research, the U.S. capacity to develop international research partnerships is greatly strengthened by the presence of former students from American universities in key positions in laboratories and universities around the world. In short, the appeal of the U.S. doctorate to students in other countries is one of its great, essential strengths. That importance is borne out by the data collected for this study. The continued success of the United States in this increasingly competitive arena, which is crucial to sustaining the excellence of U.S. doctoral education, is the responsibility of research universities, of state and federal government policy makers, of powerful funding agencies and foundations, and of all other stakeholders in the American doctorate and the vast research enterprise that depends on it. Race, Ethnicity, and Gender of American Doctoral Students Doctoral programs across the nation have recognized the implications of powerful demographic trends in the general growth of the population and in college graduation rates that shape the educated workforce in an increasingly knowledge-based economy. Growth rates in the underrepresented minority population outstrip those of the majority, and more women now are awarded bachelorâs degrees each year than men. Both of these trends point to a change in the composition of participants in doctoral education in the United States. 10 Council of Graduate Schools, PhD Completion and Attrition.
22 A DATA-BASED ASSESSMENT OF RESEARCH-DOCTORATE PROGRAMS IN THE U.S. Through the 1970s doctoral education in almost all fields was largely a manâs enterprise, and the number of underrepresented minorities pursuing Ph.D.âs was very small. It became clear, however, that if doctoral education was to serve the population equitably, and if domestic doctoral production was to adequately meet the research and professorial needs of the nation, universities must increase the participation in doctoral education of underrepresented minorities and women. But success in this effort would require effective strategies that would address a range of cultural and historical forces that had long distributed educational opportunities unevenly across the population. In response to this situation, many universities have introduced programs to emphasize the importance of recruiting and supporting underrepresented minorities in virtually all fields, as well as the need to increase the presence of women in many of them. Such programs are a high priority of most graduate schools, which have developed procedures and funding mechanisms to encourage minority and female undergraduates to consider doctoral education and to provide support for women and minorities who enroll in doctoral programs. Indeed, government funding agenciesâamong them the National Institutes of Health (NIH) and the National Science Foundation (NSF) âhave developed targeted programs to stimulate the recruitment, retention, and success of women and minorities in doctoral programs. Meanwhile, the significant gains in minority and female enrollments in undergraduate education are broadening the base from which these students may be recruited. Some of the results are encouraging. The data gathered for this study show considerable progress in these areas since the 1995 NRC study was conducted. The percentage of underrepresented minorities produced by doctoral programs overall has increased somewhat, and increases are apparent in all the fields surveyed in this study. Far more women are in doctoral education now than in the 1980s and 1990s, and in some fields once dominated by males, women doctoral candidates now are the majority. Despite these significant gains, underrepresented minorities are still a small proportion of students in many areas of doctoral studyâa percentage that remains considerably lower in fact than at earlier levels of education. Women have made striking gains in some biological science and social science fields, but they remain underrepresented in many areas, especially in engineering and the physical and mathematical sciences. And the number of faculty who are women or minorities in many fields remains small.11 Areas in which increases in underrepresented minorities and women have been most prominent include some fields not included in this study. A 2009 article in the Chronicle of Higher Education by Marc Goulden et al. noted that selection of disciplines represented in this study does not capture many of the fields in which the minority population is relatively highâfor example, programs in education or social work.12 For the fields surveyed in this study, NSF data indicate that the largest gains in underrepresented minorities have been in the humanities and the biological sciences.13 11 National Research Council, From Scarcity to Visibility (Washington, D.C.: National Academies Press, 2001); S. Cole and E. Barber, Increasing Faculty Diversity: The Occupational Choice of High- Achieving Minority Students (Cambridge, Mass.: Harvard University Press, 2003). 12 M. Goulden, A. Stacey, and M. A. Mason, Assessment denied: The NRCâs sins of omission. Chronicle of Higher Education, August 31, 2009, A184. 13 Nation Science Foundation, Survey of Earned Doctorates (Washington, D.C.: NSF, various years).
CONTEXT AND MOTIVATION 23 The rate of growth from 1993 to 2006 for the humanities was 6.4 percent and 5.2 percent for biological sciences, compared with only 0.4 percent growth rate in the agricultural sciences for the same period. Nevertheless, the gains in minority and female representation in doctoral programs are one of the most notable trends in doctoral education since the 1995 NRC study was conducted. More broadly, since the 1995 study doctoral education has benefited greatly from dramatic increases in enrollments of international students and of domestic minorities and women. These gains demonstrate the ongoing desirability of American doctoral education in an educational world increasingly shaped by intense global competition for exceptional students. They also demonstrate the capacity to bring into doctoral education vital components of the national citizenry historically underrepresented in Ph.D. programs. The demographic group that has not shown gains during this period is the group that was long dominant in doctoral educationânonminority American males. Indeed, the domestic nonminority male population in doctoral education has decreased in both numbers and as a percentage of total doctoral enrollments. According to Science and Engineering Indicators, from 1996 to 2004 the percentage of doctoral degrees awarded to white, non-Hispanic U.S. citizens or permanent resident males decreased from 34.6 percent to 25.2 percent (Table 2-1). In absolute numbers, in the broad fields in this study the total number of doctoral degrees awarded to white, non-Hispanic U.S. citizens or permanent resident males decreased from 9,619 in 1993 to 8,392 in 2006. The largest decrease was in the social sciences and psychology: from 2,501 to 2,048. TABLE 2-1 Ph.D.âs Awarded to White, Non-Hispanic U.S. Citizens or Permanent Resident Males, in Selected Broad Fields, 1993 and 2006 Broad Field 1993 2006 Engineering 1,608 1,269 Physical sciences, math and computer sciences, and geosciences 2,512 2,072 Life sciences 1,290 1,416 Social sciences and psychology 2,501 2,048 Humanities 1,193 1,183 Total 9,619 8,392 Source: NSF Special Tabulation
24 A DATA-BASED ASSESSMENT OF RESEARCH-DOCTORATE PROGRAMS IN THE U.S. THE DATA All kinds of anecdotal evidence contribute to the reputations of doctoral programs, and all of them provide interesting, often useful information. Examples are stories of a long heritage of powerful research findings in a distinguished department; recollections of the accomplishments of famous graduates of years past; recounts of new faculty appointments made to strengthen particular areas of studies; lists of faculty publications that have shaped, changed, or even brought into existence whole fields of scholarship; recitations of the high hopes and aspirations engendered by the development of a new Ph.D. program; or reminders of the traditional high regard for the university in which a program is housed. These reputational dimensions can make a program look very attractive to prospective students, to prospective donors, and to funding agencies. But there are limits to the reliability of a picture of graduate program quality and opportunity that is based on reputation alone. A programâs reputation may reflect renowned professors long retired or the contributions of a handful of faculty in a large program. Doctoral programs that do not have storied histories may find it difficult to demonstrate their current strengths. Others may be more narrowly focused but excel in their areas of specialization. Some programs with excellent reputations but a narrow focus may not match the preferences of all students. Even when reputations for high quality are soundly based and current, they may not help guide prospective students to the best fit for the needs and ambitions they bring to doctoral study. Several important dimensions of doctoral programs become much clearer when viewed from the vantage point of reliable data. How long does it typically take for a student to earn a Ph.D. in chemistry at University A? How much financial support will likely be available for a doctoral student in history at University B, and for how long and in what form? How many students enroll for each one accepted for doctoral study in electrical engineering at University C, and what range of Graduate Record Examination (GRE) scores were likely expected for admission? Of the students who initially enroll for a doctorate in anthropology at University D, what percentage completes that degree within a six-year period? Which universities provide adequate health insurance programs and child care services for their doctoral students? Most universities now produce compelling statements in support of diversity in graduate education, but which doctoral programs demonstrate strong records in recruiting, retaining, and graduating underrepresented students? Will newly enrolled doctoral students be expected to join a graduate employees union if they attend University X? How much teaching, and of what sort, is expected of teaching assistants? Are funds available to doctoral students for travel to conferences or for research? Other questions might be: what is the record of research productivity as determined by objective measures among the faculty in any given doctoral program? Do the faculty fully understand their own disciplinary and cross-disciplinary academic interests? Are fields of study supported by enough faculty to make research in that area a viable doctoral option? What does the university do to facilitate interdisciplinary study, how fully are the faculty engaged in it, and how is such work across disciplinary lines reflected in the degree structure?
CONTEXT AND MOTIVATION 25 Good, well-organized data in such areas provide ways to begin answering these questions and many more. The program questionnaire collected data relevant to many of these questions. The answers to some of them can be found in the full dataset, which is larger than the amount of data in the spreadsheet that accompanies the report. The data in the spreadsheet are being made easily accessible because they bear directly on the rankings or they had high response rates or both. The full dataset contains all the responses to all the questions on the program questionnaire, but response rates to some questions may be lower. This NRC study has engaged the surveyed universities in an unprecedented effort to identify the most appropriate data categories and the most effective ways to collect and organize data within them. The universities have responded in kind. An active group of institutional representatives, institutional researchers, and staff from the NRC and Mathematica Policy Research (MPR), the survey research firm engaged for data collection, spent many months refining the questionnaires that organized the information collected from universities. Once the data were collected, they were checked and rechecked via continued correspondence with the participating universities. The data are useful for comparative purposes only insofar as they are generated using definitions and collection procedures that are consistent across all programs and universities. Reaching a shared understanding of the kind of questions that would produce the best data in the most pertinent categories was a challenge of significant dimensions. On each campus enormous efforts went into collecting data consistent with the NRC definitions and methodology. Because this is the first study to make such extensive demands on programs to provide so much comparable data, nearly all programs had to adapt existing practices, or devise new ones, to produce the information required by the questionnaires. Individual doctoral programs, whether they were expected to organize their own records or check the data supplied from central sources, or, as in most cases, both, put much time and effort into the data collection process. Faculty asked to fill out questionnaires providing information about their scholarly records responded at an exceptionally high rate, as did doctoral students in the five disciplines selected for an experimental student survey. Graduate schools, or other institutional units asked to submit the data to the NRC, mobilized exceptional efforts to complete the forms. A productive side benefit of this study is that in many institutions the effort required by the NRC survey has contributed to better internal practices and improved understanding, both centrally and in individual doctoral programs, of data collection and self-assessment. Even though the data collected in 2006 for this NRC study are already dated, they will increase in usefulness as long as they are regularly updated. Updatable data in the key dimensions of doctoral study will enable programs to, for example, measure the success of their own reforms, identify possible slippage in quality, learn extensively from other universities that have introduced changes into their doctoral programs, and gauge program solidity through performance over a more extended period of time. Identifying the procedures needed to ensure that the data collected by this NRC study will be systematically updated at intervals timed to enable doctoral programs to assess their achievements and efforts to improve practices will pose a new set of challenges both for the NRC and for universities. But developing such procedures will be crucial to realizing the maximum benefit of the process begun with the extensive collection of data for this study.
