Review of Specialized Degree-Granting Graduate Programs of the Department of Defense in STEM and Management (2014)

3

Value Proposition for Department of Defense Institutions Offering Advanced Degrees in STEM and Management

INTRODUCTION

All military Services need and depend on military and civilian personnel with advanced technical education across a broad array of academic disciplines. Technical capabilities have always been critical to military roles and missions and are likely to become more critical in the future as both the military and society enhance their focus on improving human performance through human-computer interactions, developing autonomous robots and vehicles, creating renewable energy sources, exploiting technology to increase safety and improve efficiencies in manufacturing processes, writing complex embedded software to spur growth in the “internet of things,” and enabling defensive and offensive cyber capabilities. As described in Chapter 1, the defense of the United States is based solidly on a policy of technological superiority where needed.

The 2012 National Research Council (NRC) report on the Department of Defense (DoD) science, technology, engineering, and mathematics (STEM) workforce recommended that DoD expand its STEM workforce with technically qualified U.S. citizens and non-U.S. citizens and increase re-education of nontechnical employees in STEM disciplines:

The DoD should ensure that the education and training, and the re-education and retraining, opportunities for its civilian STEM workforce are both commensurate with similar opportunities afforded career military personnel and tailored to the needs of the civilian workforce. ¹

With more than 30 percent of DoD civilians eligible for retirement, competition with private industry for qualified STEM candidates, and the increasing technical complexity of defense solutions, DoD requires an innovative and multipronged graduate education approach to ensure that it retains and enhances the quality of its technical workforce.

The Air Force and Navy established the Air Force Institute of Technology (AFIT) and the Naval Postgraduate School (NPS), respectively, many decades ago to increase military-relevant STEM and management (STEM+M) graduate education options for DoD personnel. Today, each school is centrally funded by its respective Service, primarily to educate its own military workforce. Both institutions also educate a small number of military personnel from sister Services, a few civilian personnel, and significant numbers of international military personnel, whose education costs are born by the student’s Service, organization, or country—i.e., on a reimbursable basis. ² In addition, AFIT and NPS play a vital role in providing military-relevant education programs and developing military-to-military relationships between Service members and foreign allies. Like their civilian education counterparts, both institutions also have research programs and centers aligned with DoD needs to enhance their education missions and provide value beyond simply educating DoD’s workforce.

This chapter examines how NPS and AFIT accomplish their education and research objectives. Focusing on value, the chapter examines details on the history, current status, research enterprise, costs, and value proposition of both institutions.³ The chapter also addresses operational and organizational obstacles that impede mission quality and execution, particularly for AFIT.

_________________

¹ National Research Council (NRC), Assuring the U.S. Department of Defense a Strong Science, Technology, Engineering, and Mathematics (STEM) Workforce, The National Academies Press, Washington, D.C., 2012, p. 11.

² Reimbursable funding is the name given to work that is requested and funded by another agency or organization. Interagency reimbursables within DoD are funded by Military Interdepartmental Purchase Requests (MIPR).

³ Value proposition can best be defined as the sum total of benefits derived, both actual and perceived, from a service or product. This can include return on investment, quality, speed of service, among other factors. A good value proposition will produce convincing reasons for a customer to buy a service or product.

OVERVIEWS OF THE AIR FORCE INSTITUTE OF TECHNOLOGY AND THE NAVAL POSTGRADUATE SCHOOL

Overview of the Air Force Institute of Technology

AFIT grants graduate school of engineering and management degrees and offers technical, professional continuing education. AFIT is accredited by the North Central Association of Colleges and Schools of the Higher Learning Commission, a regional collaboration of higher education 4-year and graduate institutions. Located on Wright-Patterson Air Force Base (WPAFB), AFIT is synergistically collocated with the Air Force Research Laboratory (AFRL) and the School of Aerospace Medicine. Their geographic proximity allows all three institutions to share personnel, library, and laboratory facilities. Additionally, AFIT has strong ties with the National Air and Space Intelligence Center, the Air Force Life Cycle Management Center, and Air Force Materiel Command Headquarters, all located on WPAFB.⁴ The history of AFIT, described on its website,⁵ is given below.

AFIT began as the Air School of Application, established within the Engineering Division at McCook Field, Dayton, Ohio, by the War Department in November 1919. The school was designated the Air Service Engineering School following the creation of the Air Service in 1920. School operations ceased during World War II, then reopened in 1944 as the Army Air Forces Engineering School with the mission to conduct a series of accelerated 3- and 6-month-long courses to meet emergency needs in 1944. In 1945, a board of officers “recommended that the Army Air Force establish a technological school under the immediate supervision of the Commanding General, Air Technical Service Command, using the existing Army Air Force Engineering School as a nucleus for expansion to accomplish the recommended action.” The Army Air Forces Institute of Technology was officially opened on September 3, 1946. The Army did not establish an equivalent institution when the Air Force became autonomous, but chose to rely on mainly civilian universities for their graduate education needs. The institute was “composed of two colleges: Engineering and Maintenance, and Logistics and Procurement.”⁶

When the Air Force “became an autonomous unit in the military establishment during 1947,” the institute was given its current name. On April 1, 1950, “command jurisdiction of the institute was transferred from Air Materiel Command to Air University. On August 31, 1954, President Eisenhower signed Senate Bill 3712,” giving AFIT authority to grant degrees. In 1955, AFIT was accredited in aeronautical and electrical engineering. It awarded its first degrees the following year. AFIT was

_________________

⁴ Taken mainly from the AFIT website at http://www.afit.edu.

⁵ AFIT, “AFIT History,” http://www.afit.edu/ABOUT/page.cfm?page=126, accessed June 6, 2014.

⁶ Ibid.

granted institutional accreditation at the master’s level in 1960. Its accreditation was extended to the Ph.D. level in 1965.⁷

Overview of the Naval Postgraduate School

NPS is accredited by the Senior Commission of the Western Association of Schools and Colleges of the Higher Learning Commission, a regional collaboration of higher education 4-year and graduate institutions. The university is operated by the U.S. Navy and is located in Monterey, California. NPS awards a full spectrum of master’s and doctoral degrees in areas of interest to the Navy, a large percentage of which are STEM+M degrees. The history of NPS, described on its website, is given below.⁸

On June 9, 1909, the Secretary of the Navy opened a school of marine engineering at Annapolis, Maryland. This small program, “consisting of 10 officer students and two Navy instructors,” would later become NPS. On October 31, 1912, it was renamed the Postgraduate Department of the Naval Academy with “courses of study in ordnance and gunnery, electrical engineering, radio telegraphy, naval construction, and civil engineering, as well as continuing the original program in marine engineering. In 1945, “Congress passed legislation to make the school a fully accredited, degree-granting graduate institution. Two years later, Congress authorized the purchase of the Hotel Del Monte and 627 acres of surrounding land for use as an independent campus for the school. In December 1951, the Postgraduate School moved to its current campus in Monterey.”⁹

Enrollment, Accreditation, and Faculty

Table 3-1 contains a recent historical summary of AFIT and NPS faculty over the past 5 years. Although both school’s faculties have trended upward in size between 2009 and 2013, the NPS faculty size is several times the size of the AFIT faculty. In addition, military members constitute a much larger fraction of the total AFIT faculty population than they do at NPS. And NPS has a much larger portion of its faculty funded on a reimbursable basis.

Finding 3-1. NPS and AFIT have student-to-faculty ratios, respectively, of 4 and 8. Based on U.S. News and World Report rankings, top engineering schools such as the Massachusetts Institute of Technology (MIT) and Stanford University main-

_________________

⁷ Ibid.

⁸ Much of this was taken from NPS, “NPS History,” http://nps.edu/About/NPSHistory/History.html, accessed June 6, 2014.

⁹ Ibid.

TABLE 3-1 Faculty Numbers at the Naval Postgraduate School and the Air Force Institute of Technology

NOTE: —, information not provided to the committee. In some instances the institutions provided us with data that was categorized differently or simply was not provided.
SOURCE: Data from the Air Force Institute of Technology and the Naval Postgraduate School.

tain ratios between 5 and 8. Therefore, it appears that both AFIT and NPS have sufficient faculty numbers to deliver accredited graduate master’s degrees and certificates. Based on the committee’s graduate STEM education expertise, to include leadership and evaluator roles with the Accreditation Board for Engineering and Technology (ABET) accreditation bodies, NPS and AFIT teaching and research methods are pedagogically consistent with other leading universities.

Both NPS and AFIT offer a wide variety of certificates, master’s, and doctoral degrees.¹⁰,¹¹ Many of the engineering degrees are accredited by ABET at both universities.¹² They are leaders in graduate level engineering accreditation. With nine programs accredited at AFIT and four at NPS, they comprise more than 50 percent of the graduate engineering programs accredited by ABET. The Graduate School of Business and Public Policy programs at NPS are accredited by the Association to Advance Collegiate Schools of Business (AACSB). The Master of Business Administration program at NPS is accredited by the National Association of Schools of Public Affairs and Administration (NASPAA). AFIT does not have a business-focused program but does offer degrees in logistics management and engineering management, which are also ABET accredited. Table 3-2 contains a historical summary of accredited program enrollment numbers by student type for both schools. Table 3-3 shows the number of degrees as a function of time and delivery mode.

Table 3-2 indicates that while the Army relies on NPS and AFIT to educate a portion of its officer corps, the number is relatively small compared to the Navy and Air Force.¹³

_________________

¹⁰ A list of degrees offered by NPS can be found at http://www.nps.edu/Academics/GeneralCatalog/701.htm#o197.

¹¹ A list of degrees offered by AFIT can be found at http://www.afit.edu/en/academicprograms.cfm?a=mastdoc.

¹² ABET is a recognized accreditor in the U.S. by the Council for Higher Education Accreditation for engineering, technology, and computer science programs.

¹³ This observation, along with a request to comment on the need for an Army equivalent of NPS or AFIT, was posed by the committee to the presenter from the Office of the Assistant Secretary of the Army for Acquisition, Logistics, and Technology. Based on dialog with the Army presenter, there are two possible explanations for this situation: (1) unlike the Air Force and Navy, each of which has a centralized corporate graduate education fund for officers, the Army’s decentralized funding approach requires each organization to fund graduate education from their own budgets. Consequently, there has never been a critical funding or sponsorship mass to advocate for a centralized Army equivalent of AFIT or NPS; (2) the AFIT and NPS curriculums are designed to focus on Air Force and Navy problems and technologies. AFIT and NPS credentials are therefore more valuable for officers in their respective Services. With the exception of a few programs, such as nuclear engineering and operations research, Army officers may not see the value of Air Force- and Navy-oriented graduate institutions. Without additional data from the Army to review, it is not feasible to draw any further conclusions on this topic.

TABLE 3-2 Enrollment Numbers by Student Type at the Naval Postgraduate School and the Air Force Institute of Technology

NOTE: —, information not provided to the committee.

SOURCE: Data from the Air Force Institute of Technology and the Naval Postgraduate School.

Finding 3-2. The Army manages and fulfills its STEM+M graduate education needs in a highly decentralized manner. It was therefore not possible to gain a holistic understanding of the Army’s STEM+M graduate education needs and sources from the single Army presentation provided by the Office of the Assistant Secretary of the Army for Acquisition, Logistics, and Technology. In addition, it was not possible to determine, due to the lack of data provided by the Army, if the Army STEM+M graduate education model, which does not significantly rely on DoD education institutions like AFIT and NPS to educate its STEM+M workforce, offers a viable alternative to the blended DoD and civilian institution models used by the Air Force and the Navy.¹⁴

_________________

¹⁴ Nancy Harned, Executive Director, Strategic Planning and Program Planning, “STEM and Army S&T Enterprise,” presentation to the committee on January 9, 2014.

TABLE 3-3 Types of Graduate Degrees Awarded at the Naval Postgraduate School (NPS) and the Air Force Institute of Technology

NOTE: The NPS distance learning (DL) program enrolled 1,005 students for academic year (AY) 2012. With 624 civilians enrolled in the DL programs needed by the Department of Defense and a distributed campus worldwide, the NPS is a model for DL in higher education (NPS Provost update briefing to the Board of Advisors February 20, 2014).
SOURCE: Data from the Air Force Institute of Technology and the Naval Postgraduate School.

NPS offers numerous certificates and master’s degrees online, including a Ph.D. in systems engineering.¹⁵,¹⁶ The programs employ both asynchronous and synchronous modes of distance education delivery. Many of NPS’s 26 certificate and 17 master’s programs are taught both on-campus in Monterey and online. AFIT currently only offers two degrees and five certificate programs online. Neither school offers joint degree programs with other universities. Additional discussion of online, distance-learning approaches at both schools can be found in Chapter 4. Table 3-3 summarizes enrollment figures for on-campus and online programs at both institutions.

_________________

¹⁵ See http://www.nps.edu/Academics/DL/DLPrograms/dlProgramSearch.html.

¹⁶ The systems engineering (SE) program has the most robust distance learning (DL) program at either institution. With 406 in master’s DL programs and 22 PhD students the DL programs in SE are significantly greater than the on campus program. Both the resident and DL SE programs are ABET accredited. The NPS DL program for SE is unique in that most of the faculty members are “embedded” close to major student concentrations, such as the National Capitol Region, NAS Patuxent River, and San Diego. All faculty members on the main campus teach in the DL program. All courses are the same, with the same faculty, texts, and syllabi for the DL and on campus program. Because of laboratory access issues, some specialization tracks are not offered to DL students.

TABLE 3-4 Research Funding and Sources at the Naval Postgraduate School (in millions of dollars)

NOTE: CRADA, cooperative research and development agreement.
SOURCE: Data from the Naval Postgraduate School annual reports provided on its website.

Research Enterprise

Consistent with their graduate level education mission, both AFIT and NPS maintain internally funded research programs in support of master’s thesis and Ph.D. dissertation studies. Both schools also maintain externally sponsored research programs funded by a number of government sources. Tables 3-4 and 3-5 summarize the research funding sources and amounts for both universities. ¹⁷ The data shows that the number of externally funded, sponsored research programs at NPS is much larger than AFIT, as expected from the relative sizes of the school’s non-tenure track faculty previously shown in Table 3-1. The difference is due in part to the larger student and research faculty populations at NPS. Based on data from the Center for Measuring University Performance, NPS annually ranks around 100th in the country in external research funding. ¹⁸ The Carnegie Foundation classifies AFIT as a Doctoral Research University.¹⁹ For 2010, AFIT and NPS ranked 248th and 113th, respectively, in federal research funding for universities.²⁰

Both AFIT and NPS maintain multiple research centers that bring together experts from diverse disciplines to focus efforts on specific interdisciplinary chal-

_________________

¹⁷ The specifics are in the format of a pie chart, see http://www.afit.edu/EN/docs/Research/AFITpercent20Annualpercent20Reportpercent202011.pdf, p. 33. Numbers were also provided by NPS Office of Research.

¹⁸ From http://mup.asu.edu/research2012.pdf.

¹⁹ From http://classifications.carnegiefoundation.org/.

²⁰ From http://mup.asu.edu/research_data.html, accessed April 17, 2014.

TABLE 3-5 Research Funding and Sources at the Air Force Institute of Technology (millions of dollars)

NOTE: Only totals given. CRADA, cooperative research and development agreement.
SOURCE: Data from the Air Force Institute of Technology.

lenges of relevance to DoD and the national security communities. The interdisciplinary approach combines people, facilities, and equipment required to solve complex problems. Research center projects are typically sponsored and funded by external government sources and, like civilian institutions, are key supplements to the university’s operation and maintenance (O&M) budget. Research projects enhance the educational mission by contributing to the development of faculty and students, and by allowing “margin of excellence” activities not fully covered in the schools direct allotted funding. Examples of research centers found at both universities include the Centers for Cyberspace Research, Directed Energy, and Advanced Navigation hosted by AFIT, and the Centers for Asymmetric Warfare, Autonomous Vehicle Research, and Cyber Warfare hosted by NPS.

How sponsored research at NPS and AFIT centers might save DoD funding elsewhere, thereby possibly offsetting the costs of the institutions for the Services or DoD, was briefly examined. Many examples were provided by AFIT and NPS to provide concrete evidence of cost savings. A few examples from those submitted include a 2011 AFIT graduate research paper that studied airlift fuel loads on cargo missions to Afghanistan, resulting in changes that were adopted by Air Mobility Command Control Center in 2012 and now save the Air Force up to $111 million per year.²¹ An NPS graduate education research paper studied the Air Tasking Efficiency Model (ATEM) that has been in use since 2006. As a result of this NPS thesis work, the ATEM was able to justify withdrawing six more C-130s from Iraq

_________________

²¹ Walter J. Lesinski III, “Tankering Fuel: A Cost Saving Initiative,” AFIT/IMO/ENS/11-06, May 2011, p. 37.

with no reduction to service levels; savings were estimated at $2.5 million in convoy mitigation costs and the potential, non-estimated reduction of personnel casualties. ²² Finally, a 2012 paper, which won the 2013 INFORMS Prize for outstanding military operations research, described a state-of-the-art design of experiments for simulation models using unmanned air vehicles (UAVs). A former director of the U.S. Army Training and Doctrine Command (TRADOC) Analysis Center (TRAC) estimates that the model harvested $6 billion in savings and 6,000 to 10,000 billets.²³ These examples are a small sample. It only takes a few actual savings of this magnitude to offset the cost of educating many students. AFIT and NPS provide a significant return on investment beyond the education that they provide to their students.

Many civilian universities have strong partnerships with university-affiliated research centers (UARCs) located on or near their campuses. UARCs create synergies between applied and classified DoD research and more theoretical, unclassified research conducted in the institution’s research laboratories. There are several UARC research institutions, such as Lincoln Laboratories, the Jet Propulsion Laboratory, and Lawrence Livermore National Laboratory, located near top-tier civilian institutions, where classified academic research might be performed. This same capability exists in many government laboratories, such as AFRL, the Army Research Laboratory, and the Naval Research Laboratory, which also have several civilian university affiliations. However, because UARC laboratories are typically located on secured sites away from the affiliated university campus and are not formal academic institutions (e.g., different cultures, different goals), they pose an impediment to academic research. Given that most DoD students are under tight timelines to complete their coursework and theses, these additional impediments create risks and stress for DoD employees that do not exist at AFIT or NPS, where research facilities are located on campus or adjacent to campus, as is the case with AFIT and AFRL.

For example, AFIT currently enjoys a strong relationship with AFRL whose headquarters and five of its research directorates are within walking distance of the campus. Students can use laboratory facilities and leverage subject matter experts (SMEs) beyond available faculty. NPS’s larger research enterprise enjoys similar synergies between its education arm and its research centers. However, its somewhat isolated location lacks geographical proximity to major Navy laboratories, although the breadth of research sponsors and partners is noteworthy.

_________________

²² Brown, Carlyle, and Dell, “Optimizing Intratheater Military Airlift in Iraq and Afghanistan,” Military Operations Research 18(3): 35-51, 2013.

²³ Design and Analysis of Experiments, Volume 3, Special Designs and Applications (K.Hinkelmann, ed.), Wiley & Sons, 2012.

The November 2012 report of the Naval Inspector General (IG) concerning the leadership of NPS, its mission, and its activities was reviewed during the course of this study.²⁴ The IG report stated:

The focus on research by NPS management and faculty has detracted from the importance of educating naval officers. NPS has focused on increasing research funding and research positions at NPS, which is a component of becoming a larger research institution but not necessarily a top-tiered research institution. NPS can increase its status as a research institute by encouraging an increase in faculty achievement of recognized research accomplishments and creating a legacy of students that achieved research excellence.²⁵

No views are presented herein regarding personnel and financial procedures; however, the committee differs with the report concerning the importance of research in graduate education at NPS and, by extension, AFIT. Research drives the activities and intellectual efforts of both professors and students by exposing them to emerging technologies that may enter the force or fleet by the time students enter senior decision-making roles. In addition, active research programs serve as a tool to teach students how to think more creatively and collaboratively to solve complex military problems. Immersing students in a research environment that challenges status-quo thinking and encourages innovative thought and broad discussion benefits them in their DoD careers. This research, which is often funded by government agencies such as the National Science Foundation, the Office of Naval Research, the Defense Advanced Research Projects Agency, and numerous other DoD and intelligence sources, cannot only lead to solutions that save DoD money, but can also offset direct university costs. Finally, research breadth and recognition serve as measures of excellence for leading universities and allows AFIT and NPS to attract the best and brightest educators from top U.S. schools who expect environments that facilitate the development of strong publication credentials. These educators are then readily available to DoD decision makers seeking technical advice about complex problems. In short, research plays a fundamental and necessary role in graduate-level education at AFIT, NPS, and all graduate-level institutions.

The leadership of NPS has chosen to compare itself to a number of leading universities throughout the United States, including MIT, Stanford University, and Carnegie Mellon University. While it would be a challenge for a DoD-funded school to completely emulate a premier civilian school, the benchmark of excellence set by seeking these comparisons is appropriate and desirable. DoD students deserve the best education possible—one that combines the academic excellence of leading civilian universities with military-relevant teaching and research activities.

_________________

²⁴ Naval Inspector General, letter with the subject “Command Inspection of Naval Postgraduate School,” dated October 22, 2012.

²⁵ Ibid.

COSTS ASSOCIATED WITH THE AIR FORCE INSTITUTE OF TECHNOLOGY AND THE NAVAL POSTGRADUATE SCHOOL

Generally speaking, determining the cost of any DoD organization is difficult. Budgets often do not include all of true costs, which are distributed across several organizational budgets that contribute to a given organization. Consequently, government organizations are typically more focused on preparing and executing budgets than capturing actual costs. For example, the cost of military personnel and all facility-related costs are usually paid through separate accounts for multiple entities and are not part of the direct organizational or institutional budgets for each entity. The cost of DoD schools includes military personnel and other services not provided at a civilian institution. This fact makes it difficult to accurately estimate the true cost to educate students at AFIT and NPS at a level that would bear the close scrutiny of a trained accountant and serve as a comparison to other education providers. Therefore, the following paragraphs offer a simple analysis that provides very approximate but useful cost estimates for the purpose of assessing value propositions in the next section.

AFIT’s major cost drivers are contained in its directly allocated budget provided by the Air Force, as well as military pay and facilities costs contained in other organization’s budgets. Using fiscal year (FY) 2012 data provided by these organizations, AFIT’s annual costs were as follows: $28.9 million of budget directly allotted to AFIT from the Air Force (includes fully loaded civilian pay), $15.5 million of military pay, and $3.8 million of facilities and host base support cost, for a total annual cost of $48.2 million. The Office of the Deputy Chief of Staff of the Air Force for Manpower, Personnel, and Services valued the research contribution at $38 million. Offsetting the total cost with the research contribution yields $19,800 per calendar year or $14,900 per academic year for each of the 514 students. Ignoring the research offset yields a worst-case estimate (to a first order approximation) of nearly $93,800 per calendar year and $70,300 per academic year.

Table 3-6 contains summary financial information for AFIT and NPS. Similar to civilian schools, research budgets supplement the tuition costs by charging some of the overall institutional costs to the research sponsor, thus spreading institutional costs over a larger base. This is notable for NPS where the direct allotted budget only accounts for about 29 percent of their $368.6 million total operating budget for FY2012. Supplemental research activities can also allow for a larger faculty in the sense that some fraction of the faculty salaries is borne by other sponsors, yet, these faculty members can be available on a part-time basis to teach, participate in governance, and oversee research.

Estimating military pay costs using budgets can also be challenging. For example, military personnel costs are available through the DoD Comptroller who

TABLE 3-6 Income for the Air Force Institute of Technology and the Naval Postgraduate School (millions of dollars)

NOTE: Air Force Institute of Technology and Naval Postgraduate School provided military manpower by rank, operations and mainenance (O&M) including civilian pay, base support costs, present value of facilities, and reimbursements.
SOURCE: Data from the Air Force Institute of Technology and the Naval Postgraduate School.

annually publishes the DoD Composite Rate for each Service.²⁶ The rates are given by rank and include the cost of salaries, retirement and medical benefits, housing and subsistence payments, annualized permanent change of station (moving), and all special pay. These are significantly more than the budgeted salary cost of personnel by a factor that ranges from slightly less than 2.0 to around 2.5, which is comparable to fully burdened costs elsewhere when one does not include elements such as facility capitalization, bid and proposal, and independent research and development, and fee/profit costs.

The cost of construction of DoD facilities is usually funded through the military constructions budget. The maintenance, repair, and operation of facilities are contained in the budgets of base support organizations.²⁷ Determining the

_________________

²⁶ See Undersecretary of Defense (Comptroller), “Financial Management Reports, Department of Defense FY 2014 Reimbursable Rates,” Military Personnel Composite Standard Pay and Reimbursement Rates (Tab K), May 9, 2013, http://comptroller.defense.gov/financialmanagement/reports/rates2014.aspx.

²⁷ 88th Air Base Wing for AFIT and Naval Support Activity, Monterey, for NPS.

TABLE 3-7 Reimbursable Tuition Rates

NOTE: Both are annual rates for Fall of 2013-2014.

SOURCE: Data from the Air Force Institute of Technology and the Naval Postgraduate School.

appropriate assignment of these costs toward the annual cost of operating an organization or activity is difficult. For example, construction costs need to be spread over the lifetime of the facility, and frequently, utility costs are not monitored for individual units but for an entire base. Whitestone Research, under contract to DoD and other federal agencies, has developed life-cycle cost models for facilities that capture all facility costs and express them as a percentage of the current value of the facility. In a presentation given by Whitestone Research,²⁸ the estimated percentage for all DoD facilities in aggregate is given as 4.3 percent of replacement value per year. Table 3-6 summarizes the financial information for AFIT and NPS by broad income and expense categories.

All DoD graduate degree institutions, including AFIT and NPS, have established rates they charge for students not covered by their direct budgets. These students may come from other military Services, the civilian component of the DoD workforce, other U.S. agencies, the private sector, and foreign nations. AFIT and NPS provided the rate data contained in Table 3-7. Comparing these rates to the first-order total cost approximation shows that the published rates probably do not fully recover the total actual costs to educate additional students as estimated above.

Table 3-8 contains the published total tuition costs of obtaining graduate degrees of some peer institutions. These costs are also difficult to obtain because of cost-reduction agreements between the Services and the institutions, scholarships, and the fact many military take more than the required classes and often enroll during the summer, among other factors.

In summary, the cost differential between AFIT and NPS and civilian institutions is not sufficiently significant for cost to serve as a determining factor in any decision or strategy for graduate education. As will be discussed in the next section, value propositions, rather than costs, should be the primary consideration when determining where to educate DoD employees and military members.

_________________

²⁸ Peter Lufkin, Whitestone Research, “Life Cycle Cost Models for Federal Facilities,” presentation to the National Research Council Committee on Predicting Outcomes from Investments in the Maintenance and Repair of Federal Facilities, February 18, 2010, http://www.whitestoneresearch.com/media/1353/predicting%20outcomes.pdf.

TABLE 3-8 Cost at Selected Civilian Institutions

NOTE: Department of Defense schools are based solely on budgeted cost. The costs are based on 45 quarters or 30 semester hours and a full-time load of 12 hours per term.

SOURCE: Data provided by the respective institutions.

THE VALUE PROPOSITION OF DEPARTMENT OF DEFENSE GRADUATE SCHOOLS

Introduction

Although cost is certainly an important factor for many students, it is generally considered only as one part of a broader value proposition. Figure 3-1 offers a framework that captures the value provided by DoD and civilian graduate STEM degree-granting institutions in return for the ranges of costs described in the previous section. This framework provides a useful context when comparing the value of an AFIT or NPS graduate degree against the same degree obtained at a civilian institution. The value proposition offered in subsequent sections addresses the need for DoD to maintain in-house graduate institutions rather than educate its military and civilian employees solely at civilian institutions.

Comparing Value Propositions of the Air Force Institute of Technology, Naval Postgraduate School, and Civilian Institutions

The United States has many excellent public and private civilian universities. Many offer graduate programs in STEM+M areas relevant to DoD, and many do research sponsored by DoD. Indeed, a large majority of the graduate degrees held by military and civilian DoD personnel were obtained from civilian universities. Given the quality and availability of these universities, why does DoD need its own institutions? Table 3-9, which shows the benefits and trade-offs of DoD graduate education institutions, addresses this question. In short, neither the DoD in-house nor the civilian university option is optimal for all circumstances.

AFIT and NPS clearly offer many strategic benefits to DoD. Other DoD institutions might look to them, along with the Uniformed Services University of the

FIGURE 3-1 Value proposition for the Department of Defense and civilian universities.

Health Sciences (USUHS) for “best practices” of educational content and quality. However, when sending students to DoD in-house institutions, DoD must accept the trade-offs that follow from that decision. As mentioned earlier, the correct decision depends on the circumstances surrounding a particular educational need. Continued education of DoD military and civilian personnel at both DoD in-house and civilian institutions would optimize the benefits of both institution types across

TABLE 3-9 Summary of Strategic Benefits to Department of Defense (DoD) and Trade-offs of DoD-Funded and Civilian Institution Graduate Schools

NOTE: AFIT, Air Force Institute of Technology; AFRL, Air Force Research Laboratory; NPS, Naval Postgraduate School; STEM, science, technology, engineering, and mathematics; STEM+M, science, technology engineering, mathematics, and management.

the overall workforce. In the end, this is no different than DoD’s undergraduate education strategy, where entry-level officers are commissioned through Service academies and Reserve Officer’s Training Corp programs at civilian institutions.²⁹ Any marginal or even significant difference in the cost to achieve this mix of educational benefits should be a secondary consideration.

The strategic benefits of an AFIT and NPS education, along with the trade-offs listed in Table 3-9, are discussed below.

1. Instills tradition and culture building.

Strategic benefit. Faculty consists of officers or DoD-focused civilians with a strong Service knowledge base who often incorporate their military experiences and insights into the educational experience. Service military culture is reinforced by the fact that most students in a given cohort share similar backgrounds. A strong alumni network is formed that continues throughout the student’s military career and beyond.

Potential trade-off. A homogenous military-oriented student body and culture limits exposure to non-military cultures, values, and viewpoints, creating an inward-looking atmosphere.

2. Creates teamwork/networking that will help future operations.

Strategic benefit. The student bodies at both Service schools consist of officers from all of the uniformed Services as well as officers from allied foreign countries. These students interact both socially and academically and develop a much deeper appreciation for their sister Services and for the political and cultural differences between the United States and its allies. They form bonds of friendship and trust by working together in accomplishing difficult academic goals that last a lifetime and provide the basis of a lasting network, which can be invaluable in solving Service-to-Service or nation-to-nation issues that may arise later in their respective careers. Teamwork and networking is increasingly international from a military perspective.

Potential trade-off. More exposure to civilian students and faculty could expand the size and future value of the military-oriented student’s networks.

_________________

²⁹ The Army case bears further study.

3. Offers more predictable content and schedules.

Strategic benefit. All military Services transfer officers on a regular basis using well-developed assignment systems. Officer assignments are tightly scheduled, and unexpected changes in an assignment of an officer ripple through the system disturbing other assignments. Predictability in starting and ending dates of military officers enrolled in academic programs is essential. Both AFIT and NPS are structured so that all requirements for programs are offered and met within the time window available, and both have very high on-time completion rates.

Potential trade-off. Civilian institutions offer little or no control over the content and scheduling of their courses; however, larger civilian institutions may offer a greater diversity of course choices.

4. Combines education with training.

Strategic benefit. By law, officers who are part of the acquisition workforce or are seeking a joint assignment have training and education requirements beyond those found in traditional academic programs. For example, both AFIT and NPS offer programs that satisfy the Defense Acquisition Workforce Improvement Act for education and training.
     NPS, and until recently AFIT, also provide Phase 1 Joint Professional Military Education (JPME) on campus. At NPS, JPME is offered through a partnership with the Naval War College. At AFIT, JPME was offered by a detachment from Air University Intermediate Service School. In both cases, the degree and certification requirements are met concurrently while the student is enrolled in their graduate education programs. This results in significant time and money savings to DoD. It also ensures both research and education objectives are aligned with the most recent military doctrine.

Potential trade-off. Given the structured time constraints at DoD institutions, the additional education requirements, in addition to the added training requirements of being stationed on a DoD installation, might lead to less time to devote to the pursuit of the graduate degree.

5. Conducts relevant research that fosters lifelong learning skills.

Strategic benefit. Master’s and Ph.D. students are required to conduct and publish research results in a thesis.³⁰ Topics are typically developed through the research of faculty and their connection to the many Service, DoD, and national security agencies. Occasionally students bring a topic of interest from their field experience and work with a faculty member to define a thesis effort. The process is intense, and faculty is actively involved. Sponsors are routinely asked to evaluate both the quality and applicability of the work and to assess its value.
     The thesis requirement ensures that a student has conducted an independent and original study of a problem relevant to national security, documented the results of that research, and presented the work both in writing and orally. These skills are essential throughout a student’s career and provide qualities that coursework and training alone do not. The ability to frame a complex problem, develop the means to solve it, document the results—often for external peer review, and present the results in an academic forum for evaluation all develop self-confidence and prepare the student for lifelong learning.

Potential trade-off. The broader research scope and diversity of ideas at civilian universities may lead to more “out of the box” thinking and solutions.

6. Is more adaptable to changing DoD priorities.

Strategic benefit. As accredited universities, both AFIT and NPS have well-established procedures for modifying existing programs and establishing new ones. As small institutions, they can make changes quickly and have programs in place in a few months in response to DoD needs and priorities. There are numerous examples of quick-reaction course and program development at both AFIT and NPS.

Potential trade-off. Less structured processes for curricula development and approval may make AFIT and NPS more susceptible to short-term fads than civilian institutions.

7. Offers strong peer mentorship network and structured experience.

Strategic benefit. Students at AFIT and NPS have a very structured experience to include a chain of command among students. This provides mentorship

_________________

³⁰ Some systems engineering students in the distance learning program at NPS do a group project.

for academic and non-academic activities where the roles and responsibilities of a student are well defined. The structured environment facilitates students graduating within the allotted time—the Services only allow 3 years for Ph.D. completion. While there is wide variation in statistics by degree, full-time students average between 5-7 years for STEM-related Ph.D. completion. The current practices of AFIT and NPS align with the best practices described in the Ph.D. Completion Project report.³¹,³²

Potential trade-off. At many civilian institutions, graduate students are expected to participate in many activities beyond coursework and research. For example, many programs require students to teach as part of their Ph.D. experience. This breadth of academic experience could provide a more diverse graduate experience.

8. Easier to do classified/sensitive work.

Strategic benefit. Classified and sensitive work are a routine part of curricula and research at AFIT and NPS. Each has extensive facilities to handle and store classified material through the Top Secret level. Sensitive Compartmented Information Facilities (SCIFs) are available on both campuses as are classified computer networks. Professional security staffs are available to maintain and operate the classified facilities. All faculty members have at least a Secret clearance, and many have Top Secret clearances with access to Sensitive Compartmentalized Information (SCI). A student thesis or capstone project need not be “extracted” to have a publishable, unclassified version. In summary, the environment, facilities, and personnel at both institutions support both classified course work and research. At most civilian STEM+M institutions, classification considerations can be a significant issue, particularly when a publishable thesis is required, unless the institution is already involved in classified research on campus in some fashion (e.g., UARCs and federally funded research and development centers).

Potential trade-off. Without the open scholarship focus that exists at many civilian universities, feedback and generation of ideas can have a more limited scope.

_________________

³¹ Council of Graduate Schools, PhD Completion Project: Policies and Practices to Promote Student Success, http://www.phdcompletion.org/information/Executive_Summary_Student_Success_Book_IV.pdf, March 2010.

³² See also National Science Foundation, “Time to Degree of U.S. Research Doctorate Recipients,” InfoBrief NSF 06-312, available at http://www.nsf.gov/statistics/infbrief/nsf06312/, March 2006.

9. Avoids intellectual property issues.

Strategic benefit. Unless the work is considered sensitive or classified, research produced at AFIT and NPS is published in the public domain. As civilian universities strive to be more entrepreneurial, revenue in the form of royalties and licenses and for intellectual property is becoming a major issue that does not arise in a DoD institution.

Potential trade-off. DoD students would have limited exposure to the challenges of proprietary intellectual property considerations that are universal and key to doing business with DoD.

10. Tackles any problem—even if socially unpopular or unacceptable.

Strategic benefit. Essential military research, such as weapons research, has been discouraged by civilian institutions as inconsistent with their missions and culture. A more subtle challenge is the natural bias of civilian faculty who must build a career of publishable research in some areas that may not adapt well to military subjects of relevance to DoD and of interest to the student.
     AFIT and NPS, as military organizations, readily offer programs or do research in any area of importance to national security or with potential military application. The faculties of these institutions also accept limitations on the publication or dissemination of their work due to classification restrictions.

Potential trade-off. Exposure to a broader array of socially relevant problems and perhaps future national security challenges is limited at the expense of clearer military relevance.

11. Leverages unique facilities and subject matter experts (AFIT with AFRL, centers, and local program offices, and NPS with NRL, laboratories and centers).

Strategic benefit. Both defense schools have internal, Service, and DoD centers of excellence with which to share personnel and facilities. By virtue of collocation with the AFRL, AFIT shares subject matter expertise, libraries, and laboratory facilities. AFRL Ph.D.s also often serve on dissertation committees and provide SME expertise and mentorship to AFIT students. Faculty members also form a body of technical and management experts that DoD acquisition and logistics professionals use to obtain independent opinions on challenging issues.

     NPS, through its strong reimbursable research programs, has created numerous relevant centers in topical areas of national interest. It is immediately adjacent to a campus of the Naval Research Laboratory and the Fleet Numerical Center. These centers can provide the critical mass of faculty to tackle the complex problems that are challenges for DoD.

Potential trade-off. Most major civilian universities have extensive facilities across a broader array of subject areas, as well as more links to external organizations, particularly industry. Many complex problems affecting DoD and the nation are a combination of technical, political, and sociological attributes.

12. Business processes mirror research sponsors.

Strategic benefit. Few inside or outside the government consider the business processes and regulations governing operations as conducive to efficiency and entrepreneurship. Because NPS and AFIT are DoD institutions, a relatively simple Military Interdepartmental Purchase Request (MIPR) can be used to award funding versus a contractual instrument—often a cumbersome, competitive selection process. Service and other government agencies can process MIPR paperwork often within a week to obligate funding. NPS and AFIT can use this advantage to rapidly accept year-end funding on a project order for high-priority research efforts.

Potential trade-off. DoD procurement and contracting processes are changing and becoming increasingly inflexible. Business processes of universities may be more adaptive to these changes than DoD institutions whose business processes have been finely tuned over the years to match their sponsor’s funding processes.

13. Applies graduate level focus on military-relevant problems.

Strategic benefit. AFIT and NPS only offer graduate-level programs. Faculty members are not encumbered with undergraduate responsibilities. Additionally, the research focus at both AFIT and NPS is largely driven by military needs. Many civilian university faculty members funded by DoD conduct high-quality research of interest to the military. However, when military funding shrinks or changes focus, university faculty typically seek funding elsewhere.

Potential trade-off. The difficulty of remediating students whose skills have eroded between their undergraduate and graduate experiences is increased.

Fewer opportunities for students to expand their knowledge and education experience in different disciplines are offered.

14. Provides avenues for students with marginal undergraduate grade-point averages and non-STEM+M degrees to obtain a relevant and connected STEM graduate experience.

Strategic benefit. Military officers often do not exercise their undergraduate STEM+M skills in subsequent assignments. The Navy provides the opportunity for military officers with receded skills, who might not be competitive for admission to the best civilian graduate university programs, to pursue post-graduate education at NPS. As a result, NPS provides a significant amount of remedial education for such students to enhance their STEM-related competencies prior to embarking on their graduate studies. This remediation program is also important for the retraining of students with non-STEM undergraduate degrees. The Air Force’s selection process for officers pursuing graduate degrees requires those officers to meet typical admission criteria, and Air Force students require little remedial work while in residence. AFIT has, however, sent foreign officers, Naval officers, and students selected by the Army for nuclear engineering to take undergraduate courses at local universities to prepare them for graduate work. AFIT does offer a short refresher (approximately 1 month) to all entering master’s students to help them adjust to life as a full-time student. Indeed, some at AFIT and NPS argue that they offer students who may be under-qualified for admission to the best civilian universities an education that would allow them to compete with graduates of the best universities upon graduation. While this may be an overstatement, AFIT and NPS provide a valuable service by making future DoD leaders notably more prepared for decision-making responsibilities that involve technological factors.

Potential trade-off. Top universities maintain their positions in part by controlling the quality of their incoming students. Metrics such as prior academic performance and standardized testing are used to assess student quality. These metrics drive peer perception and national rankings. By lowering admission standards, DoD schools might be perceived as lowering the quality of their educational outcomes. Also, most traditional graduate students follow a well-defined career path by proceeding directly from undergraduate to graduate school. Standardized testing might be a good assessment tool for traditional civilian students, but it probably does not reflect the ability of military students who have a break in formal education because of multiple deployments or command tours.

15. Provides an environment where culture, processes, network, and technology, among other factors, produces a more informed stakeholder/ buyer.

Strategic benefit. Acquisition is a complex interdisciplinary process involving numerous stakeholders. AFIT and NPS expose students to other students, professional development activities, coursework, and other factors that make them more informed buyers.

Potential trade-off. The DoD procurement arm has historically not performed as well as the civilian sector. Best practices from the civilian sector may not be taught in military-oriented graduate education programs.

The brief assessment of strategic benefits and trade-offs to an AFIT and NPS graduate education yields the following findings and recommendation.

Finding 3-3. AFIT and NPS are often viewed solely as education enterprises by their constituents. These institutions, however, are coupled research/education enterprises with productive research programs that improve the quality of student education, play an essential role in attracting and retaining top faculty, and generate valuable weapon system insights and technologies.

Finding 3-4. AFIT and NPS foster teamwork and facilitate the formation of intellectual networks that follow students throughout and beyond their military experiences. Developing military-to-military, joint, and interagency relationships can play a critical role as students work in future multi-Service and multi-national operations.

Recommendation 3-1. The Department of Defense (DoD) should recognize and support the comprehensive value proposition offered by the Air Force Institute of Technology and the Naval Postgraduate School. Measures of convenience, cost, and quality are not sufficient to meet the demand for a technically superior workforce. When viewed from a total value perspective, as described in the benefits column of Table 3-9, DoD’s graduate science, technology, engineering, mathematics, and management enterprise is a tremendous asset to the respective Services, DoD, and the nation.

DoD’s graduate education strategy could include a mix of graduate education options for its employees at both DoD and civilian institutions to realize all strategic benefits and minimize the trade-offs in the overall workforce. Based on the simple cost assessment described here, the difference in the costs of achieving

this mix of educational outcomes and resulting overall workforce capabilities is of secondary importance compared to the strategic benefits offered by quality, DoD-funded institutions. The next section provides more detail on how best to assess the quality of AFIT and NPS educational outcomes.

ON THE QUALITY OF EDUCATION

Traditional measures of academic institutional quality in civilian universities generally rest on the quality of the students and the faculty. The widely cited U.S. News and World Report (USNWR) rankings use many factors for measuring inputs into the system: how good are the students, how much money is the university spending on them, and how strong is the faculty. For the quality of the faculty—at both undergraduate and graduate levels—traditional measures include the institutions where they received their degrees and the outputs of the faculty in three general areas: (1) intellectual activity (e.g., peer-reviewed publications); (2) scholarly activity (e.g., service on campus and on broader academic pursuits nationally and internationally); and (3) individual recognition (e.g., prestigious awards, national academy service and membership). This traditional view of institutional quality effectively assumes that the highest educational outcomes will follow from excellence in these metrics.

Direct measures of educational outcomes are becoming more widely used in terms of educational and learning objectives that can be quantified in some way to determine the eventual outcomes for the students. In this framework, the regional accreditation boards and the subject-matter accreditors are moving toward institutional assessments of colleges and universities to assure the desired performance for the public funding provided to these institutions as state and federal budgets become tighter. Accreditors require “learning outcomes” for each course of study, and a detailed description of the assessment of each outcome, to ensure graduating students have the knowledge and skills degrees promise. Both AFIT and NPS have well-established assessment processes and, when appropriate, have achieved accreditation at the program level.

The most frequently used source of information about reputational quality of master’s programs for prospective graduate students, as well as in marketing by the universities themselves, is the USNWR graduate school rankings. All of the STEM graduate programs—for example, mathematics, biology, computer sciences, and statistics, except engineering—are ranked through collecting survey data every 3 years via a mail survey sent to all schools housing the program. Each university gets two votes, and the raters are asked to simply rate each university’s program, including their own, on a 1 to 5 scale from poor to highest quality. The average across the entire sample of respondents is used to rank the programs. It is difficult

to obtain response rates from USNWR, but it is highly likely that the response rates are not extremely high.

Engineering schools are ranked by USNWR using four sets of criteria, with indicators under each: (1) quality assessment, weight 0.40, measured by peer assessment on a survey with a 1 to 5 scale (0.25) and recruiter assessment (0.15); (2) student selectivity, weight 0.10, measured by mean graduate record exam quantitative scores of incoming students (0.0675) and acceptance rate (0.0325); (3) faculty resources, weight 0.25, measured by student-faculty ratio (0.0375), percentage of faculty in the National Academy of Engineering (0.0725), and number of doctoral degrees awarded (0.0625); and (4) research activity, weight 0.25, measured by total research expenditures (0.15) and average research expenditures per faculty member (0.10). A reputational survey, such as that described above, is used to rank the specialty programs—for example, aeronautical engineering.

While the process used by USNWR for deriving rankings of graduate programs is certainly not without flaws, the faculty raters typically are knowledgeable about the programs they rate. Perceptions about the quantity and quality of published research, the professional prestige of the faculty members, the total amount of external funding, and the prestige of the funding sources are taken into account, as well as the quality and accomplishments of the doctoral students that the program produces. Doctoral programs in STEM fields are also ranked by the NRC’s Board of Higher Education and Workforce in their Assessment of Research Doctoral Programs, which it undertakes every 10 years. The NRC ranking process is much more labor-intensive and involves measuring the productivity of the alumni of the programs over many years, as well as the scholarly productivity of the faculty members in the programs. NRC rankings are not without criticism, much like the USNWR rankings, but they are based on more data.

The USNWR and NRC quality assessment methods are designed to rank and compare civilian institutions. As described in the previous section, in-house DoD education institutions possess unique characteristics that make them difficult to assess for quality using these conventional methods. The quality of DoD graduate degree-granting institutions should not be framed using only conventional methods for the following reasons:

• DoD institutions focus primarily on research-intensive master’s level programs and less on doctoral programs, in contrast to most of the highest-quality civilian institutions.

• By and large, the student bodies of DoD institutions draw from a closed (military) population. DoD is constrained to train the population they recruited at the beginning of their careers; they do not have the option of hiring in new officers at mid-career.

• Most of the above traditional measures of quality based on enrollment and postgraduate career data simply have no bearing. The placement rate for graduates of AFIT and NPS is near 100 percent; the salaries are fixed by the military pay scale, and the list of hiring employers is limited to the sponsoring agencies.

• The publication record in peer-reviewed journals might be slightly lower than at comparable institutions because (1) some of the most important research results are classified; (2) with relatively fewer Ph.D. students, the kinds of research that can be performed by faculty and M.S. students may well limit the number of published journal articles; and (3) the ongoing limits on travel budget and conference presentations severely limit the number of conference papers the faculty can present. This lack of visibility at conferences can negatively affect peer evaluations of the universities/programs, which is the number-one factor in rankings such as the USNWR.

This is not meant to say that traditional metrics of academic quality should not be used to assess the quality of DoD institutions. In fact, traditional measures serve an important role in fostering broader educational credibility, as well as quality, in several key areas of institutional performance such as competitiveness in intellectual output and in success obtaining external research sponsors. The calculus for quality at these DoD institutions is different, however, than it is for civilian institutions. The most appropriate calculus likely comes closer to the emerging approaches for performance-based funding of state colleges and universities. With this view, quality should be measured and aligned with the value received from the primary stakeholders—students, Services, and DoD.

The value proposition and quality measures discussed above are not unique to AFIT and NPS. They are likely to be the same or at least quite similar for all DoD graduate-degree-granting institutions.³³ In a state with a large number of universities in its state system, one individual has budget and regulatory authority over the institutions. Such an individual or function in DoD could become the source for assuring the department of institutional performance and quality across all campuses and adherence to the desired value proposition at each campus and college. Indeed, such a function would probably have been able to undertake this study for DoD.

Such a chancellor function could use one of many approaches to assess the corporate governance needed for DoD graduate-degree-granting institutions, one of which assesses an organization’s ability to change. Figure 3-2, depicts the complex set of factors that affect the ability of DoD to develop and support a well-prepared STEM+M workforce in order to achieve the desired outcomes of STEM+M educa-

_________________

³³ The lack of a single DoD voice that might be the keeper of this strategic view across DoD is noteworthy and a repository of data was difficult for the committee to obtain.

FIGURE 3-2 Theory-of-change model for STEM+M graduate education for the Department of Defense.

tion for DoD. A theory-of-change model can be used for evaluative purposes.³⁴ The model traces the path of intended policies and programs, from the components that support and shape the programs, to the intended outcomes. In this case, given existing DoD policies and resources together with educational experiences of DoD uniform and civilian personnel, this model identifies expected immediate, intermediate, and longer-term outcomes from DoD-managed graduate degree institutions. The model could be applied to all DoD institutions, going beyond STEM+M institutions.

Reading Figure 3-2 from left to right, the process flows from inputs—the people, resources, DoD policies and procedures—to the activities during a STEM graduate education. Activities include classroom or online learning as well as laboratory research and field research and other professional development experiences such as conference and workshop participation. These inputs and activities are expected to affect the trajectory of outcomes such as changes that occur in the students and faculty involved in the STEM education and, ultimately, the impacts within the organizations where graduates return to work. This report separates the faculty and student outcomes to indicate the specific intended aims of STEM education on both students and faculty. The longer-term outcomes are not separated between student and faculty, indicating the desired collaborative and integrated nature of these longer-term relationships. This education process is embedded in the overall DoD system in which there may be significant changes over time, such as new threats to national security and budgetary constraints. The continued collaboration between students and teachers promotes lifelong learning, for example, and provides a means to measure it.

The ability of DoD to achieve the desired outcomes, for both the careers of STEM graduate education graduates and the workforce, is shaped by many other factors that may be beyond the control of DoD leadership—e.g., salary scales, hiring flexibility. Such factors are labeled as mediating factors because they may shape or even block achievement of desired results. Between intermediate and longer-term outcomes lie additional, potentially important mediating factors both within and outside of DoD that may impede achievement of the longer-term DoD objective of ensuring a well-prepared STEM+M workforce, both military and civilian. DoD staff can use an assessment tool of this type to be more strategic in planning and assessing how well the desired educational outcomes and DoD outcomes are attained. The model identifies key mediating factors that affect achievement of short-term outcomes, as well as intended intermediate outcomes. Many of the mediating factors are under DoD control, and some reflect the underlying DoD culture. Research

_________________

³⁴ J.A. McLaughlin and G.B. Jordan, Using logic models, pp. 7-32 in Handbook of Practical Program Evaluation, 2nd edition (J.S. Wholey, H.P. Hatry, and K.E. Newcomer, eds.), Jossey-Bass, San Francisco, Calif., 2004.

has highlighted the importance of promoting a learning culture and risk-taking to enable achievement of desired longer-term outcomes from educational programs. Leaders must be empowering, creative, and support leadership development throughout an organization to ensure effective workforce development.³⁵

The mediating factors identified in the model can also help to explain the extent to which STEM+M graduate education adds value to the students, organizations, and DoD as a whole. For example, despite gaining STEM+M knowledge and skills while in educational programs, military STEM+M graduates may not perform fully in their careers due to inevitable non-STEM+M assignments. Even if the STEM+M graduates—military or civilian—are successful in the workforce, organizational performance may not improve or not be measured appropriately. Note that one important component of the earlier value proposition model is Benefit 7—peer mentoring that occurs as part of an AFIT/NPS degree (see Table 3-9), which fosters improved organizational performance and provides a means to measure it.

The theory-of-change model for STEM+M graduate education can also help to explain why even high-quality curricula and outstanding instruction may not necessarily produce desired intermediate and longer-term outcomes for DoD’s workforce.³⁶ Of significance, the model clarifies how DoD, and more generally federal rules, processes, and procedures may constrain development of a culture that supports life-long learning and career development in DoD’s STEM workforce.

The committee considered important inputs and mediating factors identified in Figure 3-2 and identified several key influence points that seemed to be either weak or not apparent within existing DoD processes. These DoD inputs and mediating factors could be monitored to assess how to make STEM graduate education more effective in adding value to the DoD workforce and, ultimately, to national security. These observations are listed in Table 3-10.

To illustrate the assessments in Table 3-9, the currency of DoD career counselors’ knowledge about offerings and the quality of educational programs within DoD institutions and in civilian institutions is one of the first key influence points about which more information is needed. STEM graduate programs differ along a variety of important dimensions, and the distinctions make the selection process even more critical for both the prospective students and the career counselors because they need to understand the relative advantages of the options available. Five key variables are (1) whether the program is offered within a civilian university or DoD-sponsored institution such as NPS or AFIT; (2) whether or not the student has an undergraduate degree in a STEM field or not; (3) whether or not

_________________

³⁵ J.E. Kee and K.E. Newcomer, Transforming Public and Nonprofit Organizations: Stewardship for Leading Change, Management Concepts, Washington, D.C., 2008.

³⁶ E. Salas and J.A. Cannon-Bowers, The science of training: A decade of progress, Annual Review of Psychology 52:471-499, 2001.

TABLE 3-10 Illustrative Critical Influence Points Affecting Outcomes of STEM Graduate Education for DoD

NOTE: AFIT, Air Force Institute of Technology; DoD, Department of Defense; NPS, Naval Postgraduate School; STEM, science, technology, engineering, and mathematics; SWP, strategic workforce plan.

the degree requires a thesis—i.e., an individual research project; (4) whether or not the master’s degree is free-standing or is taken in conjunction with a doctoral program—i.e., combined M.S./Ph.D. programs; and (5) whether the Ph.D. program requires entering students to have completed a master’s degree before entering the

TABLE 3-11 Distinctions Among STEM+M Graduate Programs

NOTE: DoD, Department of Defense; STEM, science, technology, engineering, and mathematics; STEM+M, STEM and management.

program. To carry the example further, Table 3-11 presents these five options and suggests potential advantages as they might be provided when DoD offices provide counseling advice. The impression is that this is an area that could be improved for the overall benefit of the military workforce, particularly in STEM+M education.

While not under the influence of DoD policies, perceptions of quality of the STEM+M graduate programs and universities offering the programs also matter. As shown in Figure 3-2, it does matter how students view the relative merits of different educational options matters, and how DoD career advisors view the different programs’ relative advantages to the students and to adding value to DoD’s STEM workforce.

INSTITUTIONAL ORGANIZATION CONSIDERATIONS

Command Structure of the Naval Postgraduate School

The NPS president reports to the Chief of Naval Operations (CNO), as shown in Figure 3-3. This is also true of the Naval War College and the U.S. Naval Academy. Academic oversight is provided by the NPS Board of Advisors under the provisions

FIGURE 3-3 Organizational structure for the Naval Postgraduate School.

of the Federal Advisory Committee Act of 1972. Resourcing and budgeting actions flow through the Office of the Chief of Naval Personnel.

NPS made two significant points to the committee during its visit. First, research sponsors who invest with the Navy on issues involving intellectual property are becoming increasingly concerned about the composition of the NPS research team. Those outside DoD pose an ownership risk and make partnering more difficult. Second, NPS’s “short” chain of command is key to their relevancy, flexibility, timeliness, and added value.³⁷ Their ability to rapidly move laterally with senior Navy leaders who have decision authority is critical to their success. NPS stated that the two most important changes at NPS were the Navy’s decision to have NPS led by a civilian (continuity of leadership) and to have the school report directly to the Navy CNO (better resourcing and ownership of mission).³⁸ In short, “stability leads to innovation which drives capability.”³⁹

Command Structure of the Air Force Institute of Technology

The AFIT chancellor reports directly to the Commander, Air University (AU) who, in turn, reports to the Commander, Air Education and Training Command

_________________

³⁷ Dan Boger, NPS, presentation to the committee on November 6, 2012.

³⁸ RADM James Greene, USN (ret), discussion with the committee on November 7, 2012.

³⁹ Ibid.

(AETC), who, in turn, reports to the Chief of Staff of the Air Force (CSAF) (see Figure 3-4). The AFIT subcommittee of the AU Board of Visitors provides oversight.

In 1999, the AFIT Subcommittee of the Board of Visitors noted that AETC was the wrong advocate for AFIT. The minutes of the March 1999 AFIT Board of Visitors noted:

Postgraduate education advocacy is the responsibility of the commander of Air Education & Training Command. This is among his smallest responsibilities and among his lowest priorities. From a priority perspective, the AETC commander is driven to meet the training syllabus. … When money is short in an AETC area to meet a field need, education is an obvious resource target; there is no immediate measurable operational impact if not met. There is no immediate mission or readiness impact nor is there a senior stakeholder noting a shortfall. … Accordingly, AETC is adjudged to be the wrong functional advocate for AFIT. AFIT is not important to AETC’s metrics nor success, nor is there a united user community to advocate for it.

They recommended that AFIT remain within AU, but that the Air Force realign AU as a direct reporting unit to the Air Force Chief of Staff. In 2000, the AFIT Board of Visitors stated that if moving AFIT under the CSAF was not feasible, then AFIT should realign with its largest single customer, the Air Force Materiel Command (AFMC), which would return AFIT to a chain of command similar to what it had

FIGURE 3-4 Organizational structure for the Air Force Institute of Technology.

between 1919 and 1950. At the time, the commander of AFMC was prepared to accept the organizational assignment and advocate for AFIT. However, AU was opposed, and nothing happened to the recommendation.

The issue was raised again with the enactment of the National Defense Authorization Act of 2001 (P.L. 106-398) that called for the Secretary of the Air Force to study and make recommendations with respect to AFIT’s chain of command. The resulting report generated internal debate within the Air Force, whose leadership concluded that the command arrangement should remain the same.⁴⁰ Several observers continued to note that AFIT was a questionable fit even within AU. In 2007, a team of four higher education professionals reviewed the organization and general condition of AU.

AFIT is located hundreds of miles distant [from AU] and actually is accredited by a different regional agency (the North Central Association of Schools and Colleges) than AU, which is accredited by the Southern Association of Schools and Colleges (SACS)…. This distance means that the AFIT leadership and its faculty and student body do not often interact in the same room with comparable AU personnel located in Montgomery.⁴¹

AU views and manages AFIT as a college of AU, not as a separate, autonomous institution reporting through the university.⁴² As a result, AU’s president, not the AFIT chancellor, is the final authority on faculty hiring, faculty promotion, and tenure decisions. AFIT’s lack of autonomy is extremely unusual, if not unprecedented, for a regionally accredited institution and could represent a serious challenge to future accreditation.⁴³

As stated earlier, military-focused, graduate STEM+M education and research conducted at DoD-funded institutions plays a strategically vital role in educating the DoD workforce. DoD invests heavily in maintaining an extensive education portfolio across a wide range of STEM+M programs. How the Services organize to leverage this investment matters. Based on budgetary consequences of prior post-conflicts, it is anticipated that there will be reductions in funding for education, training, and travel, which are often the first programs cut during challenging budget environments. Education, however, is unique in how budgets impact their

_________________

⁴⁰ Report of Air Force Institute of Technology (AFIT) Study For Senate and House Armed Services Committees, submitted to Congress on February 25, 2002.

⁴¹ James L. Fisher, Ltd., Air University Review, February 2007-April 2007.

⁴² As confirmed by the commander and president of Air University during presentations to the committee.

⁴³ AFIT has been continuously accredited by the Higher Learning Commission for more than 50 years. In 2004, Air University received institutional accreditation through the Commission of Colleges of the Southern Association of Colleges and Schools. As currently structured, AFIT is managed by AU as one of its several colleges and, as such, may in the future be ineligible for separate institutional accreditation.

programs, both near and long term. Reductions taken as a “percent” across the board, sometimes referred to as “peanut butter” cuts, have a short shelf life for an educational institution. They are quickly faced with cutting entire programs, such as graduate education at civilian institutions. In addition, the impact of significant budget cuts to education are typically not seen with the normal Program Objective Memorandum cycle, perhaps several cycles. Said another way, if you want to increase the number of general officers with a STEM+M degree, you may not realize the full benefits of your efforts for 20 years or more.

AU controls AFIT’s faculty personnel decisions, including hiring, promotion, tenure, and annual appraisals and salary adjustments. Even if the process usually confirms AFIT’s recommendations, it requires time and effort that could be better spent by both AFIT and AU leaders. Furthermore, it is essential that AFIT continue to have a strong research program and expand its student base to include more sister Service personnel, international officers, and DoD civilian personnel—primarily through reimbursements. AFIT has made significant progress establishing a reimbursable program for both research and tuition, despite difficulties caused by the fact AFIT is the only component in its current chain of command requiring such a program. The difficulties arise from the length required to staff recommendations through both AU and AETC, and by the lack of experience or expertise in such matters within either organization.

The characteristics and implications of education and training are quite different. In an era characterized by the value of intellectual property, organizational agility, and rapid innovation, AFIT may experience “drag” created by a parent organization focused on Air Force-wide readiness issues. This is especially true of budget cuts that tend to get larger as they “roll downhill” through a rather extensive chain of command. The difference in reporting structures of AFIT and NPS is indicative of the difference in strategic priorities each Service assigns its respective school.

Finding 3-5. From an organizational structure and chain of command perspective, AFIT is at a disadvantage in comparison to the other graduate-degree-granting DoD organizations. This disadvantage was highlighted in the organization of the alliance between AFIT and NPS. While the alliance was purported to be between AFIT and NPS, responsibility for oversight of the alliance was given to the NPS Board of Advisors and the AU Board of Visitors. To ensure a connection between those boards, the NPS superintendent was appointed to the AU Board of Visitors and the AU commander, not the AFIT commandant, was appointed to the NPS board.

Finding 3-6. In recognition of the importance of education and the key roles DoD-funded graduate education institutions play in delivering education to the

uniformed component of the workforce, many of these institutions report at the highest levels in the Services, the profession, or the Joint Staff—e.g., the Service academies, USUHS, and NPS. This is appropriate to insure a strategic oversight that is not deterred by other priorities of interim organizations. However, this placement is not true for all DoD degree-granting institutions. In particular, AFIT organizational issues have come up several times in the past 20 years without meaningful resolution of these challenges.

Finding 3-7. AFIT’s current command structure requires it to advocate for initiatives to maintain and strengthen its research-based graduate education programs via a lengthy chain of command that has limited graduate education expertise, virtually no technical research expertise, and a focus on immediate training and professional military education requirements.

Recommendation 3-2. The Air Force Institute of Technology’s (AFIT’s) chain of command should be changed, perhaps to resemble the Naval Postgraduate School, with its own board, budget, accreditation, and program authority, in order for AFIT to maximize its value to the Department of Defense and the nation.

If DoD does not wish to have AFIT report directly to the CSAF, it might choose to align AFIT with AFMC, its largest customer—a solution the AFIT Board of Visitors recommended in 2000. AFIT cannot achieve its full potential as a subordinate organization to AETC or AU. AETC’s training mission, AU’s accrediting body and accreditation process, and the education and research mission of AFIT create significant culture and priority mismatches. As DoD continues to “bend the technology curve,” it will become increasingly import that AFIT and NPS, in partnership, move forward in a manner that allows both to be agile, value added, and innovative. AFIT organizational challenges have arisen several times in the past 20 years without meaningful resolution. DoD needs to resolve this longstanding issue.

Department of Defense Policies

DoD has been damaged by sequestration, furloughs, pay freezes, lack of professional development opportunities, unstable funding, reductions in retirement benefits, increased oversight, and decreased contracting and hiring flexibility, among other factors. This is not unique to DoD in the federal government but arguably has more direct impact on national security readiness than others may have for their missions.

One of the major findings of the 2012 NRC report addressed NPS and the erosion of its independence—i.e., travel, publication restrictions, hiring/personnel

constraints, and new and immature business processes.⁴⁴ That report also states “meeting the workforce needs associated with emerging technologies in the light of existing workforce trends and DoD policies could be problematic.”⁴⁵ Another major finding of the 2012 NRC report was “flexibility, capability, and relevance in the DoD STEM workforce are the essential characteristics sought.”⁴⁶ Current DoD processes and practices are inhibiting the hiring, development, and retention of a competent workforce.

Seemingly small, the current limitations on travel, increased travel regulations, funding constraints, and approval authority for conferences strikes at the heart of a robust graduate educational system and experience for the students. The constraints, mainly driven by short-term funding limitations, severely constrain faculty and students from participating in critical peer reviews and network opportunities, with potentially long-term ramifications. ⁴⁷ Without the opportunity for immersive interaction with peer communities, AFIT and NPS relevancy and credibility are undermined in the technical communities. Conferences for disciplines such as computer science, cyber warfare, computer-based STEM, and other key competencies for DoD are the primary outlets for learning what others are doing, publication, and peer assessment of research, including educational pedagogy. This is severely limiting DoD educational institutions on many fronts, including perceived quality (not visible among their peers), as well as the ability to remain at the leading edge of rapidly advancing technical fields.

Finding 3-8. Sequestration, furlough, pay freezes, and limitations on travel, among other factors, have hampered the ability of AFIT and NPS to provide the required educational experience needed by its students, particularly its uniformed students. Further, it is vitally important for faculty and students at these institutions to be able to attend scientific conferences to present research to their peers, network, receive feedback, and remain current. To this end, the Services would be well served to implement DoD Conference Guidance Version 2.0, dated November 6, 2013, from the Office of the Deputy Chief Management Office that states that for

_________________

⁴⁴ NRC, Assuring the U.S. Department of Defense a Strong Science, Technology, Engineering, and Mathematics (STEM) Workforce, The National Academies Press, Washington, D.C., 2012.

⁴⁵ Ibid., p. 83.

⁴⁶ Ibid., p. 7.

⁴⁷ Office of the Deputy Chief Management Officer, “DoD Conference Policies and Controls,” available at http://dcmo.defense.gov/products-and-services/conference-policies-controls/, accessed April 17, 2014.

most conferences, “approval authority at their discretion to General Officers/Flag Officers/Senior Executive Service members in their organization.”⁴⁸

Finding 3-9. When viewed from a total value perspective, DoD’s graduate STEM+M enterprise is a tremendous asset to the respective Services, DoD, and the nation. Because the Services and DoD are the consumers, research, cultural benefits, and a clear value proposition could be employed for the students in the oversight, management, and operation of these institutions, particularly in times of increasing budget constraints. One can argue that education is among the highest priorities for an investment in preparation for an uncertain future and, once neglected, is extremely difficult to remedy at almost any cost.

Recommendations 3-3. A senior-level panel should be formed composed of former senior military and Department of Defense (DoD) civilians with leadership experience in civilian educational institutions to recommend specific means to (1) remove or reduce the impediments cited in Finding 3-8; (2) advance the value of DoD science, technology, engineering, mathematics, and management (STEM+M) education institutions; and (3) assess the mission impact these impediments, and others that may develop, have on the STEM+M workforce. Such a panel should examine how the whole STEM+M education enterprise aligns programmatically and by research competencies with key DoD science and technology thrusts and make recommendations with regards to programs, people (faculty and students), and, especially, business processes.

CONCLUDING REMARKS

AFIT and NPS are quality educational institutions with relevant and robust research enterprises that enhance their educational missions. They are intellectual assets to their respective Services, DoD, and the nation. Their broader value proposition, as described in this chapter, deserves the recognition by and full support of DoD.

_________________

⁴⁸ DoD, Memorandum: Implementation of Updated Conference Oversight Requirements, November 6, 2013. For additional information, see http://dcmo.defense.gov/products-and-services/conference-policies-controls/DoD%20Conference%20Guidance%20-%206%20November%202013.pdf.