Benchmarking High-Risk/High-Payoff Research
DEPARTMENT OF DEFENSE SERVICE LABORATORIES
While there is no precise organizational analogy to NASA’s Science Mission Directorate (SMD), examining how other aerospace-related agencies allocate resources for high-risk/high-payoff endeavors can be instructive. The three service science and technology (S&T) enterprises (Army, Navy, Air Force) under the Department of Defense (DOD) provide one useful data point for addressing high-risk/high-payoff resource allocation.
To provide a context for discussing how the military services pursue high-risk/high-payoff research endeavors, it is important to begin with an understanding of the budget architecture for DOD science and technology as a whole. The budgetary architecture for science and technology (S&T, or the “R” in R&D) within the DOD consists of three budget categories:
6.1: Basic research, typically associated with technology readiness level (TRL) 1 and 2, in which new scientific phenomena are sought in an effort to discover and advance fundamental knowledge in fields important to national defense. Such research is generally broad in nature, and because of its low TRL, can be considered inherently “high risk.”
6.2: Applied research (also called exploratory development), typically associated with TRL 3 and 4, in which technology is developed based on a newly discovered scientific phenomena, or by the application of scientific phenomena in a totally different manner than currently applied.
6.3: Advanced technology development, typically associated with TRL 5 and 6, in which multiple technologies (often from cross disciplines) are integrated and demonstrated to enable the development of a new military capability to satisfy a military need.
Budget categories 6.1, 6.2, and 6.3, which collectively comprise S&T, can be envisioned as mission enabling and represent the feedstock for the “D” in R&D (D is sometimes called engineering development; it is the engineering of advanced technology into an end item that can eventually be tested, manufactured, and delivered to the military war fighter). However, within the S&T spectrum, the 6.1 category (basic research) can most readily be labeled high risk/high payoff. That is because in examining basic scientific phenomena, there are limited expectations that such research will always yield usable results; however, when such discoveries are made, the accompanying expectation is that at least some will yield exceptionally high results, eventually enabling revolutionary
military capabilities. A National Research Council report, Assessment of Department of Defense Basic Research,1 discusses the basic research (6.1) element of S&T in terms of “unfettered exploration” (p. 14) and describes it as “farsighted, high-payoff research that provides the basis for technological progress” (p. 44). Past examples of game-changing military capabilities enabled by discoveries from basic research projects include night vision; stealth technologies; near real-time delivery of battlefield information; navigation, communication, and weather satellites; and precise munitions. As noted in the study referenced in the follow-on benchmarking discussion, investments in basic research have led to revolutionary civilian capabilities as well. The Internet, communications and weather satellites, global positioning technology, and even the search technologies used by Google all had their origins in DOD basic research.
A 6.1 portfolio typically is comprised of numerous projects that are relatively small in terms of budget when compared to projects in the 6.2 budget category (fewer projects, bigger in scope and cost as compared to the average 6.1 project) and the 6.3 budget category (even fewer projects, bigger in scope and cost as compared to the average 6.2 project budget). A large number of small 6.1 projects across the full spectrum of scientific disciplines maximize the opportunity of discovering scientific phenomena that can be game changing in nature if successfully implemented. As previously stated, if only a few such projects result in the discovery of phenomena that can subsequently be translated to technology in the 6.2 and 6.3 budget categories, that 6.1 investment portfolio would be considered as having achieved its purpose. The challenge: it can sometimes take years following discovery of a new phenomenon to know whether or not the technology it spawns has really enabled a useful new capability to the sponsoring agency.
Within the service laboratories, overall 6.1 budgets are usually established by upper-management, such as by the particular service secretary or the senior-most executive responsible for the Service’s overall R&D component. For the Army and Air Force, the 6.1 budget has varied between 15 to 20 percent of the overall S&T portion of the president’s budget request between FY 1998 and FY 2008. For the Navy, the 6.1 budget has varied between 25 to 30 percent of the overall S&T portion of the president’s budget request for the same 10-year period. While not every dollar of these basic research budgets can be definitively associated with high-risk research, the percentages represent an upper bound, and the committee would surmise in its collective experience that a significant portion of such budgets are aligned with high-risk, high-payoff research endeavors. To the committee’s knowledge, there is no documented analytical basis for these specific 6.1 budget percentages. Rather, this allocation has developed over the years for each respective service, has waxed and waned, and has ultimately been determined to be a reasonable level of investment for research of a broad and unfettered nature when traded off against budget needs for more specific S&T outcomes as pursued under the 6.2 and 6.3 budget categories. It should be noted that these 6.1 percentage allocations typically decrease when looking at the actual budget appropriated by Congress, since Congress often adds resources to the president’s budget request, predominantly in the 6.2 and 6.3 program elements in support of specific projects or capabilities.
Debates often rage during annual budget formulations as to whether a specific top-down mandated 6.1 budget is over- or underfunded. Natural tensions tend to want to decrease 6.1 budgets to provide more resources for 6.2 and 6.3 (i.e., exploratory and advanced technology development), respectively, to meet more specific, shorter-term needs or desired capabilities. Yet, history has shown that fencing and protecting a basic research budget as a means of providing “seed corn” to the technology developers and encouraging the pursuit of high-risk, very-high-payoff endeavors is critical to unimagined future capabilities. The protective fencing of an overall 6.1 budget for high-risk, high-payoff research does not imply that the portfolio of individual projects should not or cannot be actively managed. On the contrary, there will be more ideas for basic research projects than there are resources for pursuing them, and actively managing the basic research portfolio is necessary to ensure that the quality and relevance of the assigned projects is maximized. A basic research portfolio plants the seeds for future technologies, cultivates successful emerging capabilities, and eliminates less successful efforts.
The services typically manage their basic research budgets through separate organizational entities vis-à-vis the organizations that manage the remainder (and much larger portion) of the S&T budget. For instance, the Air
Force Research Laboratory (AFRL) has nine technology directorates, each of which manages the 6.2 and 6.3 budgets for its respective technology discipline (e.g., sensor technology, directed energy technology, information technology, propulsion technology, etc). But the 6.1 budget is managed separately by the AFRL’s Air Force Office of Scientific Research (AFOSR).
As the sole 6.1 budget manager for the Air Force, AFOSR is organized along scientific disciplines (e.g., chemistry and life sciences; aerospace and materials; physics and electronics; and math and geosciences), and its project managers seek discovery of new phenomena in these disciplines that could enable potential game changers when translated to a technology that can, in turn, provide a significantly improved military capability. AFOSR establishes the strategic objectives and investment strategy for the Air Force 6.1 budget, and AFOSR project managers then allocate that budget to universities (who are awarded the majority of the basic research portfolio), the AFRL technology directorates, or industry for execution. One rationale for a separate agency to manage this investment is the realization that managing basic research projects, with their unfettered and open approach where failure can occur as often as success, is different from managing technology development projects that often have performance and schedule milestones typically associated with more definitive and specific projects. Another rationale is that mixing the basic research budget with the technology budget in the same organization could lead to migration of the basic research budget into technology development activities, especially when technology projects (which have more definitive milestones because of their higher TRLs) encounter technical difficulties that lead to cost and schedule overruns.
Rising Above the Gathering Storm Report Recommendations
Another source regarding the amount that R&D-related government agencies should invest in high-risk, high-payoff endeavors can be found in the seminal report Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future.2 The study’s rationale was based on a widespread consensus among scientific and technological leaders throughout the United States that the nation’s science and innovation enterprise was either stalled or in decline, with the consequence of jeopardizing future U.S. prosperity. Accordingly, the study committee was asked to identify the top 10 actions that federal policy makers could take to enhance the science and technology enterprise so that the United States can successfully compete, prosper, and be secure in the global community of the 21st century. Significantly, one of those actions concerned basic research, and the need not only for a higher level of investment but also for additional emphasis on high-risk research endeavors. Specifically:
Recommendation B of Gathering Storm says the following: “Sustain and strengthen the nation’s traditional commitment to long-term basic research that has the potential to be transformational [emphasis added] to maintain the flow of new ideas that fuel the economy, provide security, and enhance the quality of life” (p. 7). The phrase “that has the potential to be transformational” can be equated to “high risk, high payoff” and thus tends to validate the premise from the earlier benchmarking discussion that long-term basic research is indeed the appropriate budget element to sponsor high-risk research.
Of perhaps greater significance in the context of this particular benchmarking discussion, Action B-4, entitled “High-Risk Research,” states the following: “At least 8 percent of the budgets of federal research agencies should be set aside for discretionary funding managed by technical program managers in those agencies to catalyze high-risk, high-payoff research” (p. 149). The discussion accompanying this recommended action points out that an important subset of basic research is the high-risk or transformative research involving new theories, methods, or tools that are often developed by new investigators—the group most likely to generate radical discoveries or new technologies.
The Gathering Storm report highlights two additional points relevant to an R&D entity’s resource allocation for high-risk, high-payoff research:
It is not necessarily a matter of providing additional resources for high-risk research, but rather providing incentives for program managers to fund high-risk research out of a discretionary portion of the existing budget. Lack of incentives for (or barriers to) performing high-risk research include (1) a peer review system that tends to favor established investigators using well-known methods; (2) pressure from customers and management for short-term results; and (3) risk adverseness, since high-risk projects are prone to failing and increased government and public scrutiny make “projects that fail” increasingly untenable. The study points out that partly because of these barriers, the percentage of efforts allocated to high-risk research is often quite small—1 to 3 percent being common. Individual committee members, based on their long-standing expertise in this area, believed that additional discretionary funding within existing budgets between 5 and 10 percent should be devoted to high-risk research. Thus, 8 percent seemed a reasonable compromise and is reflected in the committee’s recommended Action B-4 discussed earlier. Significantly, the committee also pointed out that the degree to which a high-risk research program will be successful depends heavily on the quality of the program staff.
The study indicates the importance of government involvement in high-risk basic research. Specifically, while federal government spending in R&D has declined over the past several years, corporate R&D spending has dramatically increased and, in fact, has become the linchpin of the U.S. R&D enterprise. However, corporate R&D funds relatively little basic research, since basic research does not typically offer immediate benefits to its sponsor. The study states that basic research, by its nature, is almost by definition risky (emphasis added), and shareholder pressure for short-term results discourages long-term, speculative investment by industry.
To summarize, the consensus of the Gathering Storm committee is as follows:
Approximately 8 percent of a government entity’s S&T budget should be allocated to high-risk, high-payoff research.
Government agencies must take the lead in funding such research, since industry R&D is predominantly short-term in nature.
Additional government budget is probably not needed; rather, it is a matter of ensuring that 8 percent of the existing S&T budget is discretionary and fenced for high-risk research, and that the barriers which discourage government program managers to pursue high-risk research are eliminated for this portion of the S&T investment portfolio.