2

Governance of the MDV Enterprise

This section of the report addresses how the monitoring, detection, and verification (MDV) mission is currently carried out across the organizations involved, and how this governance could be improved.

Figure 2-1 illustrates the complex interactions that make up the MDV mission space. A network of information sources and technical capabilities provides the monitoring data that when analyzed detect proliferation or non-compliant behaviors. This information can be used by decision makers to assess compliance with treaties and agreements (verification) and/or inform a response, whether diplomatic, military, or otherwise.

2.1 POLICY, OPERATIONS, AND RDT&E INTEGRATION

As shown in Figure 2-1, for MDV research, development, test, and evaluation (RDT&E) to be most relevant, it must be well-connected to the national security policy-making processes. The success of nonproliferation and arms control agreements depends on technology attained through RDT&E. Such technology often requires long lead-times, and may involve significant investment and some technological risk taking. It is therefore vital that technology providers and policy makers work together to prioritize and anticipate future agreements and future technologies with sustained commitment.

2.1.1 Need for a Formal Interagency Planning Process

As outlined in Section 1.5, the MDV enterprise is composed of a diverse set of federal departments and agencies with RDT&E, operational, and/or policy

roles and responsibilities. No one organization is responsible for the enterprise as a whole, and MDV is currently not managed as a unified mission space. This distributed structure makes a high level of coordination imperative. To effectively manage this key national security endeavor, there needs to be a rigorous, systematic, requirements-driven structure that establishes national level priorities and associated requirements, determines an investment plan governing operations and development of new capabilities, and ensures that new technologies meet requirements and fill mission needs.

The need for this type of formalized process was previously identified by the 2014 DSB report, which stressed the need for “addressing the problem ‘whole’” and made the recommendations shown in Box 2-1. It remains unclear to the committee how this set of DSB recommendations has been acted upon;¹ this issue will be revisited in the final report.

The committee heard repeatedly in briefings from multiple agencies that routine interagency interactions and coordination processes keep all parties informed of needs and priorities. These interagency and intra-agency coordination processes seem to take various forms from National Security Council (NSC) working groups to manager-to-manager and staff-to-staff interactions. The national laboratories enhance the communication by performing and reporting on R&D sponsored by multiple different agencies and multiple offices or organizations within an agency. Both federal and laboratory personnel interact with the

___________________

¹ Presidential Policy Directive-33 (PPD-33), Detection and Early Warning of Nuclear Proliferation (August 2015), established interagency integration measures, but the committee was unable to assess these measures for this report.

International Atomic Energy Agency (IAEA) and Comprehensive Nuclear-Test-Ban Treaty Organization, gaining insights about needs and sharing recent technology advances. Based on data gathering in this phase of the study, the committee concluded that current coordination across the MDV enterprise is somewhat effective at the strategic and tactical levels, but that coordination is heavily reliant upon informal interactions and dependent on personal relationships, making it fragile and transient.

The MDV enterprise needs a formal, recurring process to anticipate, identify, discuss, prioritize, and communicate MDV needs for nonproliferation and arms control. Such a process should

• identify MDV shortcomings and technology gaps in the near and mid-term,
• create a regular process to bring the community together to anticipate upcoming technology needs over the 10- and 20-year time horizons,
• consider impacts and benefits of disruptive technologies, and
• develop and oversee the implementation of a coordinated MDV RDT&E strategy.

The committee was unable to fully assess whether such a process already exists (and potentially needs to be strengthened or improved).² However, the committee did not hear a common articulation of MDV near-term, mid-term, or long-term gaps or needs. Therefore, the committee will revisit this question in the final report.

___________________

² The committee will assess interagency integration measures established by PPD-33 in the final report.

While the National Counterproliferation Center (NCPC) has a clear role as the counter and nonproliferation focal point within the intelligence community (IC), it remains unclear to the committee how much NCPC’s coordination role extends beyond the IC to the MDV enterprise more broadly.³ At some level, NCPC interacts with agencies to understand their MDV needs and identify technology gaps, and provides some R&D funding of their own to help fill gaps (NCPC, briefing to committee, October 2, 2020). NCPC recently initiated a study to cross walk current R&D programs with the requirements framework, proliferation trends, and current and emerging technologies to identify gaps and needs (ibid.). This study does not appear to be part of a regularly recurring process, as the last study was conducted more than 10 years ago. However, it could potentially be a foundation for a future recurring interagency process to discuss proliferation trends, identify needed innovations, examine impacts of emerging and perhaps disruptive technologies, and guide applied research.

2.1.2 Anticipating Future Challenges and Needed Capabilities

The interagency process described above must involve anticipating a wide range of potential future challenges and the capabilities needed to meet them. As noted in the previous section, the committee did not see evidence of a coordinated interagency effort to identify these challenges and needs but learned about threat analysis efforts by individual agencies. NCPC noted that they conduct anticipatory threat analysis “out to 2035 and beyond” (NCPC, briefing to committee, October 2, 2020). The National Nuclear Security Administration (NNSA) historically conducted anticipatory threat analysis as well, through the NNSA Over the Horizon Initiative (see Appendix H), which was active from 2010 to 2014. However, the committee perceived that at least in recent years, the NNSA Office of Defense Nuclear Nonproliferation Research and Development (DNN R&D) relies heavily on laboratory scientists to anticipate what technologies will be needed to meet future needs.

Anticipating long-term (10–20 year) challenges is particularly difficult because addressing more immediate needs take precedence within the MDV enterprise.⁴ Maintaining momentum that transcends changes in administrations is an additional challenge but is essential since the R&D timeframe is longer than the swings in presidential policy guidance and the real-world environment. An external MDV advisory board could help increase focus on this longer timeline and bring new ideas to the MDV enterprise.⁵ Such an advisory board, broadly composed of experienced individuals outside the government with diverse talents

___________________

³ The committee will further assess the role of NCPC in the final report.

⁴ One DNN R&D official noted that NNSA’s interagency partners are operationally focused, which makes it challenging to look 10+ years into the future as an interagency (DNN R&D, briefing to committee, September 21, 2020).

⁵ It is important to note that the establishment of an external MDV advisory board would not negate the need for a formal government planning process. An advisory board can provide useful external input, but the responsibility for strategic MDV planning ultimately lies with the government.

and backgrounds, could assess the changing proliferation and arms control environment, identify gaps and needs, actively seek and identify new approaches and technologies to address anticipated challenges and evolving threats, provide feedback on current R&D programs, and suggest new avenues for exploration, including ideas beyond current mainstream thinking. An advisory board could also facilitate the incorporation of commercial technologies where appropriate. A possible draft charter is provided in Appendix I.

2.1.3 Developing a Coordinated R&D Strategy

Developing a coordinated R&D strategy first requires that requirements are communicated from operational and policy agencies like the Air Force Technical Applications Center (AFTAC) and the Department of State Bureau of Arms Control, Verification, and Compliance (DoS/AVC) to capability providers like DNN R&D. This is currently done via a mix of formal and informal communication. DoS/AVC releases an annual verification needs document. AFTAC releases a detailed technical roadmap identifying R&D needs out 20 years and conducts an annual AFTAC R&D Roadmap Forum, bringing together agency leadership and program managers;⁶ R&D providers in the national laboratories, academia, and industry; and various operational users. While DNN R&D and the Defense Threat Reduction Agency (DTRA) leadership communicated to the committee that needs and priorities are very well understood between U.S. interagency partners (DNN R&D, communication to committee, October 14, 2020; DTRA/RD, communication to committee, October 8, 2020), DoS/AVC communicated that AVC’s efforts to leverage partner agencies’ R&D programs via its verification needs document have been largely ineffective, due to a lack of sustained advocacy and the fact that each agency or R&D program has its own priorities (DoS/AVC, communication to committee, November 25, 2020).

Once MDV needs have been communicated, it is essential that the MDV R&D enterprise develop a coordinated strategy to meet them while minimizing duplication or gaps. The committee learned of two mechanisms that exist for interagency R&D planning and coordination: procedures established by PPD-33 and the Nuclear Defense Research & Development Subcommittee (NDRD). DoS/AVC communicated to the committee that there is currently a lack of regular MDV R&D coordination at a senior level, and that PPD-33 mechanisms are insufficient for coordination (DoS/AVC, communication to committee, November 25, 2020).⁷

The committee assessed the NDRD’s role in R&D planning and coordination for this interim report. The NDRD is a subcommittee of the National Science and Technology Council (NSTC) Committee on Homeland & National Security. The NDRD, which is co-chaired by the Department of Defense (DoD), Department of

___________________

⁶ Including NNSA, DTRA, the Air Force Research Laboratory (AFRL), and DoS.

⁷ The committee will assess the sufficiency of PPD-33 mechanisms in the final report.

Energy (DOE)/NNSA, and the Office of Science and Technology Policy (OSTP),⁸ occasionally develops a strategic roadmap or plan⁹ with input from the relevant agencies involved in nuclear defense R&D.¹⁰ The most recent strategic plan was released in December 2019 and examines the FY2020–2024 timeframe (NSTC, 2019). The plan is intended to coordinate federally funded R&D over the next five years in order to avoid duplication and redundancy (NSTC, 2019).

OSTP noted that the preparation of the plan is fundamentally a consensus-building exercise, and that agency buy-in occurs during the development of the report, when agencies present what they are currently working on and identify areas of needed support (OSTP, briefing to committee, September 21, 2020). However, the resulting plan does not indicate what efforts, if any, are already underway to address the identified R&D gaps. The 2019 plan is thus more of a communication of high-level policy than an interagency plan, and it is unclear to what extent the actual product, versus the process of developing it, has continued utility to the interagency to avoid duplication and redundancy or to fill gaps.¹¹ Notably, previous roadmaps published in 2008 and 2012 include success metrics and timelines associated with each R&D priority, as well as a summary of the current funding allocated to each topic area.

The utility of the NDRD plan as an interagency guiding document is further hindered by an irregular update and release schedule. The plan provides a five-year outlook of R&D priorities, suggesting that it should be produced no less frequently than every five years to ensure that there are not any gaps in strategic planning. However, the 2019 plan was released seven years after the 2012 roadmap,¹² which only provided outlook through 2017.

Individual agencies conduct strategic planning consistent with the NDRD plan, but it is not necessarily integrated across the interagency. In addition, while

___________________

⁸ The other agency members of the NDRD are the Department of Health and Human Services, the Department of Homeland Security, DoS, the Environmental Protection Agency, the Missile Defense Agency, the National Institute of Standards and Technology, NSC, the Office of the Director of National Intelligence, and the Office of Management and Budget.

⁹ The 2019 report was called a “strategic plan,” while previous iterations in 2012 and 2008 were called “strategic roadmaps.”

¹⁰ The NDRD addresses the broader scope of nuclear defense beyond just MDV. The NDRD Strategic Plan lays out five mutually reinforcing elements of the United States’ overall nuclear defense posture: (1) nonproliferation and verification; (2) detection and attribution; (3) nuclear deterrence; (4) missile defense; and (5) resilience, response, and recovery. The first two of these are relevant to the present study.

¹¹ The NDRD is not significantly involved in the implementation of the strategic plan, and the NDRD and NSTC more broadly do not seek to manage or oversee agency activities. The plan is not directive and is provided to the agencies as a resource for their coordination and planning, intended to inform, but not replace, agency-level processes. Beyond participating in the drafting and coordinating of the strategic plan, there does not appear to be ongoing interagency coordination through the NDRD to implement the guidance within their portfolios (DNN R&D, communication to committee, October 9, 2020).

¹² The effort to develop the current plan began in 2018, following the release of the 2017 National Security Strategy and the 2018 Nuclear Posture Review.

the committee saw evidence of short- and mid-term planning, a lack of long-term strategic planning was noted. Long-term planning is especially pertinent for DNN R&D, which focuses on low-technology readiness level (TRL) R&D and thus must plan for developing capabilities well in advance of formal requirements. To focus its low-TRL R&D efforts, DNN R&D prepares/renews Goals, Objectives, and Requirements (GOR) documents with a six-year time horizon for each of its topic areas¹³ to provide general guidance to the portfolio managers.¹⁴ The GOR preparation process engages various agency stakeholders and the national laboratories (DNN R&D, communication to committee, September 18, 2020). However, the committee noted that limiting strategic planning largely to the next six years may not be sufficient for planning long-term R&D.

2.1.4 Executing a Coordinated R&D Strategy

In addition to long-term strategic planning, the MDV enterprise needs to continuously assess progress made across the interagency and any changes in the threat environment and/or national polices. It is important for organizations to stay abreast of work done in other agencies. Much of this coordination is done informally at the Program Manager and Office Director level.

The interagency also conducts a number of technical and program reviews (see list in Appendix J), which are sometimes used to connect end users and capability providers. Some meetings, such as the annual AFTAC R&D Roadmap Forum, engage a broader audience including cleared academic and commercial partners. Engaging these partners is important to build networks and make rapidly advancing commercial fields such as data science (see Section 3.5) more accessible to MDV researchers and operators.

The committee noted that the nuclear test monitoring community previously held more widely accessible reviews that engaged a large number of academic and commercial partners. The large-scale Monitoring Research Review (MRR)¹⁵ held in the past has been replaced in recent years by multiple smaller-scale meetings (Nuclear Explosion Monitoring, AFRL Technical Interchange Meeting) that are more limited in attendance. University engagement is now focused on the DNN R&D’s University Program Review, which involves presentations from faculty, students, and national laboratory researchers involved in the three university consortia (see Section 2.2.3 for more information on the DNN R&D consortia).

In addition to these meetings, DNN R&D also stressed that the national test beds (see Section 2.2.2) play an important role in interagency coordination

___________________

¹³ The DNN R&D topic areas are Nuclear Weaponization and Material Production Detection, Nuclear Weapons and Materials Security, Space-based Nuclear Detonation Detection, Ground-based Nuclear Denotation Detection, and Forensics-based Nuclear Detonation Detection.

¹⁴ DNN R&D provided the committee with several GOR documents published in 2013–2015 and noted that new GOR documents are currently in draft, delayed somewhat by the COVID-19 pandemic.

¹⁵ From 1978 to 2006, the MRR was called the Seismological Research Review.

(DNN R&D, briefing to committee, September 21, 2020).¹⁶ Interagency partners are able to use the test beds free of charge (funding national laboratory participants, if relevant). Coordination meetings are held in the months prior to a campaign to make a detailed operations plan that satisfies all users’ needs (DNN R&D, communication to committee, September 18, 2020). By providing something to test against, the test beds create a standard baseline for the community. Capabilities developed in one agency can be tested in the presence of others, ensuring that the community is unified in terms of understanding current capabilities and remaining gaps (DNN R&D, briefing to committee, September 21, 2020).

2.1.5 Finding and Recommendation

Finding 1. Responsibility for the MDV mission is distributed across the government, demanding a high level of interagency coordination. However, the interagency process to assess long-term MDV trends and technology needs is largely informal and does not appear to occur on a regular schedule. As a result, there is no meaningful strategic planning process that produces long-term (10- to 20-year) MDV problem-sets and capability needs to guide the whole R&D community.

Recommendation 1. The NSC should ensure that there is an enduring, interagency planning process with a consistent periodicity to characterize potential future MDV challenges, assess the adequacy of current MDV capabilities to address these challenges, develop strategic guidance for R&D planning, and advocate for funding. The process should involve the following:

1. Conducting regular updates to the Nuclear Defense Research and Development Strategic Plan (every four years), which should contain success metrics and timelines.
2. Establishing an external advisory board to recommend priorities for nonproliferation and arms control MDV R&D. The board should be composed of experts who collectively have familiarity with the government agencies involved in MDV, as well as the national laboratories, academia, industry, and MDV user communities. The board planning horizon should be 10–20+ years. A possible draft charter is provided in Appendix I.¹⁷

___________________

¹⁶ The 2014 DSB study called for a national MDV testing capability that could act as a “focal point for planning, iterating/adapting, and operating” a comprehensive monitoring regime that is too complicated to plan or assess on paper. The test beds would connect developers and users and help integrate the various activities in the MDV mission space (DSB, 2014, p. 66).

¹⁷ The DoS International Security Advisory Board and President’s Foreign Intelligence Advisory Board may serve as possible models.

2.2 STEWARDSHIP OF MDV CAPABILITIES

The need for nonproliferation stewardship is analogous to the need for stockpile stewardship in the nuclear weapons program. Implicit knowledge, which is knowledge that is not written down but comes from building and working directly with nuclear facilities, equipment, materials, and devices, is invaluable to nonproliferation work. The U.S. nuclear weapons complex no longer has large-scale indigenous uranium enrichment and plutonium production facilities that can provide hands-on experience and opportunities to test MDV capabilities.¹⁸ Individuals with this hands-on experience are retiring and being replaced with a new generation of scientists and engineers, whose experience is largely limited to the laboratory rather than nuclear material production facilities. NNSA/DNN R&D stressed that due to a loss of facilities and expertise, the United States may be at risk of losing critical competencies that are necessary to recognize early signs of nuclear proliferation (DNN R&D, briefing to committee, September 21, 2020).

2.2.1 DNN R&D Nonproliferation Stewardship Program

NNSA recognized that an intentional, systematic capability development and maintenance effort is necessary to ensure that the enterprise is ready to respond to future challenges and has taken encouraging steps in recent years to address this issue. In FY2020, NNSA established a new Nonproliferation Stewardship Program (NSP) within DNN R&D. The program is growing, from $22.5 million in funding in FY2020 to $60 million in FY2021 (DNN R&D, communication to committee, January 14, 2021). The goal of the NSP is to build competency in three capability areas: (1) enabling infrastructure, (2) enabling science and technology, and (3) an expert workforce. There is a particular emphasis on the test beds and other infrastructure that can be used to both further R&D and provide hands-on experience to build workforce expertise.

The creation of this program is a promising step in the right direction. Its success is critical and should be regularly reviewed. DNN R&D noted that the NSP will “annually measure progress and report to the Secretary of Energy the ability of DOE’s laboratories, sites, and plants to support a broad range of nonproliferation missions throughout the USG [U.S. government]” (DNN R&D, communication to committee, October 14, 2020).¹⁹

___________________

¹⁸ The United States has had no U.S.-owned domestic enrichment capability since the shutdown of the Portsmouth and Paducah gaseous diffusion plants in 2001 and 2013, respectively (DOE, 2020a, 2020b). All U.S. plutonium production reactors were shut down in 1987 (IPFM, 2020). The United States has relied on international cooperation to compensate for this lack of facilities. For example, as described in Box 2-3, the on-line enrichment monitor technology was tested at URENCO facilities in New Mexico and the Netherlands.

¹⁹ The NSP effort will require communication and collaboration between NNSA DNN and other offices, such as the DOE Office of Science, which operate some of these facilities.

2.2.2 MDV Test Beds

Another significant stewardship effort by DNN R&D, DTRA, and other interagency partners across the MDV enterprise in recent years has been the establishment of experimental test beds.²⁰ The DNN R&D test beds serve three basic functions. First, DNN R&D uses them for RDT&E and data collection. For example, test beds are used to help understand signatures and develop technologies and capabilities to collect and interpret those signatures. Second, they are available at no cost to interagency partners to develop collection technologies and test technologies before they are deployed in the field. Finally, the test beds also serve a training role for personnel to gain experience and develop expertise with technologies. For example, the Office of Nonproliferation and Arms Control Office of Nuclear Verification (NPAC/ONV) has taken advantage of a test bed to train inspection teams. Student researchers from the DNN R&D university consortia (see Section 2.2.3) can also conduct research at some test beds. Usage of the test beds depends on the nature of the test bed. Some operate in campaign mode and others, particularly those taking advantage of operating DOE facilities like the High Flux Isotope Reactor (HFIR), operate continuously (DNN R&D, communication to committee, September 18, 2020).

Table 2-1 lists the current test beds that the committee heard about for this interim report. These test beds cover a significant portion of the processes a proliferator would need in order to develop a nuclear weapon. In addition to the test beds already established, DNN R&D and NPAC are currently exploring the establishment of a test bed focused on arms control (DNN R&D, communication to committee, September 18, 2020). The committee also heard ideas for new test beds from the national laboratories.

The test beds are an innovative use of the DOE/NNSA nuclear weapons complex to provide facilities to the proliferation research community writ large. The development of universally accessible test-beds provides needed, cost-effective, domain-relevant venues for researchers to use in their nuclear proliferation RDT&E programs. In the era without nuclear testing, and an increasingly complex proliferation environment, these test beds offer a unique opportunity to increase understanding, explore new technologies, and transfer expertise to the next generation of scientists and technologists.

The committee notes potential similarities between the test beds and the DOE Office of Science (DOE/SC) national user facilities, which draw a diverse set of researchers from around the world and are a hallmark of DOE. DOE/SC may offer valuable insight on operating a system of user facilities as the test bed concept continues to grow. The test beds have the potential to be a similar hallmark for NNSA DNN and the broader MDV community.

___________________

²⁰ The development of test beds is responsive to the 2014 DSB report, which recommended that the enterprise (specifically DTRA, NNSA, and the IC) establish a national testing capability to iterate and adapt both existing and new MDV capabilities (DSB, 2014).

TABLE 2-1 Summary of NNSA^a Test Beds as Briefed to the Committee, in Order of Year Established

^a Some of these test beds were developed and operated with additional funding from other agencies, such as DTRA. DHS has also developed test beds that could be used to support the MDV mission; the committee did not conduct data gathering on these test beds for this interim report.

^b The first Moran campaign was in 2017 (DNN R&D, communication to committee, November 3, 2020).

SOURCE: DNN R&D, communication to committee, October 14, 2020; November 3, 2020; and January 14, 2021.

To further develop the test bed concept, it is important to look for opportunities to enhance the test beds’ effectiveness. External review of the test beds and/or red-teaming (see Box 2-2) at test beds could be constructive.²¹ The 2014 DSB report also notes that test bed infrastructure should not be limited to physical (or virtual) facilities but should also include data/information management systems. The committee assessed that DNN R&D has recognized this need in the development of some of the test beds (see Section 3.5.2 for more on the need for data pipelines).

___________________

²¹ The 2014 DSB report recommended that a national MDV testing capability involve standing White and Red Teams (see Box 2-2).

2.2.3 Workforce Development

An important component of stewardship is maintaining expertise, particularly as the MDV workforce continues to age.²² Key disciplines such as nuclear chemistry, radiochemistry, material science, nuclear physics, seismology, infrasound, and hydroacoustics are essential to the MDV mission, but many of these fields have seen reductions in university programs and the number of graduates.²³ Specific facilities are also needed for the education and training of future scientists in MDV; for example, university research reactors are important for training in nuclear engineering, radiochemistry, and related fields.

In an environment of ever-increasing amounts of data, it is also becoming apparent that the MDV enterprise needs to hire more computer and data scientists, including cybersecurity and data authentication experts. The DNN R&D Office of Proliferation Detection (PD) FY19 Annual Report notes that a lack of qualified personnel across the national laboratory complex continues to be a significant challenge in executing one of its flagship multi-laboratory “Venture

___________________

²² Forty percent of DOE/NNSA’s workforce will be eligible for retirement by 2023, according to the FY2020–2024 NNSA Plan.

²³ For example, see NASEM (2012). This is a cyclical challenge; as students are drawn to more lucrative fields, such as computer science, university nuclear programs contract, making it more difficult to recruit new students.

projects”²⁴ focused on advanced data analytics for proliferation detection (DNN R&D, 2019a). The number of students studying data science, including statistical and computational sciences, has skyrocketed in recent years (Russell, 2018); the challenge is recruiting these students to the MDV mission space. Private companies can offer much higher compensation than can the national laboratories or government agencies involved in the MDV enterprise.²⁵ The enterprise must rely on the appeal of the mission over the lure of money.

The need for strengthened workforce development was recognized when Congress established an Integrated University Program (IUP)²⁶ implemented by DOE, NNSA, and the Nuclear Regulatory Commission (NRC). Each organization was appropriated $15 million annually from 2009 to 2019 to “support university research and development in areas relevant to their respective organization’s mission, and . . . to support a jointly implemented Nuclear Science and Engineering Grant Program that will support multiyear research projects that do not align with programmatic missions but are critical to maintaining the discipline of nuclear science and engineering” (ibid.) The IUP was renewed as the University Nuclear Leadership Program in the 2021 Consolidated Appropriations Act, maintaining the same funding to DOE, NNSA, and the NRC.²⁷

NNSA’s contribution to the IUP is managed out of DNN R&D, which currently funds three concurrent university consortia that support the MDV mission.²⁸ The committee focused on the DNN R&D university consortia in this interim report due to their direct relevance to the MDV mission, but note that

___________________

²⁴ DNN R&D’s Venture model, established in 2015–2016, focuses significant resources toward a small number of large-scale multi-laboratory projects, “Ventures,” many of which are associated with test beds. The Venture model promotes innovation by bringing together a diverse group of researchers across the laboratory complex to focus on long-term R&D challenges. Some of these projects have originated at the laboratories, while others are the result of top-down direction from DNN R&D (DNN R&D, communication to committee, October 14, 2020 and November 3, 2020).

²⁵ In 2019, an entry-level software engineer at Google with a B.S. could expect to make ~$189,000 in total compensation ($124,000 in salary and $43,000 in stock compensation), according to a crowdsourced Silicon Valley salary dataset (CNBC, 2019).

²⁶ Omnibus Appropriation Act, 111th Congress, Pub. L. No. 111-8 § 313 (2009).

²⁷ Consolidated Appropriations Act, 116th Congress, Pub. L. No. 116-260 § 313 (2020).

²⁸ DNN R&D previously funded university research through broad agency announcements. The move to funding university research through a consortia model has had both positive and negative impacts in terms of workforce development and innovation that deserve further attention beyond this study. Some potential limitations of the consortia model are discussed in Section 2.3.3, which addresses how the consortia are leveraged for MDV R&D.

other workforce development efforts at DOE/NNSA,²⁹ the NRC, DTRA,³⁰ DHS, and other agencies also support the MDV mission by developing talent in the nuclear sciences and other relevant fields.

DNN R&D’s primary objectives for its university consortia (as opposed to any particular consortia goals) are to

• develop the nation’s intellectual capital in nuclear science and engineering (NSE) and other scientific disciplines relevant to nuclear nonproliferation,
• support NSE and nuclear nonproliferation R&D programs at universities and integrate these programs with the national laboratories and U.S. industry, and
• build broader support for nonproliferation funding.³¹

Starting in 2011, the DNN R&D IUP initially supported one consortium, the Nuclear Science and Security Consortium (NSSC), led by the University of California, Berkeley. This consortium was renewed for an additional five years in 2016 and will end in 2021 (NSSC II).³² The NSSC’s primary objective is to recruit and train top students in relevant nuclear disciplines to support

___________________

²⁹ NNSA has several other workforce development efforts beyond the DNN R&D consortia, including the stockpile stewardship university consortia and the NNSA Graduate Fellowship Program that the committee did not focus on in this report. Another NNSA MDV workforce development effort is the Human Capital Development (HCD) subprogram of the Next Generation Safeguards Initiative in DNN/NPAC, which is funded at roughly $6.5 million annually (NPAC, communication to committee, November 18, 2020). The goal of the HCD subprogram is to recruit and train the next generation of international safeguards professionals in order to “maintain IAEA safeguards knowledge in the national laboratories and also develop a pipeline of qualified and knowledgeable Americans to work at the IAEA on safeguards” (NNSA, 2019, pp. 2–37). NNSA notes that this program is important because there is a shortage of international safeguards experts in the national laboratories due to the contraction of the nuclear industry in the United States (ibid.).

³⁰ DTRA recently established two university consortia of its own, modeled in part on the DNN R&D consortia. Two University Research Alliances (URA) were awarded in 2020 (Gardner, 2020). These URA are not specifically focused on MDV research like the NNSA/DNN R&D consortia, but each includes some relevant research. The Interaction of Ionizing Radiation with Matter URA, led by Pennsylvania State University, will involve research on improving both near-field and remote radiation detection. The Materials Science in Extreme Environments URA, led by Johns Hopkins University, will involve research to better understanding nuclear fireballs for nuclear detonation detection and post-detonation nuclear forensics. Both consortia involve workforce development efforts. The committee did not focus on these consortia in this interim report due to time constraints and because the consortia are in a very early phase of execution.

³¹ In information provided to the committee on November 2, 2020, DNN R&D noted, “Congressional delegations highly value university funding directed towards their districts and they pay attention to it. University systems not receiving these grants, especially in states and districts that include a national laboratory, also petition their representatives to consider expanding the program. The university consortia are thus very effective at raising the profile and visibility of the nonproliferation mission with Congress.”

³² At the time of this report, this consortia is being re-competed.

nonproliferation and nuclear security and to provide students the opportunity to directly engage with the national laboratories.

In 2014, two additional consortia were added, the Consortium for Nonproliferation Enabling Capabilities (CNEC) led by North Carolina State University and the Consortium for Verification Technology (CVT) led by the University of Michigan. These consortia were funded from 2014–2019, and both received no-cost extensions until 2020. Unlike NSSC, which focuses broadly on training students in nuclear science, CNEC and CVT were directed by DNN R&D to focus on specific topics in support of the DNN R&D mission. In 2019, these two mission-focused consortia, revised as the Consortium for Monitoring, Technology & Verification (MTV) and Consortium for Enabling Technologies & Innovation (ETI), were re-competed. MTV stayed under the leadership of the University of Michigan, while ETI is led by the Georgia Institute of Technology. Focus areas of these new consortia include proliferation signatures, nuclear explosion monitoring, advanced manufacturing, data science, novel sensors, and robotics. A more detailed summary of the three currently funded consortia (NSSC II, MTV, and ETI) is provided in Appendix K. Each of the consortia are funded at $5 million annually for five years.

DNN R&D noted that consortia are intended to bridge the academic and national laboratory knowledge bases (all have partnerships with national laboratories), which is primarily done by having university student researchers conduct research at the laboratories under a laboratory scientist(s) mentorship (DNN R&D, communication to committee, October 14, 2020). The laboratories do not receive funding from the consortia but rather are expected to use existing DNN R&D funding to support their involvement with the consortia.³³ DNN R&D does provide a small amount of funding ($1 million annually in total, across all laboratories) directly to DOE national laboratory team members to administer and assist in managing the laboratory-university interactions, but this laboratory funding does not support the associated research or the mentoring of consortia postdocs and students. Providing the national laboratories with additional funding may help ensure that the maximum benefit of the current consortia investments is realized to build a robust workforce pipeline and ensure quality research mentoring and lasting collaborations are created.³⁴

___________________

³³ This includes overseeing the integration of student internships, providing training/oversight for students working in nuclear laboratories or with laboratory equipment, and providing staff to mentor students during their time at the laboratory.

³⁴ In information provided to the committee on October 27, 2020, DNN R&D noted that the funding mechanism will need to be outside the consortia because there are good reasons behind barring consortia from sending funding to the national laboratories including (1) the grants are mandated by Congress to support universities, not the national laboratories; (2) the intent is to integrate the university faculty and students into ongoing research already supported by DNN R&D at the laboratories; and (3) the indirect costs of administering laboratory funds through a university are prohibitively high.

The consortia have funded many students,³⁵ but it is not clear whether this represents a significant increase compared to the baseline of results before the consortia model began, or whether these numbers are meeting the actual workforce needs of the MDV enterprise. The consortia track a series of metrics (e.g., number of students supported; number of degrees granted by level; and number of supported personnel accepting positions in academia, industry, or national laboratories), but DNN R&D noted that these metrics “are not meant to serve as a means to compare consortia performance or provide a threshold goal for achievement. Rather, they are a means to assess the effectiveness of the overall university program and can be used to help identify areas for improvement or future emphasis” (DNN R&D, communication to committee, October 27, 2020). Without benchmarks associated with the metrics, it is difficult to assess the success of the consortia.

Other organizations may serve as a useful model for establishing metrics and benchmarks for the consortia.³⁶ The National Science Foundation (NSF) operates several large multi-institutional centers out of its Engineering Education and Centers Division (NSF, 2021a). The types of centers range from Engineering Research Centers (NSF, 2021c) to Materials Research Science and Engineering Centers (NSF, 2021b) to Science and Technology Centers, among others. As required by the sponsor, these centers report their metrics, similar to those of the DNN R&D consortia. Additionally, the centers benchmark their performance against historical norms across the centers for items such as technical output, diversity, outreach impact, etc. Given that the DNN R&D consortia program has been operating for nearly a decade, it would be appropriate to establish historical benchmarks with which new centers can compare. The consortia could also compare their benchmarks with those supported by other government agencies such as the aforementioned NSF centers or others like the DoD university centers of excellence (Apan, 2020). While the technical areas may be different, many of the workforce development goals are similar.

The consortia seem to largely be addressing MDV workforce development needs in relevant fields. Of particular note, DNN R&D has recognized the growing importance of data science and advanced manufacturing expertise to the MDV mission with the establishment of two consortia, CNEC (2014–2020) and ETI (2019–2024), that are focused on emerging technologies to support nonproliferation, including data science and advanced analytic techniques like artificial

___________________

³⁵ DNN R&D, in information provided to the committee on September 18, 2020, summarized the consortia workforce development numbers as follows: “Over the past 10 years, at least 250 undergraduates and more than 500 graduate students have been supported in part by the consortia; 440 degrees, including 169 PhDs, have been awarded. More than 250 of these students have accepted nuclear nonproliferation related jobs, including more than 115 new career placements in the national laboratories and 135 in the nuclear nonproliferation community.”

³⁶ A workforce development study could also potentially be a useful tool in developing metrics and benchmarks.

intelligence (AI) and machine learning (ML). On the other hand, the committee heard concerns about the sustainability of nuclear test explosion monitoring expertise at AFTAC and the national laboratories (AFTAC, briefing to committee, September 4, 2020, and September 23, 2020). The MTV consortium includes some research on seismological techniques, but it is a limited effort compared to previous nuclear test explosion monitoring programs. Contracted funding programs run by AFRL and DoS may be incapable of sustaining university research expertise and workforce development in this area.

2.2.4 Findings and Recommendations

Finding 2. NNSA has taken significant steps since the release of the 2014 DSB report to ensure that key MDV capabilities are sustained, especially within the DOE complex, with the development of a new Nonproliferation Stewardship Program (NSP) and the establishment of test beds.

1. The NSP recognizes the need for an intentional and systematic approach to maintaining arms control and nonproliferation capabilities within the DOE complex. Sustaining and continuously improving this program will be critical to its success.
2. The test beds are a cost-effective, innovative use of the DOE/NNSA complex to provide research facilities to the nonproliferation and arms control RDT&E community. The vision, communication, and access to the test beds have potential for improvement.

Recommendation 2. The nonproliferation stewardship and test bed programs should be expanded where appropriate and maintained as a vigorous part of the DNN R&D portfolio.

1. The NSP annual assessment of capabilities should look forward at least 10 years, be endorsed by the NNSA Administrator, and include input from laboratory/site/plant leaders on key metrics and their assessments.
2. NNSA should better develop and communicate the vision and objectives of the test beds, and assess opportunities for expanding access to all relevant parties including academic, commercial, and international partners.
3. DNN R&D should evaluate whether external review or red-teaming would enhance the test beds’ effectiveness.
4. Test beds should take advantage of experience from DOE/SC user facilities best practices.

Finding 3. The DNN R&D university consortia have focused a select subset of universities, faculty, and students on the MDV mission space. These consortia ensure five-year funding to the university programs to develop the next

generation of experts for the MDV enterprise and have supported hundreds of undergraduate, graduate, and postdoctoral students.

1. The consortia are increasingly engaging forward-looking disciplinary needs of the MDV enterprise beyond nuclear engineering, such as data sciences.
2. The committee believes the consortia are a positive element of MDV sustainment and capability development; however, without benchmarks associated with their metrics, it is difficult to assess whether or not the consortia are successfully meeting MDV enterprise needs.
3. The national laboratories are expected to bear the significant majority of the cost to oversee the integration of student internships, provide training/oversight for students working in nuclear laboratories or with laboratory equipment, and provide staff to mentor students during their time at the laboratory.

Recommendation 3. DNN R&D should continue to fund and seek continuous improvement of the university consortia. In particular, DNN R&D should do the following:

1. Incorporate best practices, including the development of benchmarks similar to other relevant university consortia programs, such as those run by NSF or DoD.
2. Ensure that there is a long-term plan for sustaining and evolving the workforce pipeline and research contributions, including how many and what consortia, in balance with other academic engagement.
3. Strengthen the connectivity between the national laboratories and the consortia by more fully involving laboratory researchers in planning and review meetings and providing funding to laboratory researchers to be fully engaged as mentors.
4. Continue to be responsive to changes in the disciplinary needs of the MDV enterprise.

2.3 INCREASING RDT&E EFFICACY AND INNOVATION

The MDV RDT&E enterprise is complex, with several agencies and offices responsible for different stages of technology development. Ensuring that new capabilities are delivered when needed requires significant coordination and collaboration between these different entities. Figure 2-2 illustrates the technology development pipeline from basic R&D (TRL 1–2) to operationally implemented mature technologies (TRL 9), with key players in each TRL range identified. This technology maturation pipeline reflects the wide spectrum of expertise and activities needed to span the technology development process to arrive at operational technical capabilities. Innovative basic research explores the science and provides solution options for consideration and integration into a workable system concept.

Through applied research and engineering, system concepts are developed into prototype systems that are tested (sometimes iteratively) to identify design and/or implementation flaws. As the system design and implementation matures, it proceeds to operational test and evaluation in realistic environments and eventually to operational capability.³⁷ Any gaps in this pipeline could prevent needed new capabilities from reaching operators and hinder execution of the MDV mission.

2.3.1 Meeting the Needs of Operational End Users

In order to be implemented, new capabilities must be transitioned from provider to user. However, many R&D projects fail to make the transition from science to engineered reality (from TRL 3–4 to TRL 7–8).³⁸ This phase of the de-

___________________

³⁷ The RDT&E process is not always sequential. Especially in the case of software development, it is common for applied R&D, development, and implementation to be conducted iteratively to improve upon an initial product with limited functionality. This model is also beneficial in instances where the technology requirements are uncertain or likely to change over time. The IAEA has embraced this iterative development model for these reasons (Finker, 2020).

³⁸ The middle part of the technology maturation process from TRL 4 through TRL 7 is euphemistically referred to as the “Valley of Death” because many technology projects do not successfully transition to operational capability. Multiple agencies noted difficulties in transitioning technologies through this phase of development. A 2017 Government Accountability Office report found that out of a sample of 91 DNN R&D and NPAC projects, 33 technologies were transitioned to end users, 17 of which were subsequently deployed. Cited reasons why technologies were not transitioned and/or deployed included the following: the technology was not developed enough, an end user had not yet been identified, the technology addressed future needs rather than an existing requirement, and additional end-user steps were needed before the technology could be deployed (GAO, 2017).

velopment process is often expensive and time consuming. Moreover, it requires constant interaction between developers and users to address issues such as treaty restrictions on the technology itself, natural and politically imposed access limitations, time available to employ the technology, software development and user interface design, certification, and authentication.

During this phase of development, the technology must also undergo extensive testing to assess reliability, maintainability, and functionality in a natural operating environment (i.e., subject to shock, vibration, temperature, humidity, etc.). Using test ranges and/or actual operating facilities as hosts for operational test and evaluation can be prohibitively expensive because of conduct of operations and safety procedures. The creation of MDV technology development test beds (see Section 2.2.2) is a significant, positive contribution to technology transition as they provide access to needed facilities and materials at no or reasonable cost to the technology developers.

A technology is more likely to successfully proceed through this phase of development if it has a champion, often the operational user, who can supply or successfully advocate for funding and other necessary resources (e.g., accommodating classification issues, facilitating test range or pertinent operating facility access, providing nuclear material). A champion is especially important because the development process usually takes longer and requires more resources than anticipated due to unforeseen roadblocks, planning errors, funding lapses, and changing requirements.

Part of the challenge of transitioning MDV technologies is that, for the most part, different organizations are responsible for low-TRL basic/applied research and for high-TRL development and operationalization.³⁹ DNN R&D/PD, the primary funder of early-TRL R&D in the MDV enterprise, rarely funds technology development above TRL 4. On the other end of the spectrum, operational users are expected to use their own RDT&E funding to operationalize high-TRL technologies. Organizations are limited in their ability to step out of these lanes,⁴⁰ whether to push or pull technologies through the mid-TRL transition.

This development gap is a particular challenge to AFTAC, which has limited R&D capabilities and relies on NNSA and the national laboratories to deliver operationally ready technologies. AFTAC is a large operational user and the majority of its funding is budgeted as operations and maintenance. The small amount

___________________

³⁹ An exception is the DNN R&D Office of Nuclear Detonation Detection, which matures space sensors through R&D to operational systems that are delivered to the Air Force at TRL 8/9. For FY2020, this program has averaged ~29.5 percent of the DNN R&D budget, a fraction that emphasizes the relative expense of crossing the valley of death.

⁴⁰ DNN R&D and NPAC noted that these “lanes” are not explicitly defined but that congressional appropriators expect agencies and departments with operationally driven requirements to fund their own high-TRL directly, while DNN R&D focuses on low-TRL R&D (with the exception of providing space-based nuclear detonation detection sensors to the Air Force at TRL 8/9 per a long-standing memorandum of understanding between DoD and NNSA) (DNN R&D, communication to committee, November 3, 2020; NPAC, communication to committee, November 18, 2020).

of R&D funding that AFTAC receives is generally insufficient to meaningfully support the cost of transitioning technologies to operational status (AFTAC, briefing to committee, September 4, 2020). DNN R&D noted that “NNSA can develop technologies for years and coordinate with end-users on transition planning, but if the requirements don’t make it through the DoD program, planning, and budget process to be funded, there can be no transition” (DNN R&D, communication to committee, October 14, 2020).

The model for transitioning low-TRL technologies to the IAEA is more successful because the NPAC Office of International Safeguards (OINS) has the role of maturing technologies to a TRL that the IAEA can accept and then providing them to the IAEA. In this model, the U.S. technology champion fills the void between early TRL R&D and operational capability. A similar organization to fill the mid-TRL transition for national nuclear test explosion or arms control MDV systems is not apparent, with the exception of space-based nuclear explosion detection sensors that are developed and matured by DNN R&D. Historically, DTRA had partially filled this role, but in the past four years or so, DTRA has pivoted away from traditional MDV R&D efforts in favor of supporting the Combatant Commands’ weapons of mass destruction counterproliferation mission.

The specific R&D processes that exist for safeguards and for arms control capabilities are outlined below.

Development of International Safeguards Capabilities

As noted in Section 1.5, the IAEA Department of Safeguards relies on external RDT&E to keep pace with emerging challenges and ensure continued safeguards effectiveness and efficiency.⁴¹ The process by which NNSA and DoS transition international safeguards capabilities to the IAEA is illustrated in Figure 2-3. An example of the process is provided in Box 2-3.

As shown in Figure 2-3, the IAEA communicates safeguards R&D needs to its member states through two documents, the IAEA R&D Plan (IAEA, 2018) and IAEA Development and Implementation Support Programme (DISP) for Nuclear Verification (IAEA, 2020a), as well as through informal communications. The IAEA DISP has a near-term focus and lists specific projects with brief descriptions to inform the inspectorate, member states, and other contributing organizations and stakeholders about the efforts of the IAEA Department of Safeguards in terms of development activities and needs for additional support for the implementation of safeguards. The DISP is developed with input from international partners and national laboratory researchers. NPAC/OINS uses this plan as a source document for safeguards technology development project planning.

___________________

⁴¹ The IAEA ensures that R&D needs are well planned, prioritized, and communicated to member states via official Support Program (SP) requests, referred to as SP-1s.

The IAEA R&D plan, updated periodically and last published in 2018, communicates long-term R&D needs without timelines, due to the “inherent uncertainty associated with long-term R&D planning,” (IAEA, 2018) and does not always point to or indicate what specific requirements must be met by specific R&D efforts. DNN R&D/PD uses this plan to guide early TRL safeguards R&D.

Another way that IAEA needs are informally communicated to DNN R&D/PD and NPAC/OINS is through U.S. technical experts who serve tours at the IAEA and gain valuable knowledge and insight into IAEA operational needs and constraints.⁴² This insight is informally communicated to the NNSA leadership and program managers during the tours and when the experts return to the United States.

As shown in Figure 2-3, early TRL work successfully completed in DNN R&D/PD must be transitioned to NPAC/OINS and then to the IAEA once matured. For this process to work ideally, NPAC/OINS must communicate future IAEA needs to DNN R&D/PD and ensure that innovative research conducted in DNN R&D/PD is communicated to pertinent IAEA staff. The committee assessed that this is currently happening to some extent but could be strengthened. It is especially important that NPAC/OINS and DNN R&D/PD communicate closely so that NPAC/OINS can represent DNN R&D/PD efforts to the IAEA.

Development of Arms Control MDV Capabilities

Another example of needed coordination between those developing and using technologies is the development of new arms control MDV technologies, where DoS/AVC (policy negotiation lead), NNSA (technology lead), and DTRA (inspectors) must work together. Figure 2-4 illustrates how arms control monitoring capabilities are developed and implemented currently and where there is room for improvement.

DoS is responsible for negotiating future treaties, with the input of the interagency consultative process. To put the United States in the best position to successfully achieve its policy goals, there needs to be a two-way street between DoS and NNSA, with DoS communicating what future treaties may look like and NNSA communicating the current status of monitoring technologies that would enable different types of limitations and/or verification processes and needs. There is evidence of this communication happening recently during the New START extension negotiations with Russia when the DoS negotiation team visited some of the DOE laboratories to better understand existing and new technologies (NNSA, 2020). Ideally, this communication should happen on a continuing basis so that the enterprise is prepared when negotiations begin. However, policy organizations are often consumed with more immediate priorities. In order

___________________

⁴² In addition to U.S. technical experts hired as IAEA staff, the United States also embeds cost-free experts (CFEs) within the IAEA to work on specific tasks. These CFEs function essentially as IAEA staff for their tour.

to help ensure focus on longer-term needs for R&D planning purposes, the external advisory board recommended in Section 2.1.2 could help ensure that DoS is considering the full range of potential monitoring and verification capabilities that could be used to support future treaties.

DTRA has some responsibilities for implementing U.S. treaty monitoring, particularly for bilateral treaties like New START. DTRA maintains the treaty data-exchange and notification databases, trains and maintains escorts and inspector teams, escorts Russian inspectors for U.S. inspections, and conducts on-site inspections in Russia, as it did for the previous START treaty. According to DTRA and NNSA, there is no formal process for DTRA arms control inspectors to communicate ideas for better, replacement, or additional equipment for either extended or new treaties. Instead, R&D needs are communicated informally based on the working relationship between DTRA and NNSA personnel on treaty implementation (DNN R&D, communication to committee, October 14, 2020; DTRA/OB, briefing to committee, October 29, 2020).

RDT&E for arms control monitoring is primarily funded by NNSA/DNN R&D and NPAC, with DNN R&D supporting lower TRL efforts and NPAC supporting higher TRL efforts. At the present time, there is not a clear hand-off partner for NPAC to complete technology maturation, assure certification and authentication, and prepare for inspection installation and/or use. DTRA/RD historically played this role but has shifted focus away from MDV R&D in recent years.

As the United States looks ahead to future arms control treaties that may limit all nuclear weapons rather than just deployed strategic nuclear delivery vehicles and associated weapons, the need to monitor nuclear warheads through

their entire lifecycle could arise for the first time. NNSA is the agency responsible for developing warhead monitoring capabilities that could be employed in such a scenario. The majority of this R&D is currently being undertaken by NPAC/ONV through its Warhead Verification Program (WVP), which recently released a Warhead Verification Capability Development Plan (NPAC, 2020). The WVP Plan, discussed in more detail in Section 3.4.2, outlines needs capabilities for Baseline, Additional, and Stretch MDV regimes. NPAC/ONV has focused on the baseline and additional approaches in recent years but has not started any significant work toward the stretch approach. Furthermore, while DNN R&D/PD has historically conducted early TRL research on warhead verification and other arms control technologies, it is currently less active in this area, as discussed further in Section 3.4.2.

Given the budget, it is understandable to take such a linear approach, but the danger of working in this fashion is that the technologies that are necessary to support the stretch approach may not be ready when the policy requires the capability. The more difficult “stretch” R&D may take many years to develop and could have higher added value despite the challenges. If this R&D is continually delayed, the capabilities will not be available in the future when a new treaty negotiation may demand them.

2.3.2 Basic Research and Innovation at the National Laboratories

At the same time that the enterprise is ensuring that all near-term MDV requirements are being addressed, there must be continual focus on long-term, foundational R&D at early TRLs. NNSA/DNN R&D noted “long-term R&D should not be bounded by current policy. Rather, R&D needs to prepare for potential shifts in the strategic security environment and should thus take a broader look beyond just what is considered currently feasible” (DNN R&D, communication to committee, September 18, 2020). This type of research (TRL 1–2) is primarily supported by DNN R&D and carried out at the national laboratories and at the universities that participate in the DNN R&D consortia.

As an early R&D organization, DNN R&D has fewer strict R&D requirements to deliver on than operationally oriented R&D organizations like NNSA/NPAC and DTRA (the exception being in delivery of space-based satellite sensors to the Air Force), and is better able to pursue innovative R&D projects with the potential to make revolutionary, rather than evolutionary, advances in MDV capabilities. Taking risks is necessary in this area, and DNN R&D noted that they expect 20 percent of projects to fail.⁴³ However, in FY2019, DNN R&D/PD noted

___________________

⁴³ Failure here is defined by DNN R&D as a project failing to achieve the specified TRL goal at the end of the project (DNN R&D, 2019a).

that 94 percent of early proliferation detection projects and 100 percent of nuclear weapons and material security projects reached their TRL goal (DNN R&D, 2019a, 2019b).⁴⁴ This high rate of success is to be commended, but may indicate that DNN R&D is not incentivizing enough high-risk, high-reward projects.

The national laboratories also fund some innovative MDV R&D with internal Laboratory Directed Research and Development (LDRD) funds.⁴⁵ Since these funds are not directed by DNN R&D and are not limited to MDV-relevant projects, the MDV enterprise should not be reliant on them. However, this has not been the case historically. A Nonproliferation Task Force established by the Secretary of Energy’s Advisory Board (SEAB) found in 2015 that “as a regular practice, the only source of funding for breakthrough innovations in the labs is coming not from projects initiated by the Department, but from laboratory directed R&D (LDRD)” (SEAB, 2015). To alleviate this reliance on LDRD for nonproliferation innovation, the SEAB task force recommended that DOE consider setting aside some nonproliferation R&D funds specifically to be awarded for high-risk, high-reward ideas.

In 2017, DNN R&D established such an innovation portfolio with this goal (DNN R&D, communication to committee, November 3, 2020). The innovation portfolio is managed by the DNN R&D/PD Deputy Office Director and is intended to support both DNN R&D/PD and the DNN R&D Office of Nuclear Detonation Detection. The innovation portfolio is funded at approximately $10 million annually and currently limits the number of proposals accepted from each laboratory (ibid.).⁴⁶ With this current structure, this portfolio feels somewhat like an afterthought rather than a prioritized effort. Increasing the funding of this portfolio and lifting the laboratory proposal limit could help ensure that the largest possible pool of innovative ideas is communicated to NNSA.

The MDV research enterprise is relatively small and made up of researchers with considerable experience in the field. While this expertise is indispensable, the enterprise could benefit from an injection of new ideas. One potential way to encourage innovative new ideas from researchers not yet associated with the

___________________

⁴⁴ It is important to note that a project reaching its TRL goal is not the same as the technology being transitioned to an end user. NNSA (DNN R&D in particular) has a much lower rate of technology transition, as noted by GAO (2017), for reasons discussed in Section 2.3.1.

⁴⁵ Congress has authorized the DOE national laboratories to devote a small portion (usually 6 percent) of their annual revenue, LDRD, to creative and innovative work that supports basic S&T disciplines relevant to national security missions. LDRD allows the flexibility to encourage multidisciplinary teams to engage in high-risk R&D and serves as a proving ground for R&D concepts that can then be funded by NNSA or other sponsors.

⁴⁶ This number is determined based on how much unallocated money is available in DNN R&D. In FY2019, the portfolio accepted up to five proposals per laboratory, while in FY2020, it accepted only up to two proposals per laboratory.

MDV mission is through prize challenges.⁴⁷ NNSA/DNN has begun to recognize the utility of prize challenges. A recent DNN R&D project established prize challenges to solve a challenging radiation detection problem. The challenge was run in two phases, the first open to government laboratory scientists and the second open to the general public (Topcoder, 2020). While most of the algorithms submitted in the government competition used traditional physics-based approaches, the algorithms submitted in the public competition almost all used data science techniques such as ML (DNN R&D, 2019a).

The DNN R&D/PD FY2019 annual report (DNN R&D, 2019b) noted the challenge of finding appropriate mission problems that can be shared with the community. Instead of focusing on finding appropriate unclassified mission problems, it may be more effective to develop surrogate problems that mimic sensitive, real problems. This concept is especially applicable for algorithm development challenges, since an algorithm developed to solve a surrogate problem can be later applied to the real problem. This approach is commonly used at the Intelligence Advanced Research Projects Activity (IARPA), which runs several open prize challenges (IARPA, 2020). NNSA could potentially benefit from learning prize challenge best practices from IARPA and other organizations with significant experience conducting this type of R&D.

2.3.3 Leveraging Academia

DNN R&D primarily leverages academia through their university consortia, which, in addition to their purpose described in Section 2.2.3 as a workforce pipeline, play a significant role in basic R&D. The DNN R&D consortia perform research in a number of innovative research areas, for example, in-situ biota sensors, antineutrino-based detection methods, advanced manufacturing for nonproliferation, and data fusion and other advanced analytic techniques. However, the committee has concerns that with the current consortia model, which is very focused on students, faculty expertise is not being fully leveraged for innovation.

Whether the consortia model is an effective method of leveraging academia is an open question that deserves additional attention beyond this study. There are several potential limitations of such a model. For one, the consortia model may limit innovation by concentrating funding at a smaller subset of universities that often have pre-existing collaborations. Universities that are not part of a consortium may have good ideas and contributions that are missed by limiting university engagement to the consortia model. In addition, it remains unclear how effective the consortia model is for engaging a broad range of research topics, including emerging areas of interest. Whether or not the consortia model is continued in DNN R&D, there are opportunities to better leverage academic contributions.

___________________

⁴⁷ For more information about the role that prize challenges can play in fostering innovation, see NASEM (2020).

In the development of consortia proposal calls, DNN R&D relies on the national laboratories for innovative ideas and not academia. For the mission-focused⁴⁸ consortia at least, DNN R&D dictates specific topic areas, developed with input from the national laboratories, for the consortia to focus on. DNN R&D could broaden its pool of innovative ideas by soliciting input from academic researchers as well, allowing them to bring new ideas to the table that might be outside the scope of what DNN R&D and the laboratories are focusing on. The DNN R&D consortia are funded for five years. This extended funding model provides the universities with the stability needed to work on long-term, low-TRL research. However, in fields like data science that are rapidly changing, important new research avenues may emerge after a consortium has been established and its focus areas set. Allowing flexibility for consortia to pivot in new directions and incorporate new research and technologies that may not have been on the horizon when the original funding call was written is essential in order to leverage the intellectual expertise of academia to the maximum advantage.

It is important to assess whether the consortia model is providing the scientific advances necessary to ensure successful execution of the NNSA’s nuclear security and nonproliferation mission, and whether there is enough risk in the research portfolio. The NNSA DNN FY2020–2024 Plan notes that DNN transitions basic R&D from university consortia to national laboratories (NNSA, 2019). However, DNN R&D informed the committee that they do not specifically track the transition of university projects. The committee is concerned that NNSA’s investment in R&D is not being fully leveraged, and without tracking the transfer of knowledge, it is impossible to fully assess the value of the program in terms of research output.

2.3.4 Leveraging Commercial Industry

Nowadays, businesses and commercial entities are responsible for more than 70 percent of the total U.S. RDT&E budget,⁴⁹ in stark contrast to the 1950s when the federal government was the primary funder of RDT&E across the country. The growth in commercial RDT&E has happened steadily over the past several decades and will very likely continue. With the vast amount of resources being put into RDT&E by the commercial sector, the MDV RDT&E enterprise should examine whether some of this investment could be leveraged for furthering the MDV mission.

___________________

⁴⁸ The currently funded mission-focused consortia are MTV and ETI. See Section 2.2.3 and Appendix K for additional details about these consortia.

⁴⁹ The federal government is responsible for roughly 20 percent of R&D, with philanthropy making up the difference (NSF, 2013, 2021b). This is average across TRL levels. Federal funding is the largest share of support for low-TRL research, and business accounts for nearly all of the highest TRL development, with a mix in the middle.

The committee found that NNSA/DNN R&D, the primary funder of low-TRL MDV R&D, has limited engagement with commercial industry. Industry engagement is limited to the congressionally mandated Small Business Innovation Research (SBIR) and Small Business Technology Transfer program, which accounts for less than 4 percent of total R&D spending by DNN R&D (DNN R&D, communication to committee, January 14, 2021). Much of the current interaction with industry focuses on hardware for sensors, for example, radiation detectors. The committee found that NNSA has fewer connections to commercial industry partners focused on data science, including the analytics of disparate data streams, where more early-TRL development is underway.⁵⁰ The development of algorithms that can sort through data and automatically determine patterns of life, anomalies, and change detection is critical to advancing MDV capabilities as the amount of data available to analysts in this space continues to grow. These algorithms are evolving at a rapid pace in the commercial world, but is unclear whether these advances are being absorbed by DNN R&D at the same rate as commercial entities are maturing them (see Section 3.5 for more technical details on DNN R&D’s data science efforts).

Other government agencies have made explicit efforts to rapidly identify and absorb highly effective industrial advancements. An example is the Air Force, which established AFWERX to foster a culture of innovation within the service. As part of an effort to accelerate technology development and transition, AFWERX targets making SBIR awards within 30–60 days of application (AFWERX, 2020). This quick contracting process is effective in rapidly acquiring tools and algorithms for DoD-specific mission sets in data science fields like AI and ML that are changing quickly. These new tools and algorithms then transition to the service through the small business contractor, sometimes in collaboration with larger DoD R&D organizations such as AFRL.

Some of NNSA’s reluctance to leverage the commercial sector may stem from a failure to recognize the value. The committee was told by multiple NNSA officials and laboratory scientists that the MDV mission is unique and unrelated to the challenges faced by industry. While it is true that few commercial companies have subject matter expertise to apply to the MDV mission, many are developing solutions to analogous challenges. For example, at the 2020 IAEA Emerging Technologies Workshop, a representative from the diamond industry discussed how his field was using distributed ledger technologies (DLT) and other emerging technologies to track diamonds from the time they are mined to when they are sold (IAEA, 2020b). DLT is of interest to the safeguards industry, which has an analogous need to trace nuclear material through its lifecycle. Furthermore, working with sparse data, a challenge faced by the MDV mission,

___________________

⁵⁰ Other organizations in the MDV enterprise have higher reliance on the commercial sector, particularly those that conduct higher TRL RDT&E. For example, 30 to 35 percent of DTRA’s R&D funds are sent to industry partners (DTRA/RD, briefing to committee, September 21, 2020).

is a commercial driver for industries for which obtaining experimental data is expensive, such as drug discovery (Intelligens, 2020). Box 2-4 gives an example of commercial technology development in data science areas that the MDV enterprise could leverage. While it is fair that large companies may not have the interest in tailoring algorithmic tools for the MDV mission set, smaller companies can and should be part of the MDV innovation ecosystem and could perform the tailoring of the data science tools.

Establishing more robust ties to industry and developing a more streamlined acquisition process may enable NNSA to fast-track new algorithms (e.g., for sparse datasets) into NNSA-relevant testing and potentially into deployment. An important initial step is to gain greater familiarity with developments in the commercial sector and determine best practices for leveraging relevant technology development, with an emphasis on data science. NNSA can learn from the

Central Intelligence Agency (CIA) and DoD in this regard, who have embraced this need with InQTel, DIU, and DreamPort, described in Box 2-5.

2.3.5 Findings and Recommendations

Finding 4. Challenges persist in transitioning low-TRL MDV R&D to operational systems and tools. R&D and operational organizations are limited in their ability to support prototype development and operational test and evaluation in facilities with access to real processes, data, and/or materials. Classification issues, facility access, conduct of operations and safety procedures, and lack of pertinent facilities and materials often make technology maturation complicated, slow, and expensive. These challenges exist for multiple MDV focus areas:

1. NNSA/NPAC Office of International Safeguards (OINS) works closely with the IAEA to address IAEA capability needs and mature technologies to the necessary level for IAEA implementation.
2. NNSA/NPAC Office of Nuclear Verification (ONV) plays a key role in the mid-TRL development of arms control technologies. However, there appears to be a lack of formalized communication and coordination between arms control operators (DoD) and technology providers (NNSA). This gap is partially a result of DTRA/RD pivoting away from MDV efforts.
3. Coordination between NNSA/DNN R&D and NNSA/NPAC to identify nonproliferation and arms control MDV technology priorities and transition low-TRL R&D to higher TRLs could be improved.
4. AFTAC faces challenges in transitioning R&D conducted by interagency partners to operational systems and tools for its nuclear explosion monitoring mission. Unlike for international safeguards and arms control MDV, an organization with the mandate, funding, and knowledge to mature MDV technologies for implementation by AFTAC is not evident.

Recommendation 4. MDV R&D organizations and operational end users should take steps to address the challenges in transitioning technologies.

1. Needs of operational users should be taken into consideration for projects, especially those at TRL 3 or higher. Operational users should maintain close communications and coordination with the technology providers throughout the technology development and transition process. Connecting operators and developers earlier in the technology development process will ensure that requirements are better communicated and allow for more agile and responsive development if requirements are still uncertain. As the TRL progresses, the operators should provide increasingly specific technical and operational requirements.

1. NNSA should broaden access to key facilities, processes, and materials via streamlined conduct of operations procedures, through the test beds or otherwise.
2. NNSA/DNN Deputy Administrator should institutionalize a process for close communication between DNN R&D and NPAC (both OINS and ONV) to facilitate selection of high-priority innovative ideas and transition of promising safeguards and arms control technologies.
3. To continue DoD’s historic and unique responsibilities in arms control and counterproliferation activities, it should appoint a relevant internal organization to help establish requirements for NNSA arms control technology development and testing activities, especially but not solely as they mature (TRL 3 and above). The organization selected should have real-world knowledge about nuclear weapons storage and deployment conditions in the United States and elsewhere and should be well-versed in the experiences and lessons-learned from the DTRA/On-Site Inspection and Building Capacity Directorate inspection teams.

Finding 5. MDV innovation emerges from work funded by DNN R&D but also through national laboratory LDRD projects, academia, and the private sector. Rather than consistently funding early-TRL projects in support of MDV priorities, DNN R&D is reliant on the laboratories to support and foster early work before committing resources for ongoing support. This approach risks gaps in availability of innovative solutions to high-priority MDV missions.

Recommendation 5. The MDV R&D enterprise should look for ways to sustainably drive the innovation pipeline for high-priority MDV objectives, while also maintaining channels to identify and build on basic research developed through national laboratory LDRD.

1. DNN R&D should consider how to allow greater participation in its innovation portfolio, including from the national laboratories, academia, and industry.
2. DNN R&D should ensure that its university consortia have agility to incorporate new research directions and technologies that may emerge after a consortium is established. DNN R&D should also track how consortia R&D investments are transferred to the national laboratories and industry for further development.
3. DNN R&D and other parts of the MDV R&D enterprise should use the best practices of other government agencies to optimize the use of prize challenges and solicit innovative ideas from researchers outside the traditional MDV mission space, including the use of surrogate datasets.

Finding 6. DNN R&D and the national laboratories have limited engagement with commercial industry, especially in the emerging technology areas of open-source and data sciences, where data collection and algorithm development are evolving at a rapid pace and have the potential to benefit the MDV mission space.

Recommendation 6. NNSA, in coordination with the national laboratories, should engage industry to fast-track new data science methods (e.g., algorithms for sparse datasets) into NNSA-relevant testing and potentially into deployment.

1. NNSA should learn how other government agencies have done this successfully (even for classified operations).
2. NNSA should invest in technology scouting to be familiar with developments in the commercial sector that could be applicable to the MDV mission.

REFERENCES

Finker, D. 2020. “The Role of innovation in IAEA safeguards.” Presentation at the Stanley Center for Peace and Security and Institute of Nuclear Materials Management virtual panel on emerging digital technologies for nonproliferation and nuclear security, September 9, 2020. https://www.youtube.com/watch?v=LG5mLupnnAc.

FIS. 2020. “FIS teams with FICO on advanced anti-money-laundering solution for U.S. financial institutions.” June 25. https://www.fisglobal.com/en/about-us/media-room/press-release/2020/fis-teams-with-fico-on-advanced-anti-money-laundering-solution-for-us-financial-institutions.

GAO (Government Accountability Office). 2017. Nuclear nonproliferation: Better information needed on results of national nuclear security administration’s research and technology development projects. GAO-17-210. Washington, DC.

Gardner, D. 2020. “DTRA to award $51.5 million to university research alliances.” Defense Visual Information Distribution Service, October 19. Fort Meade, MD. https://www.dvidshub.net/news/370351/dtra-award-515-million-university-research-alliances.

IAEA (International Atomic Energy Agency). 2018. Research and development plan: Enhancing capabilities for nuclear verification. STR-385. Vienna, Austria. https://www.bnl.gov/ISPO/docs/STR-385-IAEA-Department-of-Safeguards-RD-Plan.pdf.

IAEA. 2020a. Development and implementation support programme for nuclear verification 2020–2021. STR-383. Vienna, Austria. https://www.iaea.org/sites/default/files/20/01/d-and-s-programme-2020.pdf.

IAEA. 2020b. Emerging technologies workshop. Vienna, Austria. https://www.iaea.org/topics/safeguards-implementation/emerging-technologies-workshop.

IARPA (Intelligence Advanced Research Projects Activity). 2020. Prize challenges. Washington, DC: Office of the Director of National Intelligence. https://www.iarpa.gov/index.php/working-with-iarpa/prize-challenges.

Intelligens. 2020. “Novel deep neural network imputation technique predicts bioactivity from incomplete data.” White paper. Cambridge, UK. https://intellegens.ai/drug-discovery-novel-deep-neural-network-imputation-technique-is-able-to-predict-bioactivity-from-incomplete-bioactivity-data/.

IPFM (International Panel on Fissile Materials). 2020. Countries: United States. Princeton, NJ. http://fissilematerials.org/countries/united_states.html.

Nakasone, P. M., and M. Sulmeyer. 2020. “How to compete in cyberspace.” Foreign Affairs, August 25. https://www.foreignaffairs.com/articles/united-states/2020-08-25/cybersecurity.

NASEM (National Academies of Sciences, Engineering, and Medicine). 2012. Assuring a future U.S.basednuclearandradiochemistryexpertise. Washington, DC: The NationalAcademies Press. https://www.nap.edu/catalog/13308/assuring-a-future-us-based-nuclear-and-radiochemistry-expertise.

NASEM. 2020. The role of inducement prizes: Proceedings of a workshop in brief. Washington, DC: The National Academies Press. https://doi.org/10.17226/25892.

NNSA (National Nuclear Security Administration). 2019. Prevent, counter, and respond: NNSA’s plan to reduce global nuclear threats; FY 2020-2024. Washington, DC.

NNSA. 2020. “NNSA hosts special presidential envoy for arms control.” September 14. Washington, DC: https://www.energy.gov/nnsa/articles/nnsa-hosts-special-presidential-envoy-arms-control.

NPAC (Office of Nonproliferation and Arms Control). 2020. Warhead verification capability development plan. Washington, DC: National Nuclear Security Administration.

NSF (National Science Foundation). 2013. National patterns of R&D resources. Arlington, VA. National Center for Science and Engineering Statistics. https://nsf.gov/statistics/natlpatterns.

NSF. 2021a. Engineering education and centers. Arlington, VA. https://www.nsf.gov/div/index.jsp?div=EEC.

NSF. 2021b. MRSEC program overview. Arlington, VA. https://mrsec.org/.

NSF. 2021c. Welcome to the ERC program. Arlington, VA. https://erc-assoc.org/.

NSTC (National Science and Technology Council). 2019. Nuclear defense research and development strategic plan for fiscal years 2020-2024. Washington, DC: Committee on Homeland and National Security, Subcommittee on Nuclear Defense Research and Development. https://www.whitehouse.gov/wp-content/uploads/2019/12/Nuclear-defense-research-development-strategic-plan-2019.pdf.

Russell, M. 2018. “University students flock to data science as interest and demand surge.” Center for Digital Education, October 19. https://www.govtech.com/education/higher-ed/University-Students-Flock-to-Data-Science-as-Interest-and-Demand-Surge.html.

SEAB (Secretary of Energy’s Advisory Board). 2015. Report of the Task Force on Nuclear Nonproliferation. https://www.energy.gov/sites/prod/files/2015/04/f21/2015-03-31_FINAL_Report_SEABNuclearNonproliferationTaskForce_0.pdf.

Tenet, G. 1997. At the center of the storm: My years at the CIA. New York City: Harper Press.

Topcoder. 2020. Detecting radiological threats in urban areas. https://www.topcoder.com/challenges/d616d145-d7e7-4d62-9323-9c558b216b40.

Treasury (Department of the Treasury). 2015. National money laundering risk assessment. Washington, DC. https://www.treasury.gov/resource-center/terrorist-illicit-finance/Documents/National%20Money%20Laundering%20Risk%20Assessment%20%E2%80%93%2006-12-2015.pdf.

Treasury. 2020. History of anti-money-laundering laws. Washington, DC. https://www.fincen.gov/history-anti-money-laundering-laws.

Wilkers, R. 2020. “In-Q-Tel enters new quantum, synthetic data investments.” Washington Technology, April 14. https://washingtontechnology.com/articles/2020/04/14/inqtel-quantum-data.aspx.

Yannuzzi, R. E. 2007. In-Q-Tel: A new partnership between the CIA and the private sector. https://www.cia.gov/library/publications/intelligence-history/in-q-tel.