National Academies Press: OpenBook

Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief (2022)

Chapter:Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
images Proceedings of a Workshop—in Brief

Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape

Proceedings of a Workshop—in Brief


On June 9, 2021, the Intelligence Community Studies Board (ICSB) of the National Academies of Sciences, Engineering, and Medicine convened a virtual workshop on behalf of the Office of the Director of National Intelligence. This unclassified workshop was the second of two data-gathering workshops. Panel 1 on North Atlantic Treaty Organization (NATO)/Department of Defense (D0D) Perspectives featured panelists Bryan Wells, NATO; Captain James Borghardt, USN, ONR Global; Jason Matheny, Office of Science and Technology Policy (OSTP); and Steward Remaly, DoD/ASD/SOLIC/Irregular Warfare Technical Support Directorate. Panel 2 on U.S. Government Perspectives featured Kevin Wolf, Akin Gump LLP; Dana Johnson, OUSD(R&E); and Dewey Murdick, Georgetown University. Panel 3 on Emerging Technologies and National Security featured Richard Johnson, Global Helix, and Melissa Flagg, Georgetown University. This workshop will help to inform a consensus study report investigating how the Intelligence Community (IC) could leverage the evolving research and development (R&D) ecosystem to meet its future needs and garner the most value from its investments in science and technology (S&T). This, the second of two workshops, focused on the IC’s connections with foreign R&D and approaches for protecting classified IC information while remaining abreast of cutting-edge advances (often made by foreign researchers) in critical S&T areas. The consensus study committee vice chair Michael Marletta, C.H. and Annie Li Chair in the Molecular Biology of Diseases, University of California, Berkeley, invited panelists and participants to discuss the international perspectives of how the IC could develop strategies to take advantage of this R&D landscape.

NATO AND DOD PERSPECTIVES

Bryan Wells, NATO’s chief scientist, described its emphasis on emerging and disruptive technologies (EDTs), such as autonomy, artificial intelligence (AI), data, space, hypersonics, biotechnology and human enhancement, quantum technologies, and novel materials. In March 2020, NATO’s Science and Technology Organization (STO) published Science & Technology Trends 2020–2040: Exploring the S&T Edge.1 He underscored the value of this public release document as a way to increase dialogue among a broad range of non-NATO organizations. It provides leadership with an overview of EDTs; and it explores why EDTs are important, how they could be developed in the future, and implications for the alliance. It also highlights future opportunities for NATO, and, to the extent possible in a public

__________________

1 NATO STO, 2020, Science & Technology Trends 2020–2040: Exploring the S&T Edge, http://www.nato.int/nato_static_fl2014/assets/pdf/2020/4/pdf/190422-ST_Tech_Trends_Report_2020-2040.pdf. Other relevant NATO publications include 2020 Highlights of the Science and Technology Organization: Empowering the Alliance’s Technological Edge; NATO STO-CMRE: 2020 Annual Report; and 2021 Collaborative Programme of Work: NATO’s Science and Technology Organization.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×

document, identifies potential threats. NATO defines the best new technologies as intelligent, digital, interconnected, and distributed, and it provides professional scientific judgments on the implications of EDTs. Wells remarked that it is more likely for a combination of technologies to create a disruptive military effect than an individual technology. The most potentially impactful combinations are AI, data, and autonomy; AI, data, and biotechnology; AI, data, and materials; data and quantum; space and quantum; and space, hypersonics, and materials. Technologies are also expected to come to military maturity on different time scales—for example, data, AI, autonomy, space, and hypersonics are expected to be disruptive within 5–10 years, while quantum, biotechnology, and novel materials are expected to be disruptive in 10–20 years. He explained that EDTs are inexpensive and easy to use, dominated by the civil sector, and spread over a large range of technical areas. This creates challenges for NATO in that the civil sector now dominates technology development, and Western societies have to find ways to surpass their adversaries. Thus, he advocated for NATO and other defense organizations to reach out to the civil sector to better understand new technologies. He expressed optimism about maintaining a technological edge, noting that 18 of the top 20 universities in the world are located in ally nations, and he highlighted the value of NATO’s Technology Watch in prioritizing technology development.

Wells briefly discussed NATO 2030,2 an initiative launched in June 2020 with an independent group of experts, emerging leaders, and representatives from civil society and the private sector. When the initiative launched, the NATO Secretary General defined its intention, within the decadal time frame, to develop further collaborations with like-minded countries (e.g., Australia, Japan, New Zealand, and South Korea) and to ensure that work on new technologies reflects the norms and standards of allied nations. Wells noted that all of NATO’s innovation work is connected via the Innovation Board, which oversees EDT strategy implementation—with a focus on technology, policy, legal, and ethical issues—and is chaired by the Deputy Secretary General. Members include the Chairman of the Military Committee; the Supreme Allied Commander, Transformation; the Supreme Allied Commander, Europe; the Director General of the International Military Staff; the Assistant Secretary General on Emerging Security Challenges; and the Chief Scientist. An independent advisory group of academics, subject-matter experts, and chief executives is also included on the Innovation Board. This board looks at broader innovation priorities for NATO. It looks at innovation from many points of view and to develop strategies of implementation for emerging and disruptive technologies. The board deals with more than just technical issues; its remit also encompasses policies and legal and ethical issues resulting from this new set of technologies.

Serving as session moderator, Alan Shaw, director of ICSB, National Academies, asked how NATO characterizes success in its engagement with other countries. Wells replied that four non-ally nations (Sweden, Finland, Australia, and Japan) are engaged closely with CPoW, which maintains a low level of classification. This makes it easier for individual nations to invite non-NATO partners into projects without security barriers and to create an environment of mutual benefit. Robert Hale, adjunct senior fellow, Center for a New American Security, inquired about Technology Watch as it pertains to potential adversaries, and Wells commented that Technology Watch Cards describe the extent to which specific technologies are expected to evolve in the coming years. Given the low level of classification, these focus more on the opportunities for rather than on the threats to NATO allies. However, classified work is being conducted on counter-EDTs and other threats. Bernard Meyerson, chief innovation officer emeritus, IBM Corporation, wondered if NATO-funded schools still exist, and Wells responded that NATO offers a range of educational opportunities and facilities. For example, the NATO Defense College in Italy includes science, political, and military programs. NATO also offers specialist courses at the von Karman Institute for Fluid Dynamics in Belgium, and other members of the network conduct research seminars at universities. In response to a question from Marletta about why biotechnology is not yet mature within the military, Wells said that the biotechnology that has been developed in the laboratory or in the civilian sphere needs to be “ruggedized” to be effective on the battlefield. He believes that the transition to a proven military capability will happen, but it will take time.

Capt. James Borghardt, Commanding Officer, Office of Naval Research (ONR) Global, explained that ONR Global focuses on

__________________

2 See the NATO 2030 website at https://www.nato.int/nato2030/.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×

fundamental research and transitioning it to ONR, the Naval Research Laboratory, and the broader Naval R&D enterprise. He noted that all of the research that ONR Global funds is open and published, although some eventually shifts to classified projects. Sharing several statistics about the state of S&T, he said that according to the 2017 Global R&D Magazine Funding Forecast, 80 percent of the world’s researchers are outside of the United States. The Congressional Research Service’s Global R&D Expenditures 2020 Fact Sheet reported that the U.S. share of global R&D funding decreased from 69 percent in 1960 to 28 percent in 2018. According to the Times Higher Education World University Ranking 2021, 25 of the top 50 universities and 63 of the top 100 universities are outside of the United States. And of the $2.1 trillion in global research funding, DoD basic research funding accounts for 0.1 percent. Thus, he asserted that the United States (and DoD in particular) is no longer driving global basic research.

One of the primary efforts within ONR Global is to better understand state-of-the-art science in both neutral and adversarial countries. Instead of investing in specific capabilities, Capt. Borghardt continued, ONR Global invests in trusted partnerships—the 0.3 percent of the DoD basic research funding that ONR Global invests is used to connect leading international researchers. The ability to create a collaborative network of research networks to accelerate discovery overrides the funding imbalance. He noted that these trusted partnerships can be difficult to establish, yet they are successful owing to ONR Global’s “enduring presence.” ONR Global increases trust among its partners by spending time in international laboratories, with students, and at conferences of global scientists. ONR Global’s 50 scientists and engineers work in 2 offices and 25 deployment sites to transition the network of connections into a network of trusted partnerships. These relationships continue when scientists return from their deployments to the Naval R&D laboratories, federally funded research and development centers (FFRDCs), and university affiliated research centers (UARCs). In closing, he referred workshop participants to a publicly available annual prospectus3 that details how ONR Global has allocated its time, resources, funding, and staff during the past year.

Shaw asked how the IC could gain trust with scientists around the world. Capt. Borghardt reiterated the value of establishing an enduring presence. For example, in an effort to preserve trust, ONR Global decided at the start of the pandemic that as long as individual scientists were comfortable, they would not be removed from their locations. He advised the IC that building a network of trusted partners takes time but added that the IC has ample opportunity to develop such a network. Tomas Diaz de la Rubia, vice president for research and partnerships, University of Oklahoma, questioned whether there is a forum by which ONR Global shares its findings about global S&T with the IC. Capt. Borghardt confirmed that, as one of the five active-duty military officers in ONR Global, part of his role is to ensure that topics of interest to the IC are shared via the proper channels. Peter Schiffer, vice chancellor for research and professor of physics, University of Illinois at Urbana-Champaign, wondered how ONR Global tracks the full breadth of research and maps it onto what might be relevant to ONR, the military, and the IC. Capt. Borghardt acknowledged that this is a difficult endeavor. He has been advocating for a tool to increase information sharing across networks as well as an AI-enabled knowledge management tool to improve collection and dissemination. With approximately eight scientists per region who have domain knowledge and a specific area of interest knowledge, it is possible to better collate information. However, the fact that this knowledge is managed manually is not optimal. Schiffer asked about the likelihood of deploying an AI-enabled tool that would be effective throughout the diverse global research enterprise. Capt. Borghardt responded that the U.S. Army, Navy, and Air Force support this type of tool because it would enable synergy without needing to be in the same country. He expressed cautious optimism about deploying such a tool in the future: people understand the need and the value, but it takes a long time to make changes. He emphasized that the technology is readily available in the civil sector; it is a matter of securing resources and transitioning the tool for defense use.

Jason Matheny, Deputy Director for National Security, OSTP, commented that the IC is not currently well organized (in terms of resources, tools, and expertise) to evaluate developments and trends in globally emerging commercial and dual-use technologies. This is in part because using tools for open source

__________________

3 ONR Global, 2020, ONR Global International Science Prospectus for FY20, https://www.onr.navy.mil/en/Science-Technology/ONR-Global/About-ONR-Global.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×

intelligence can be unwieldy on the IC’s classified systems. At Georgetown University’s Center for Security and Emerging Technology (CSET), it is possible to do such analysis at scale in an open setting with access to the university’s modern data science tools and expertise. He advocated for the scaling up of CSET to cover more technical areas, perhaps by leveraging the network of UARCs, where students and faculty are already engaged in relevant research and competitive analysis of the S&T landscape. He championed drawing not only on the competitive intelligence of academia but also on the competitive intelligence of the commercial sector. The challenge in both cases is that it can be difficult for the IC to reach out directly to those universities and companies without a contract in place (i.e., multiple layers of approval are needed). Matheny suggested creating a clearinghouse that is separate from but accessible to the federal government to facilitate interactions with these academic and commercial communities, which could lead to more timely, cost-effective, and accurate intelligence on global S&T.

Matheny expressed his hope that the more than $100 billion expected from the Creating Helpful Incentives to Produce Semiconductors for America Act and the Endless Frontier Act would create more opportunities for S&T funding. However, he said that new mechanisms for spending those funds would be useful. He remarked that although both grants and contracts have important roles within government S&T funding, grants, for instance, do not clearly specify desired research results and make it difficult to measure progress. He advocated for conducting experiments in how S&T is funded as a way to develop more cost-effective, higher performing mechanisms (e.g., prize challenges, regulatory review vouchers, advanced market commitments, advanced purchase commitments). He noted that it is also important to explore how the government makes decisions about who receives these awards. Most rely on a group judgment of experts, but such group judgments are prone to error. Using the research on forecasting could help better anticipate which proposals are likely to yield the most valuable scientific research. Another approach, he continued, would be to use randomized awards. He suggested requesting proposals on innovative decision making for funding science and creating a controlled experiment, in which all proposals are funded, although via different mechanisms. Five years later, a determination could be made about which research projects were most successful. Matheny underscored the value of conducting more science on the funding of science.

Schiffer wondered if the funding agencies are willing to consider alternative funding mechanisms. Matheny credited the Defense Advanced Research Projects Agency (DARPA) for its creative approaches both in soliciting ideas from the research community and to funding them. He added that the National Science Foundation is interested in experimenting with new approaches, and he is hopeful that people will have top-cover from OSTP and the Office of Management and Budget to experiment with the tools of science funding. Shaw asked Wells if NATO has discussed alternative approaches to funding innovation. Wells noted that while non-traditional funding mechanisms have been considered, NATO has not identified any that would be appropriate. He said that funding mechanisms that work at a national level may not work at an international level, owing to different intellectual property rules and competition rules and thresholds throughout the international community. And when there are 30 allies and decisions are taken by consensus, the desire to take on risk decreases. Marletta observed a general reluctance to change the scientific peer-review process, despite the fact that the approach is not always effective. He supported Matheny’s idea to have a controlled experiment on improved proposal assessment and expressed his hope that the academic community would embrace it. Matheny added that opportunities are missed when almost all of the science organizations in the federal government make funding decisions using nearly identical processes.

Meyerson observed that private-sector companies use a variety of strategies to remain competitive, and he wondered if studies have been conducted on the longevity of the 20 most successful companies. Following that line of thought, Matheny questioned whether there have been studies on technologies that accelerate the overall rate of innovation within a society as well as whether there is enough investment in such technologies, as the government tends to fund individual research projects instead of clusters of research projects that could develop a tool to elevate an industry. Matheny highlighted the importance of better understanding pre-competitive industry consortia that could enable the acceleration of such tools. Meyerson mentioned the challenge of convincing the IC that leveling the playing field (via open source work) would be better than being left behind by its competitors.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×

Steward Remaly, Program Manager for Surveillance, Collection, and Operations Support, Irregular Warfare Technical Support Directorate (IWTSD), described IWTSD as an organization of “experienced professionals” under the umbrella of the Assistant Secretary of Defense, Special Operations/Low Intensity Conflict. Its mission is to identify and develop capabilities for DoD to conduct irregular warfare against all adversaries, including Great Power competitors and non-state actors, and to deliver those capabilities to DoD components, the interagency, and foreign partners through rapid R&D, advanced studies, and technical innovation. He emphasized that IWTSD’s work spreads to IC partners. Its objectives for the upcoming year are to support the National Defense Strategy and the Annex for Irregular Warfare; provide forums to solicit and collaborate on R&D requirements; rapidly advance technology development, deliver prototypes for operational tests and evaluations, and assist in product transition; promulgate technology and information exchange; and influence policy and identify enablers. Remaly asserted that new processes and increased speed of innovation make it possible to remain relevant in the R&D business. IWTSD increasingly relies on AI, big data analytics, and machine learning as it works to strengthen and increase partnerships to solve difficult problems. IWTSD aims to enhance lethal capabilities and to improve survivability, for example, within 1–2 years of receiving a requirement instead of within 5–30 years. IWTSD is leading more than 269 research, development, test, and evaluation projects, including 128 International Task Plans—maintaining memoranda of understanding (MOUs) with Australia, Canada, Israel, Singapore, and the United Kingdom. He explained that these strategic partnerships drive innovation in that they allow 50 percent cost sharing and 50 percent capability sharing. While some of the projects within the International Task Plans enable information sharing, others turn into prototypes that are fielded and become programs of record.

IWTSD has a user-focused business process, Remaly continued. No action is ever taken without a validated requirement, which comes from end users, sponsors, and subject-matter experts in an effort to drive innovation. IWTSD then turns a requirement into a broad agency announcement (BAA), which is distributed to industry partners. After a series of white papers and proposal reviews, contract negotiations begin several months later. For example, IWTSD distributed 45 requirements in February 2021 to industry via the fiscal year (FY) 2022 BAA and hopes to release funding in October 2021 (IWTSD operates on a $70–100 million annual contract). He detailed common IWTSD challenges for FY 2022: vehicle telematics data retrieval; multi-role, field-configurable, offensive small unmanned aerial system; unmanned aerial vehicle payload for rapid chemical plume detection, identification, and mapping; casualty tracking and monitoring system; and advanced cyber physical test bed. Specific opportunities for collaboration include Challenge Programs (e.g., U.S./Australia Counter Improvised Threat Grand Challenge; U.S./Israel Mobile Standoff Autonomous Indoor Capabilities Challenge; U.S./U.K. Chemical Munitions Destruction Industry Challenge) and Planned Information-Sharing Workshops (e.g., U.S./U.K. Exploiting Emerging Commercial Space, National Aeronautics and Space Administration Robotics Workshop, U.S./Israel Less Than Lethal Capabilities, U.S./U.K. Maritime Security Data Exchange).

Gerald Epstein, distinguished research fellow, Center for the Study of Weapons of Mass Destruction, National Defense University, posed a question about the role of U.S. export controls. Remaly replied that while MOUs allow IWTSD to share capabilities across defense industries, if future development or production will require International Traffic in Arms Regulations (ITAR) approval, IWTSD has to work through all of the agreements that the vendor has to work through. Dewey Murdick, interim director, CSET, Georgetown University, asked how efforts are prioritized within IWTSD. Remaly explained that during the requirements generation, collaborative discussions with end users, other R&D organizations, and IWTSD’s team of experienced professionals occur, which include consideration for what others are doing, whether an idea makes sense, if something similar did not work in the past, and potential long-term ramifications. However, Remaly stated that prioritization is ultimately determined by user need; if the need is critical, it will be addressed.

PERSPECTIVES FROM INSIDE THE U.S. GOVERNMENT

Kevin Wolf, partner, Akin Gump LLP, explained that after World War II, military- or intelligence-related material required a license, no matter the level of sensitivity or relationship with the recipient. He said that ITAR hindered national security, owing to the information-sharing limitations it placed on allies. In August 2009, President Obama directed the agencies involved in the U.S. export control system to conduct a review of export controls to

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×

identify strategies to enhance U.S. national security, and, in 2010, former Secretary of Defense Robert Gates’s policy objectives were as follows: reform the export control system to increase interoperability with NATO and other close allies, reduce the current incentives for companies in non-embargoed countries to design out or avoid U.S.-origin content, and allow the administration to focus its resources on the transactions of greater concern. Wolf added that spending so much time and money on reviewing and clearing trade with NATO allies reduced the time available to focus on adversaries. U.S. agencies then began reviewing the U.S. Munitions List (USML) to determine what items no longer warranted control. To implement former Secretary Gates’s objectives, Wolf continued, “the administration needed to identify the specific sensitive and other items on a more positive USML that warrant individual license reviews even for ultimate end use by NATO and other regime allies; and amend the Export Administration Regulations and the Commerce Control List to control all formerly USML items that would no longer be on the revised USML so that they still could be adequately controlled but in a more flexible way regarding such allies.” The embargo on China remained, as did significant limitations for Russia and complete embargoes on Iran, North Korea, and Cuba. The Strategic Trade Authorization, a liberal license exception, allowed for the sharing of commodities, technology, and software among nationals and governments of the allied countries. Wolf noted that many of the descriptions in the regulations pertaining to things of concern to the IC were deliberately vague; his team worked with the agencies to describe these in more detail, but barriers remain.

Serving as session moderator, Epstein pointed out that many of the technologies of interest to the IC will not be developed by IC partners; collaboration is one of the best ways to understand global developments, and two-way interaction is necessary to gain the most benefit from partnerships. He observed that export controls intersect with the issues of the IC, and he asked Wolf to address strategies to deal with high-tech companies overseas. Wolf said that, motivated by changes in Chinese technology acquisition efforts, civil–military fusion policies, and human rights abuses, an important issue now concerns the AI, robotics, nanotechnology, and quantum computing technologies that are not on the export control list but warrant control. He added that the international export control system is not built to address the kinds of threats and novel technology acquisition efforts of the commercial space. In 2018, Congress passed more expansive authority for the regulating agencies. The Department of Commerce followed up with an interagency effort to identify emerging technologies that are not controlled but are essential to national security, and the Biden administration is defining a new concept of national security. The challenge is that U.S. allies’ systems embrace the previous export control mindset of only regulating weapons systems–related items. Wolf commented that a significant amount of rethinking is required on what warrants control to prevent technology of concern from going to China or Russia, without limiting it among the allies.

Dana Johnson, Director, International Outreach and Policy (IO&P), Office of the Under Secretary of Defense (OUSD)/Research and Engineering (R&E), explained that IO&P emerged in 2018 as the focal point for all international S&T4 engagement activities within OUSD(R&E). IO&P advises USD(R&E) and supports OUSD(R&E) on defense strategy, policy, and engagement for international matters. IO&P works closely with DARPA, the Defense Innovation Unit, the Missile Defense Agency, the Space Development Agency, the Defense Technology Information Center, and the Test Resource Management Center; provides subject-matter expertise; and assists OUSD(R&E) in pursuing international S&T cooperative activities aligned with the DoD Modernization Priorities.5

In December 2020, DoD International Science and Technology Engagement Strategy: A Unified Approach to Strengthen Alliances and Attract New Partners (ISTES)6 was released, providing a rationale for engaging with other countries. Johnson remarked that the mission of ISTES is to leverage foreign defense S&T capabilities, develop relationships with other countries to access these capabilities, maximize coalition interoperability, and achieve U.S. national security objectives. The document provides strategic guidance and coordinates individual DoD Component

__________________

4 Johnson defined S&T as basic and applied research through experimentation and prototyping.

5 The DoD Modernization Priorities focus on AI/machine learning; fully networked command, control, and communications; quantum science; autonomy; space; biotechnology; microelectronics; directed energy; cyber; hypersonics; and 5G.

6 DoD, 2020, Department of Defense International Science and Technology Engagement Strategy: A Unified Approach to Strengthen Alliances and Attract New Partners, https://www.cto.mil/wp-content/uploads/2020/12/Signed-International-ST-Engagement-Strategy.pdf.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×

engagement activities toward common objectives; directs international outreach efforts toward opportunities with the highest return on investment; seeks new opportunities with friendly nations that are pursuing niche S&T capabilities of interest to the United States; and provides a structured approach for creating enhanced awareness, coordination, and strategic planning of defense international S&T engagements. ISTES does not direct which nations to partner with or what S&T to pursue because S&T priorities may change over time, and alliances and regional interests may affect those choices. The vision of ISTES, she continued, is to have consistently deliberate engagement with allies and partners; share awareness of global sources of technology, which enables identification of gaps or duplication of efforts as well as any priorities subject to technology protection or controls; create well-established international relationships and effective mechanisms for cooperation (e.g., using existing agreements to share information); have visible senior leaders to engage with foreign counterparts and steer cooperation; create thriving international networks of researchers with collaborations in priority S&T areas producing high-volume outcomes; and maintain continuous improvement to business processes. She explained that ISTES principles

  • align with U.S. interests, including national security objectives, specific S&T needs, and international policy;
  • balance strengthening existing alliances with forging new partnerships to secure U.S. access to world-class S&T and achieve desired posture;
  • prioritize S&T investment and resources according to U.S. S&T needs, foreign S&T strengths and opportunities, and U.S. policy;
  • protect security of critical U.S. technologies;
  • are justified with benefits that exceed those that could be achieved independently;
  • are equitable for all parties;
  • strive for measurable outcomes that accelerate the pace of U.S. R&D and ultimately benefit defense missions;
  • are underpinned by suitable agreements and arrangements offering flexibility; and
  • enable timely and effective interactions that accommodate foreign governments’ requirements.

In response to a question from Epstein about specific partnerships with other countries, Johnson highlighted The Technical Cooperation Program (TTCP) with the Five Eyes partners, which has several ongoing activities and has been active for 60 years. Another example is a partnership with STO, for which the United States has three principal members, one of whom is a voting member from the Services. IO&P has also used the STO databases to track U.S. participation in global technology collaborations and determine where to increase U.S. engagement. Epstein asked how gaps are identified and R&D investment targeted toward those gaps. Johnson replied that the first step is talking to counterparts in the embassies and overseas. IO&P relies on open source and government documents, and an OUSD(R&E) strategic analysis cell links to the IC, helping to determine whether a potential partner is the right fit based on mutual priorities. She expressed her hope for increased IC engagement in the future. Schiffer commented that the IC is less centralized in its organizational structure than DoD. He questioned how to navigate bureaucratic challenges and make connections with the IC. Johnson said that it could be challenging but emphasized the value of leveraging that relationship. She proposed the creation of an international community of interest and welcomed suggestions about strategies to gain better insight for decision making.

Murdick explained that CSET helps policy makers understand the security implications of emerging technologies by connecting them to high-quality analysis. Two efforts launched by the Intelligence Advanced Research Projects Activity (IARPA)—Foresight and Understanding from Scientific Exposition (FUSE) in 2011 and Forecasting S&T (ForeST) in 2014—help analysts target their attention within an expansive space and detect and forecast new technical capabilities. He emphasized that the IC plays only a minor role in R&D; however, it has an important role in monitoring threats and opportunities that could be relevant to national security.

Murdick commented that FUSE was created to achieve validated, early detection of technical emergence. It aimed to reduce technical surprise by developing reliable forecasts and indicators from English and Chinese scientific and patent literature. He stressed that complex indicators were helpful for context but were not useful in terms of predicting what would disrupt technology; the simplest indicators were the most predictive. He

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×

pointed out that ForeST picked up where FUSE left off, taking scientific- and patent-type indicators and asking the crowd for other future S&T milestones. FUSE and ForeST are now being used by CSET to explore the following questions: What fundamental innovations are coming fastest, will have the most impact (e.g., advancements, capability disruption, or performance changes in mission critical systems), and need the most attention? How can those be prioritized to respond in time? Which basic science AI-related research areas will become emerging (security-related) technologies? Who should we listen to and from which interest community? In what areas do the United States and its allies have an advantage over China? He emphasized that these are very difficult questions to answer, but this forecasting technology makes it possible to determine which technologies are likely in the next few years to grow quickly. Understanding where the attention should be focused and connecting the data from these methods to top-down strategic context supports decision making—without context, the data are not useful.

Murdick highlighted that FUSE, in its new format at CSET, takes approximately 240 million research articles and groups them into problem- and citation-based clusters. Approximately 126,000 research problem–related clusters represent defined technical areas of research and have at least 50 papers each. Next, extreme growth research cluster forecasts are done at scale (selecting the top 3–4 percent most quickly growing clusters, estimated to grow at least 8 percent per year for the next 3 years) to begin to understand and prioritize which are growing most quickly and are connected to military interests. Multiple filters and linkages can then be used (e.g., for military interest, sectoral/corporate interest, news, social media). He described this approach as evidence-driven down-selection, and he emphasized the value of this tool for the IC—which would never have enough resources or expertise to track and understand the research—to contextualize and determine what could hinder competitive advantage. An important lesson learned, he continued, is that forecasting is useful for discriminating and helping eliminate unhelpful models; simple models perform better forecasting than complex models. Describing ForeST (now referred to as Foretell), he noted that IARPA’s Aggregative Contingent Estimation program established the accuracy of crowd forecasting on well-defined questions. Challenges arose for “big questions” relevant to policy makers. Foretell’s solution was to combine expert and crowd judgment: use expert judgment to break policy issues down into forecastable questions, elicit crowd forecasts on the questions about which experts disagree or are uncertain, and aggregate the results across questions to provide actionable insight. In closing, Murdick shared several insights: human judgments about technical emergence do not make for effective ground truth, owing to poor temporal resolution and linearization of memories; subject-matter expert “favorites” persist; and small-group subject-matter expert forecasting often has a low level of accuracy. He explained that technology transfer to the IC is particularly challenging because the IC is not well organized for using open data sources, and reorganizations change S&T analysis resourcing.

Schiffer asked if Murdick’s team has conducted retrospective studies on technologies of relevance to the IC to understand whether the emergence of these technologies could have been predicted if forecasting technology had been applied. Murdick said that approximately 30 case studies were performed on relevant emerging technologies over the past 20–30 years. While it was possible to validate methods, it was difficult to find methods that were discriminative. He added that most retrospective studies are not helpful because they provide a false sense of achievement; the reality of the future is what matters most. Murdick noted that there is more work to be done in making successful predictions: approximately 87 percent of what was forecasted in 2014 materialized in 2017, and of the things that achieved that threshold, approximately 48 percent had been successfully predicted. Epstein observed that the data points Murdick described were only from publications and patents, and inquired about gathering and analyzing unpublished developments from companies. Murdick commented that the bulk of basic research emerges in publications and patents, but there is a dearth of information in terms of process methods, because one has to wait for a product to emerge.

Marletta posed a question about addressing the IC’s challenges with technology recognition and transfer. Murdick noted the importance of accepting that the IC has limited competency to tailor and implement highly technical capabilities. Clear communication of expectations would be beneficial; and demonstrations, use cases, and defined problem sets would also be helpful in attracting “champions.” He added that the IC’s organizational constructs could be changed to place greater value

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×

on innovations created outside of the IC. Marletta also asked about ways to confront the challenge of the structural reorganizations that result in a loss of champions. Murdick remarked that, in many cases, S&T staff continually rotate between being put together and being dispersed among mission groups. The best approach is to implement a solution that is robust to this changeability. He emphasized that reorganization is unavoidable in any institution, but because of the IC’s constant reorganizations, he expressed concern about its ability to perform R&D threat assessments at the necessary scale.

EMERGING TECHNOLOGIES AND NATIONAL SECURITY: EXAMPLES OF THE CONNECTIONS

Richard Johnson, chief executive officer, Global Helix, emphasized that 21st-century innovation will rely increasingly on biology and suggested that the IC increase its awareness of and engagement with R&D developments in synthetic biology and engineering biology (SB/EB) in particular. He explained that SB/EB has the potential to harness the intrinsic capabilities of biological systems to build life-like systems to explore the mechanisms that govern biology; manufacture products that are safer, more sustainable, and environmentally friendly; improve human health; develop computing and information storage devices; and be used as active materials that sense and respond to their environment. More than 40 countries have SB/EB strategies and R&D programs, the most comprehensive and advanced of which are in China, the United States, and the United Kingdom.

Johnson emphasized that the IC could monitor investments in cutting-edge SB research. For example, SynBioBeta releases quarterly market reports, projecting $36 billion in new investment for SB companies in the United States this year. And The Bio Revolution: Innovations Transforming Economies, Societies, and Our Lives7 anticipated that 60 percent of the world’s physical inputs could be made using biological means as well as a projected $2–4 trillion of annual direct economic potential globally for SB in 2030–2040. He added that the IC could track university resource allocation trends (e.g., new centers, faculty hiring, emerging institutes) and international funding trends (e.g., Shenzhen Institute of Synthetic Biology). The IC could also build, deploy, and iterate a new robust analytical toolkit for SB/EB R&D, Johnson continued. For example, the Engineering Biology Research Consortium (EBRC) offers a critical assessment of the potential of EB and related roadmaps,8 with contributions from more than 90 scientists and engineers from a range of disciplines, representing more than 30 universities and 12 companies. Developing next-generation analytical mapping and modeling tools is key, he asserted, as is focusing more on anticipatory intelligence and the changing landscape of SB (via horizon scanning and improved identification of novel risk pathways). He suggested that the IC reframe its mindset for next-generation SB/EB, broadening the scope of U.S. national security interests.9 In the midst of a tool revolution, SB is increasingly focused on the automation of biology and workflows, use of AI and new data, and digitalization of biology. He maintained that if the IC does not focus on these tools and applications, it will miss many opportunities. Biodesign, the notion of designing R&D to achieve particular functionalities or outcomes, is also important for the IC. He referenced the Global Biofoundries Alliance as an example of next-generation critical infrastructure that the IC could consider. It is crucial, he continued, for the IC to leverage biology as a technology platform for multiple transformative applications. This drives the next production revolution, which will have major implications for U.S. economic growth and for the competitiveness of its manufacturing base. As the IC expands the focus of its landscape (e.g., bioinformatics, neurotechnologies, gene editing, engineering microbiomes), he indicated that it is also important to consider risk mitigation and to focus more strategically on dual-use technologies. Johnson explained that there are estimates that 40 percent of the next generation of storage computing and semiconductor technologies will be bio-based, and it is crucial that the IC keeps track of those next-generation technologies.

Johnson underscored the value of a more coordinated, whole of government strategy for critical emerging technologies in R&D. For example, interagency workshops on SB could be integrated

__________________

7 McKinsey Global Institute, 2020, The Bio Revolution: Innovations Transforming Economies, Societies, and Our Lives, https://www.mckinsey.com/industries/pharmaceuticals-and-medical-products/our-insights/the-bio-revolution-innovations-transforming-economies-societies-and-our-lives.

8 See the EBRC Roadmaps website at https://roadmap.ebrc.org.

9 Johnson referred participants to the following publications: National Research Council, 2009, A New Biology for the 21st Century, Washington, DC, The National Academies Press; National Research Council, 2014, Convergence: Facilitating Transdisciplinary Integration of Life Sciences, Physical Sciences, Engineering, and Beyond, Washington, DC, The National Academies Press.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×

with both the IC and the defense community. A proactive strategy could be implemented for international efforts, and an enterprise similar to the six-academies initiative10 could be developed to provide insights.

Serving as session moderator, Tony Fainberg, senior program officer, National Academies, asked how EBRC is addressing security. Johnson described Malice Analysis, a program that is gauging how university students think about risk, and noted a broad range of activities in security; some are funded by the Department of Homeland Security or DoD, and others are funded by non-governmental organizations.

Melissa Flagg, senior fellow, CSET,11 Georgetown University, explained that all of the IC and DoD institutions that focus on science were created and optimized in a fundamentally different world than exists today. Now, the United States and China split approximately 50 percent of the $2.2–2.4 trillion of annual R&D globally; $1–1.2 trillion of R&D funding comes from the rest of the world. This demonstrates that S&T and R&D are not bilateral issues, which creates a challenge for the IC. Many countries are doing high-quality work in niche areas, yet the United States makes the error of discounting them because they lack a comprehensive portfolio of R&D and R&D funding. Flagg noted that CSET’s global models are invaluable in providing context to understand where technologies are unique or are emerging quickly and require quick decisions (e.g., awareness of China’s activities is insufficient when 40 other countries are engaging in the same activities)—a non-contextualized focus on adversarial S&T is untenable.

Flagg pointed out that it is also difficult for U.S. institutions to adjust to the new standard of highly collaborative science, which has enabled small English-speaking nations to increase their presence and capabilities. The United States often asks its allies to make the same choices it is willing to make, but it is important to recognize that such choices could hurt those allies. She explained that in the United States, not only is 78 percent of science now funded by non-governmental entities but also 90 percent of science is performed outside of the government, and a similar trend is apparent across the world. Thus, she said that it is important for the United States to rethink structures for forming alliances, improve engagement with and understanding of the domestic R&D landscape, and reconsider the IC’s overall approach. Flagg underscored that the United States is unaccustomed to building partnerships in R&D where it needs something from another country. Because of this mindset, the United States often duplicates or ignores research. This presents a challenge for DoD and the IC, she continued, especially in terms of emerging technologies. She reiterated Murdick’s assertion that internal experts are not necessarily the best people to validate research; novel structures would be useful.

Flagg noted that the level of internationalization and collaboration across topics of science varies dramatically—for example, China has a low level of collaboration for material science but a high level for energy and the environment. When the United States enters into conversation with potential partners, it is important to understand not only the risk but also the risk of protective choice that the United States is asking them to make. She asserted that the IC often overlooks lost opportunity costs and attempts to eliminate all risks, which is an unrealistic perspective. Furthermore, “military-only” technologies are becoming irrelevant, as most technologies have dual/commercial use. Novel approaches to consortia are one way to address these challenges. She suggested, however, that if the United States is committed to developing fruitful partnerships, neither DoD nor the national security apparatus should lead, because the allies’ international S&T strategies focus on the economy instead of on national security. She cautioned that if the United States does not understand its allies, it will not be able to understand its adversaries or the broader S&T landscape.

Flagg described recent CSET research on the top 50 global defense companies by revenue, which comprise about $500 billion of annual revenue from global militaries alone. Investment activity and mergers and acquisitions activity were tracked using Crunchbase, and relevant data sets were used to study disclosed

__________________

10 See National Academy of Engineering and National Research Council, 2013, Positioning Synthetic Biology to Meet the Challenges of the 21st Century: Summary Report of a Six Academies Symposium Series, Washington, DC, The National Academies Press.

11 Flagg shared the following links to expanded explanations of CSET’s efforts: https://cset.georgetown.edu/publication/global-rd-and-a-new-era-of-alliances/, https://cset.georgetown.edu/publication/research-security-collaboration-and-the-changing-map-of-global-rd/, https://cset.georgetown.edu/publication/comparing-the-united-states-and-chinas-leading-roles-in-the-landscape-of-science/, https://cset.georgetown.edu/publication/the-public-ai-research-portfolio-of-chinas-security-forces/, and https://cset.georgetown.edu/publication/tracking-ai-investment/.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×

investments in AI companies. When looking through the lens of investment, a different S&T landscape emerges. In closing, she mentioned that she is more knowledgeable of emerging technology today at CSET than she was during her tenure as Deputy Assistant Secretary of Defense for Research, in part because the IC fails to leverage open source data.

Schiffer asked how Flagg would ensure that the Pentagon and the IC gain a better understanding of the S&T landscape. Flagg stated that ONR and the Air Force Research Laboratory (via the Air Force Office of Scientific Research) are the only institutions that have created a structure to support ongoing access to data for decision making. However, these data are rarely used to contextualize decisions by leadership. The basic research literature cannot be used to answer every question, and a misunderstanding of diffusion problems, in particular, persists. CSET has access to 90 percent of the global literature, as well as patents, investment data, on-staff translators, and survey capabilities; this suite of lenses with which to look at technology allows CSET to answer a more distinct set of questions. She asserted that governmental organizations simply need to commit to investing in and providing ongoing support of open source efforts. Fainberg asked how DoD and the IC could engage in collaborative R&D while maintaining situational awareness and protecting items of critical interest. Johnson highlighted opportunities for collaborations with allies, and he suggested adapting the In-Q-Tel model on a global scale. He also advocated for engaging with key players in emerging technologies who are outside of the government. Flagg commented that it is important to have traditional institutional relationships as well as awareness of the fact that the vast majority of global R&D is disconnected from government. She suggested funding a seat at the table in a consortium. Most importantly, if the United States wants to become more knowledgeable of global developments, it has to enter the conversation in a way that benefits partners and does not create an opportunity cost (e.g., via export controls and other security measures). The United States has to begin to articulate access to data sets, infrastructure at laboratories, and other opportunities that it can offer potential partners, she continued. Fainberg wondered if opportunities exist for research in unclassified areas at the international level. Flagg remarked that there are many applied opportunities (e.g., manufacturing, productization, and investment pooling) with the Five Eyes if the United States relaxes its export controls. She reiterated that the United States has to be realistic about the choices it is asking partners to make.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×

PLANNING COMMITTEE MEMBERS Frederick R. Chang (Chair), Southern Methodist University; Michael A. Marletta (Vice Chair), University of California, Berkeley; Lilian Alessa, University of Idaho; Tomas Diaz de la Rubia, University of Oklahoma; Vishva M. Dixit, Genentech; Donald D. Duncan, Johns Hopkins University Applied Physics Laboratory; Gerald L. Epstein, National Defense University; Kathleen Fisher, Tufts University; James R. Gosler, Johns Hopkins University Applied Physics Laboratory; Laura M. Haas, University of Massachusetts Amherst; Robert F. Hale, Center for a New American Security; Daniel E. Hastings, Massachusetts Institute of Technology; Frances S. Ligler, Texas A&M University; Willie E. May (until May 2021), Morgan State University; Bernard S. Meyerson, IBM Corporation; Lisa J. Porter, LogiQ; Peter Schiffer, Yale University; Anthony J. Vinci, Center for a New American Security; Michael S. Witherell, Lawrence Berkeley National Laboratory.

STAFF Dionna Ali, Associate Program Officer; Anita Eisenstadt, Program Officer; Shenae Bradley, Administrative Assistant; Anthony Fainberg, Senior Program Officer; Michael Niles, Senior Program Officer; Nia Johnson, Program Officer; Alan H. Shaw, Director.

DISCLAIMER This Proceedings of a Workshop—in Brief was prepared by Linda Casola as a factual summary of what occurred at the workshop. The statements made are those of the rapporteur or individual workshop participants and do not necessarily represent the views of all workshop participants; the planning committee; or the National Academies of Sciences, Engineering, and Medicine.

REVIEWERS To ensure that it meets institutional standards for quality and objectivity, this Proceedings of a Workshop—in Brief was reviewed by Bernard Meyerson, IBM Corporation; Melissa Flagg, Flagg Consulting, LLC; and Peter Sharfman, The MITRE Corporation.

SPONSORS This workshop was supported by the Office of the Director of National Intelligence.

SUGGESTED CITATION National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. https://doi.org/10.17226/26604.

Division on Engineering and Physical Sciences

Copyright 2022 by the National Academy of Sciences. All rights reserved.

images
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page1
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page2
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page3
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page4
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page5
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page6
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page7
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page8
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page9
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page10
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page11
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community - Understanding the International Aspect of the Landscape: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26604.
×
Page12
Leveraging the Future Research and Development Ecosystem for the Intelligence Community—Understanding the International Aspect of the Landscape: Proceedings of a Workshop—in Brief Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

On June 9, 2021, the Intelligence Community Studies Board of the National Academies of Sciences, Engineering, and Medicine convened a virtual workshop on behalf of the Office of the Director of National Intelligence. This unclassified workshop was the second of two data-gathering workshops.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!