National Academies Press: OpenBook

Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief (2022)

Chapter: Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
images Proceedings of a Workshop—in Brief

Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community

Proceedings of a Workshop—in Brief


On April 19, 2021, the Intelligence Community Studies Board (ICSB) of the National Academies of Sciences, Engineering, and Medicine convened a virtual workshop on behalf of the Office of the Director of National Intelligence. This unclassified workshop was the first of two data-gathering workshops. Panel 1 on Intelligence Needs and U.S. Industry featured panelists Eliahu Niewood, The MITRE Corporation, and Bernard Meyerson, IBM Corporation. Panel 2 on Intelligence Community Needs and U.S. Academia featured John Forte, Virginia Tech Applied Research Corporation, and Randy Katz, University of California (UC), Berkeley. Panel 3 on Intelligence Community Needs and Federal Research and Development (R&D) agencies featured Fleming Crim, National Science Foundation (NSF), and Chris Hassell, Department of Health and Human Services (HHS). This workshop will help to inform a consensus study report investigating how the Intelligence Community (IC) could leverage the evolving R&D ecosystem to meet its future needs and garner the most value from its investments in science and technology (S&T). The consensus study is being carried out by a committee assembled by ICSB. The consensus study committee has also gathered input over the course of several committee meetings from various experts and will also gather more data from a following workshop in June. The consensus study committee vice chair Michael Marletta, C.H. and Annie Li Chair in the Molecular Biology of Diseases, UC Berkeley, invited panelists and participants to share their perspectives on the central question of this workshop: How could the IC better leverage the R&D work of various sectors of the U.S. research community, including industry, academia, and the federal R&D agencies?

INTELLIGENCE COMMUNITY NEEDS AND U.S. INDUSTRY

Eliahu Niewood, vice president, The MITRE Corporation, helped to stand up MITRE’s Cross-Cutting Urgent Innovation Cell, a cross-sponsor endeavor that synchronized operational analysis with prototyping and experimentation. This organization helped to identify and close existing gaps with impactful new technology. Prior to joining MITRE, Niewood served as an Intergovernmental Personnel Act (IPA) Mobility Program participant, advising the director of the U.S. Air Force Rapid Capabilities Office (RCO) on technical issues and helping the Department of Defense (DoD) to embrace an innovation initiative. Niewood pointed out that although RCO fields capabilities rapidly and is one of the more innovative organizations within DoD, it is difficult to break down government bureaucracy and foster change on a larger scale.

Niewood guided workshop participants to a publication by Anthony Vinci, former senior intelligence official and chief technology officer and associate director for capabilities, National Geospatial-Intelligence Agency (NGA), who stressed that “intelligence agencies must break down cultural barriers, invest in technology, and dedicate entire offices to artificial intelligence

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×

(AI) and automation-driven intelligence.”1 Niewood noted that AI and automation-driven intelligence are only two of the available technologies emerging from industry’s R&D ecosystem that are of interest to the IC; others include high-performance computing, biotechnology, nanotechnology, and microelectronics. He also referenced the 2019 National Intelligence Strategy, which asserted that the IC has to “bolster innovation to constantly improve [its] work.” Niewood emphasized that “advances in technology are driving evolutionary and revolutionary change across multiple fronts.” Niewood underscored that for the IC to become more agile, innovative, and resilient, it would need to dramatically reshape how it operates and leverages this evolving R&D ecosystem. Otherwise, he continued, the IC risks becoming irrelevant, unable to identify trends in a timely fashion: the nation could be left without the necessary critical intelligence to prosper in a world that is increasingly threatened by climate change, pandemics, inequalities, and the growing capabilities of peer and non-peer adversaries. Despite this recognition that change is needed, the situation has not improved—for example, NGA is still considering how to use machine learning and machine vision to change how reams of imagery are processed; the National Reconnaissance Office (NRO) is slow to adopt approaches to enable rapid evolution and take more risk around space systems; and the Federal Bureau of Investigation (FBI) struggles to integrate large volumes of information. Niewood referenced Maintaining the Intelligence Edge: Reimagining and Reinventing Intelligence Through Innovation,2 which suggested that “there is no shortage of opportunities to apply technology across intelligence missions today” and that “the primary obstacle to intelligence innovation is not technology, it is culture.” He described this as an innovation problem: the federal government has created barriers that prevent the IC from effectively partnering with industry to leverage technology.

Niewood noted that these barriers often arise from acquisition processes. Large acquisition efforts (common in the IC and DoD) are focused on writing and reviewing detailed (100+ pages) requirements. These rigid documents are then distributed to a contractor, who has no flexibility to prototype solutions, experiment, or inject new technology. The focus seems to be on modernizing an existing capability, he explained, instead of on analyzing operator needs and applying a new technology to address a critical mission gap. Bernard Meyerson, chief innovation officer emeritus, IBM Corporation, echoed the problems involved with this “gross overspecification of requirements.” He wondered why no feedback mechanism exists and why there is no person of authority to override these processes when a better approach arises. Niewood acknowledged that there are very few organizations in the IC and DoD with this level of flexibility in their large acquisition processes. One exception is RCO, which writes requirements only at the paragraph level and has the systems engineering and analysis capabilities in-house to make trade-offs when appropriate. Lilian Alessa, President’s Professor and director of the Center for Resilient Communities, University of Idaho, remarked that requirements can enable precision. She wondered if a better approach could be to develop a core requirement alongside adaptive, context-based requirements. Niewood alluded to a similar model, portfolio-based acquisition,3 which is effective only if the acquisition team has the ability to make trade-off decisions, to understand whether systems will perform appropriately when shifts are made, and to evaluate associated costs. Giving acquisition programs the system engineering and operational analysis capabilities to make smarter decisions would likely result in better products, he stated. Alessa pointed out that portfolio requirements and adaptive requirements are slightly different, but she asked if these more flexible approaches would also require that the user have the appropriate training for and understanding of potential applications of the technology. Niewood suggested that acquisition should be done with the user and the usability of the technology in mind; if users are given a good product, they will learn how to use it. In terms of creating space for adaptation, Meyerson wondered whether it is the user who lacks the right skill set or those who write the requirements who lack the ability to delegate authority. Niewood observed the strictures placed on acquisition departments, which lack both tools and authority. Meyerson and Niewood agreed on the value of dynamic acquisition—that is, meeting the overall goal in whatever way makes the most sense in terms of evolving technologies and programs. Niewood commented that some sectors of NRO do nimble work in

__________________

1 A. Vinci, 2020, The coming revolution in intelligence affairs: How artificial intelligence and autonomous systems will transform espionage, Foreign Affairs, August 31, 2021.

2 Center for Strategic and International Studies Technology and Intelligence Task Force, 2021, Maintaining the intelligence edge: Reimagining and reinventing intelligence through innovation, https://csis-website-prod.s3.amazonaws.com/s3fs-public/publication/210113_Intelligence_Edge.pdf, p. ix.

3 P.J. Modigliani, 2015, Portfolio acquisition: How DoD can leverage the commercial product line model, https://aida.mitre.org/wp-content/uploads/2018/08/Portfolio-Acquisition-NPS-Paper.pdf.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×

prototyping and experimentation, but this practice is not embedded throughout larger acquisition programs. The IC is challenged by a lack of time and resources (and, at times, a concern for vulnerabilities) to leverage new, possibly expensive, emerging technology across the enterprise.

Anthony Vinci, adjunct senior fellow with the Technology and National Security Program at the Center for a New American Security, commented on Niewood’s presentation. Vinci highlighted the differences both within the IC and between the IC and DoD in terms of program manager expertise, contracting officer expertise, and ability to work iteratively with industry. He asked about specific models to leverage R&D that could be applied more broadly throughout individual agencies or across the IC. Niewood pointed to Project Maven as a model of success because it identified specific areas of overlap between a critical mission gap and an emerging technology. Moving away from the large, rigid acquisition process, Project Maven succeeded in delivering capabilities to the user in a meaningful way. Even though this is not an example of partnering directly with technology developers, Niewood continued, it still demonstrates successful leverage of available technology. Donald Duncan, senior advisor, Asymmetric Operations Sector, Johns Hopkins University Applied Physics Laboratory, wondered why the lean start-up methodologies4 to motivate innovation that were being taught at Stanford University were not embraced by the IC or DoD. Although Niewood was not familiar with that particular initiative, he mentioned that there are differences in the national security model and the venture capital model. Although the latter could be leveraged in some cases, innovation is needed around the IC’s biggest mission problems, which often cannot be addressed via the venture capital model.

Niewood reiterated that the IC would benefit most from identifying specific quantitative mission challenges5 and describing them in an unclassified but sufficiently detailed manner to allow the broader industrial community and the academic community to think about solving those problems with emerging technology. This is a better approach from a national security perspective, he continued, than having a massive strategy. A current challenge for the IC is to minimize the impact of the malign influence campaigns from adversaries that increase political divides within the United States. AI and data analytics could be applied, and metrics could be developed to measure the impact of these campaigns. He remarked that involvement from industry to bring this technology to bear to predict, understand, and deter threats would be useful. Meyerson commented on the prevalence of open innovation and wondered how that could affect the IC’s involvement with the R&D ecosystem. Although open innovation might not create an advantage for the United States, it creates a level playing field and prevents surprises from an adversary. Niewood advocated for breaking down the barriers that previously prevented the United States from taking advantage of opportunities to engage in that level playing field as well as for increasing open dialogue and open innovation among the broader community.

Committee chair Frederick Chang, Bobby B. Lyle Centennial Distinguished Chair in Cyber Security, Southern Methodist University, noted that while diversity in gender, ethnicity, and age are typical goals for organizations, he wondered about diversity in discipline and expertise (e.g., psychology, biology, or chemistry) in the IC. Niewood acknowledged that the IC has made progress in achieving diversity in technical and non-technical backgrounds. He emphasized the value of diverse teams; different viewpoints lead to innovation, and it is not possible for the IC to predict the actions of other countries and cultures if it is not as diverse as the rest of the world. The IC has been less successful in achieving diversity in gender, race, and sexual orientation.

James Gosler, senior fellow, Johns Hopkins University Applied Physics Laboratory, observed that mission owners tend to take their sensitive problems to organizations with whom they have existing relationships (e.g., MITRE, national laboratories, university-affiliated research centers [UARCs], Lockheed Martin, Northrop Grumman), whether or not those relationships are efficient. He inquired as to how the IC would develop relationships with others who could have insights into leading-edge technology but might not realize their relevance to the IC. Niewood suggested starting with problems that can be described at an unclassified level as a way to attract non-traditional industry partners. Once these relationships with smaller companies begin to take shape, when larger or classified problems arise, “interpreters” can break a problem down into components that can be dispersed among appropriate companies. Robert Hale, adjunct senior

__________________

4 S. Blank, 2017, The red queen problem: Innovation in the DoD and intelligence community, https://steveblank.com/2017/10/17/the-red-queen-problem-innovation-in-the-dod-and-intelligence-community/.

5 E.H. Niewood, 2021, Intelligence after next: Mission-based challenges for the intelligence community, https://www.mitre.org/publications/technical-papers/intelligence-after-next-mission-based-challenges-intelligence-community.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×

fellow, Center for a New American Security, asked if the IC would benefit from special organizations with extra flexibility. Niewood replied that the IC has a few such organizations that reduce bureaucracy, although those arose to address deeply classified problems—the most important, immediate goal for the IC is to focus on its mission problems.

In his talk as a presenter, concluding the first session of the workshop, Meyerson explained that IBM has been driven by innovation for more than 100 years. He portrayed successful innovation as a “creative disruption” that can change the global business landscape. For example, the world’s largest taxi company (Uber) has no vehicles, and the largest popular media platform (Face-book) creates no content. He stressed that the IC would benefit from greater awareness of creative disruptors. Manufacturers are working in virtual and emergent environments, which makes it even more difficult for the IC to stay current. He stated that the IC has struggled to make “big bets” on technology, owing to boundaries that prevent adaptation—it is essential to look both forward and backward to avoid technological surprise. Meyerson described System/360, IBM’s original mainframe, as an example of a big bet that IBM spent its entire revenue over 1 year to develop. If System/360 had not been successful, IBM would have been destroyed. Bets can be especially risky in emerging fields, but the benefits of the resulting innovations can be substantial. For example, IBM learned early that it could remain competitive only if it pursued open technology, and it placed a big bet on Linux to replace AIX. This is a situation in which a successful program (AIX) was sacrificed for a new program (Linux): the technical advantages of Linux outweighed the profitability of AIX. He emphasized that the IC faces similar difficult trade-off decisions, although perhaps more in terms of functionality than profitability.

Meyerson indicated that both the Defense Advanced Research Projects Agency (DARPA) and DoD intelligence agencies have helped to enable some of IBM’s big bets on innovation. He shared an anecdote about bipolar transistors that began collapsing after several hours of operation, which prompted the belief that there would be no future for—and therefore no need for investments in—the core of silicon technology. This led the IC to invest heavily in gallium arsenide and indium phosphide, with an interest in optical properties and computing capabilities. However, there were foundational issues that prevented either technology from being useful. A Bell Laboratories paper from the 1950s had demonstrated that the same technologies used to make gallium arsenide and indium phosphide could be applied to silicon, if germanium was added to the silicon. Yet, fundamental work with silicon germanium did not begin until the 1980s. IBM determined that using standard commercial technology (e.g., silicon) could provide a better and much less expensive solution than gallium arsenide and indium phosphide. Meyerson noted that people were interested in the possibility of reducing the cost of a major DoD product, but most expected the technology to fail. DoD and DARPA were the only organizations willing to invest in this work, and IBM came to an important realization during one of its experiments: despite 30 years of research on the need to remove an oxide layer from the silicon at 1,000°C in order to make a transistor (a temperature much too high for silicon germanium to remain intact), that oxide was found not to exist. Thus, IBM successfully created silicon germanium, which prompted a drastic increase in transistor performance in orbit at nearly no cost differential (versus the use of basic silicon) and opened up brand new markets for the IC. He explained that the IC essentially built a dual-use program—IBM’s interest was in using the technology for WiFi. Previously, WiFi was a commercial product sold almost exclusively to the military; IBM created a partnership with Intersil, claiming more than 99 percent of the WiFi market for nearly 10 years. Meyerson summarized that silicon germanium technology was enabled by a big bet from the intelligence agencies, and IBM solved a problem on which the IC had been spending so much but making little progress. He added that this potential for flexibility and innovation on the part of the IC should still exist today. He championed the value of partnering with the best experts to innovate.

INTELLIGENCE COMMUNITY NEEDS AND U.S. ACADEMIA

Frances Ligler, Ross Lampe Distinguished Professor of Biomedical Engineering, North Carolina State University, introduced the topic of the second session of the workshop: how the interface between academia and the IC has successfully fostered the translation of important R&D products, as well as how it could improve. Randy Katz, vice chancellor for research, UC Berkeley, explained that approximately 60 percent of the $800–900 million in the university’s sponsored research comes from the federal government. NSF and the National Institutes of Health (NIH) are the primary sponsors; the more mission-oriented agencies have less of a presence, as the UC System has a policy that it will not

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×

undertake classified research.6 He commented that, prior to 2016, collaboration in science and engineering research and advanced development was embraced as a positive international norm. Over the past few years, issues have emerged around the security of research and foreign engagement of American universities, primarily with nations perceived to be scientific or geopolitical competitors. An additional UC System policy prohibits any projects that restrict participation based on national origin or citizenship. Katz mentioned that approximately 40 percent of UC Berkeley’s international graduate student population is from China, and another large cohort is from Iran, all of whom are highly qualified. When national origin restrictions were being promulgated in certain open research projects during the prior administration, he turned away multiple millions of dollars in potential awards that would have limited participation of international students.

Katz described an Australian website that classifies the ways in which universities in China are connected to the Chinese military, a list that could be used to restrict U.S. academic collaboration with higher education institutions in China. Despite the fact that Tsinghua University is known as one of the most prestigious universities in China and one that has collaborated with U.S. institutions for decades, it is listed as a “high-risk” institution. Yet, the workforce from this and other Chinese universities that is drawn to the United States becomes the “intellectual leaders, faculty members, company founders, and engines of translation of best practice from university to industrial activities.” He underscored the importance of understanding the dynamics of international collaboration and what that means for the ability to work on projects of relevance to the IC. Owing to UC Berkeley’s large population of researchers who may not have U.S. citizenship and the IC’s hesitancy to work in open environments, partnerships with the IC sometimes pose challenges.

Katz provided a few examples of UC Berkeley’s interactions with the IC writ large. UC Berkeley’s RISELab7 develops state-of-the-art open-source software. The main software artifact that emerged from the project is Spark, which is widely used for big data analyses. The funding that transferred through DARPA for this project originated with the National Security Agency (NSA)—demonstrating that an intelligence agency can work successfully within an open research project. He also described a successful law enforcement agency collaboration (with the FBI) that occurred during the 1990s. The FBI was particularly interested in small-scale microphones that could harvest energy from the environment for wireless communication. This project propelled an understanding of both the science and engineering of low-power, high-integration chipsets. This project spun off dozens of companies and the Berkeley Wireless Research Center,8 which works closely with federal agencies and industry to advance research. Both are examples of UC Berkeley’s ability—with consideration for its commitment to open research, international collaboration, and an international workforce—to pursue projects of importance to federal agencies with intelligence or law enforcement problems. In closing, Katz said that there is no impediment to working with the IC on cutting-edge technological projects in the kinds of models of collaboration used with NSA and the FBI. The impediment has only been the perception that the university’s open environment cannot be pursued in the context of projects of interest to the IC. He emphasized that UC Berkeley will not change its commitment to an open environment or its collaboration with international researchers; he expressed his hope that the federal government will continue to expand its acceptance of this model.

Meyerson inquired as to how often programs become truly dual-use, with agency engagement as well as the development of a commercial enterprise. Katz replied that Spark is in the hands of Databricks, a company with an $80 billion valuation. In this case, a substantial commercial entity emerged, NSA gained early insights into the technology, and faculty earned recognition for moving technology into engineering practice. He added that many faculty members have start-up companies that are pursuing the commercialization of technologies funded under federal support. Tomas Diaz de la Rubia, vice president for research and partnerships, University of Oklahoma, wondered if there are opportunities for UC Berkeley to work with the IC on an unclassified project, but as it transitions into higher levels of readiness could partner with NSA laboratories for follow-on activities that would lead to classified products. Katz noted that this could be an avenue to explore. However, he pointed out that collaborations with Lawrence Livermore National Laboratory and Los Alamos

__________________

6 Unless in an exceptional case, primarily at national laboratories that are part of the university system’s portfolio.

7 See the RISELab website at https://rise.cs.berkeley.edu/.

8 See the Berkeley Wireless Research Center website at https://bwrc.eecs.berkeley.edu/.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×

National Laboratory (LANL) are more complicated for UC Berkeley than a collaboration with the Lawrence Berkeley National Laboratory, which is an open science laboratory. He also mentioned Department of Energy projects that have restricted facility access to individuals based on national origin—UC Berkeley would not be able to accept an unclassified project that would prohibit non-U.S. citizens from entering a facility.

Peter Schiffer, Frederick W. Beinecke Professor of Applied Physics, Yale University, questioned whether the IC is successful in attracting UC Berkeley’s best graduate students to its workforce. Katz responded that many aspire to pursue academic careers; however, because there are limited jobs in academia, exposure to industry and government career paths would serve graduate students well. Although compensation is much higher in Silicon Valley than in the Central Intelligence Agency, for example, some graduates (with U.S. citizenship) may be attracted to the science and national security missions of the federal agencies. He noted that NSA has thus far done a better job recruiting technical workforce from universities than the IC writ large, which has not successfully communicated its technical and scientific opportunities, but only 1 of Katz’s 52 Ph.D. students works at NSA and none work for law enforcement federal agencies. Ligler asked how the IC could better leverage UC System resources. Katz suggested that the IC learn from federal agencies that have worked successfully with the universities on scientific and technical issues. Ligler also wondered whether IPA experiences with the IC could lead to faculty recognition. Although Katz described his own IPA experience with DARPA as transformational, he had little support at the time from academia. He said that the barriers for an IPA participant with the IC are even greater because those faculty cannot share their research. When faculty work in industry, mechanisms exist to protect company intellectual property (IP) while still allowing visibility to the research community; similar rules of engagement would be valuable for faculty taking leave to work in the IC.

Providing another perspective on the interface between academia and the IC, John Forte, chief executive officer and president, Virginia Tech Applied Research Corporation (VT-ARC),9 explained that VT-ARC was established in 2009 as a 501(c)(3) nonprofit research organization with a mission to deliver tailored analysis, research, and engineering to address problems of national and global importance. He clarified that although VT-ARC is university-affiliated, it is neither a UARC nor a federally funded research and development center (FFRDC). With public, private, academic, and industry partners, VT-ARC leverages VT’s $500 million/year multidisciplinary research and innovation enterprise to build highly collaborative ecosystems that advance science, focus research, and enable the integration of cutting-edge capabilities across multiple domains. One important collaborator is the Hume Center for National Security and Technology,10 the mission of which is to “perform interdisciplinary research and development, provide experiential learning and education, [and] solve challenges in national security by strengthening technology, standards, public policy, and the national security workforce.”

Forte summarized the results of a poll that he conducted among a small number of researchers to whom he had access through VT-ARC to better understand (1) how well the IC works with academia, (2) examples of successful and unsuccessful IC–academia projects, (3) how to leverage R&D outside of the IC, and (4) suggestions for improving IC–academia relationships and transitioning and operationalizing concepts and technologies. Poll respondents suggested that relationships between the IC and academia are “mediocre at best.” In general, Forte explained, the IC does not work well or broadly with academia. Informing and funding basic and low technology readiness level (TRL) research is often not a starting point of engagement for the IC, which minimizes the success of future capabilities being operationalized. Recruitment is successful only in pockets, he continued, and lasting relationships with recruiters is difficult. It is also challenging to submit concepts to targeted IC organizations and to get research appropriately scoped—an intermediary (e.g., FFRDCs) is often needed. Most requests for proposals require access to high-side networks, and contract and funding mechanisms vary by agency. There is often a greater focus on near-term operational support (months to 1–2 years) and less emphasis on preparing for the future, he noted. Many university IC-focused hubs and centers exist; however, they are typically only connected to the IC in very specific ways, funding levels often do not support what is required to fully escalate TRLs, and cross-center collaboration and associated connections into IC agencies is confusing.

__________________

9 See the VT-ARC website at https://vt-arc.org/.

10 See the Hume Center for National Security and Technology website at https://hume.vt.edu/.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×

Forte revealed that in 2019, nearly 4,300 degree-granting U.S. colleges and universities had $84 billion in R&D activities, but that opportunity space is not being leveraged by the IC: university-based applied research is highly capable but is utilized less than that of industry counterparts. Sharing additional results from the poll, he commented that the IC and the federal government more broadly could better understand that working with academia is not the same as working with contractors. Relationships with faculty are unique and offer opportunities beyond leveraging research (e.g., externships, connections to industry partners). Long-term relationships can lead to professional development for faculty and capture talent for the IC—the university environment in the United States has a vast and increasing global pull of faculty and students. Freedom of academic research is an essential part of a university ethos, he continued, and there is an important link among research, students, and faculty, in terms of the funding that is injected into the system (i.e., responsibilities to grant degrees to students and to advance S&T).

Forte’s poll respondents listed the following R&D project successes: the IC postdoc program; the Intelligence Advanced Research Projects Activity (IARPA) Open Source Indicators program;11 any project set up to enable dual-use technologies; the Scholarship for Service Program;12 longer-term programs with direct relationships; the IC Centers for Academic Excellence;13 and North Carolina State University’s Laboratory for Analytic Sciences,14 University of Maryland’s Applied Research Laboratory for Intelligence and Security,15 NSA–DoD Student Ed programs (e.g., DoDCySP16), and the Senior Military College Cyber Institute. Forte added that when the IC is interested in both project and student outcomes, this often leads to a stronger future workforce, better relationships, and more impactful capabilities. Regarding R&D challenges, poll respondents observed that some projects, particularly those of longer duration, failed to aid in pushing student clearances. Even those universities with hubs and centers that are deep in staff with clearances still have difficulties engaging with the IC. Furthermore, requirements are rarely articulated in a way that ensures that research and products will be aligned with a mission need. Forte emphasized that timelines with universities are different; fielding a capability in 1 year or less under current scenarios is difficult at best, and burying a university under a prime contractor often limits impact and partnerships. Poll respondents provided suggestions about leveraging R&D from other sectors: replicate DoD’s approach to a more open R&D environment; search for ways to enable R&D programs that have dual use and to scan industry for S&T that has dual use; and identify counterparts in other agencies with similar goals, allowing them to be the more transparent partner in the relationship with those in other sectors. Last, Forte shared poll respondents’ overarching suggestions on improving collaboration between academia and the IC.

  • The IC should fund more R&D tasks that require more than months to accomplish and have applications broader than what are now the most common assignments—urgent responses to specific, short-term operational IC needs;
  • Devise more transparent time frames for returns on investment by the IC from academic researchers with whom it contracts;
  • Enable efforts to demystify the IC and educate academia;
  • Determine how to overcome the challenge of non-U.S. researchers at an institution that performs classified research and protection of research from dissemination that is not desired by the IC funder;
  • Acquire tools to identify where great S&T is occurring (beyond the “big five” universities);
  • Increase long-term funding (money in the academic community facilitates support of graduate students, post-docs, faculty summer salaries, and operation of research laboratories);
  • Create and articulate transition plans and partnerships before funding, which may involve different university, industry, and government players, as well as different phases;
  • Adopt mechanisms that enable flexibility in research and impact, including indefinite delivery–indefinite quantity contracts, cooperative R&D agreements, educational

__________________

11 See the Open Source Indicators program website at https://www.iarpa.gov/index.php/research-programs/osi.

12 See the Scholarship for Service Program website at https://www.sfs.opm.gov/default.aspx.

13 See the Centers for Academic Excellence website at https://www.dni.gov/index.php/iccae.

14 See the Laboratory for Analytic Sciences website at https://ncsu-las.org/.

15 See the Applied Research Laboratory for Intelligence and Security website at https://www.arlis.umd.edu/.

16 See the DoDCySP website at https://public.cyber.mil/cysp/.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×

    partner agreements, and partnership intermediary agreements;

  • Avoid pitfalls in promising new programs;
  • Accept more risk, adopt an ethos of allowing research to fail (fast), and be bold;
  • View university R&D providers as partners, not suppliers;
  • Determine how to build incentives for publication and commercialization;
  • Develop more pathways to aid in describing the research desired by the IC to potential academic participants, including easier access to submit concepts and provide feedback; and
  • Gather insights on what the IC needs from a particular project, which will result in more impactful capabilities (i.e., the greater the focus on requirements, the greater the chance of delivering something that is already old upon delivery).

Alessa wondered whether a stigma exists among faculty about those working for the IC as opposed to working with the IC in the spirit of collaboration. Forte suggested that the IC codify better processes and relationships. He stated that every academic organization is different; it is important for the IC to understand these cultures and avoid trying to change them. Ligler asked about the most effective way to facilitate dual-use applications for the IC. Forte advised looking at the requirements of other government agencies and augmenting those with a series of threshold requirements for research products. Trusted and close relationships between government agencies (e.g., parts of the IC and DoD)—for example, DoD mechanisms for dealing with funding of applied research and the needs for secrecy involving some products—could enable the IC to resolve such dual-use issues. In other cases, it may be possible to reach out directly to specific industry players and pull in their technology, although that requires a specific function for adapting the technology to the mission capability. Ligler added that companies can often surpass the “valley of death” if their product is useful for both civilians and the IC. Forte indicated that the role of intermediaries could be explored in greater depth to build this type of ecosystem. Gosler asked what the IC system could do to help universities better understand counterintelligence issues, thus building a bridge between the IC and academia for the benefit of the nation. Katz commented that universities have to follow existing rules and regulations and emphasized that universities are ready to work with the counterintelligence community on its investigations. However, he asserted that universities are not police forces; they depend on partnership with federal agencies such as the Department of State and the U.S. Citizenship and Immigration Services to help ensure that the people in the campus workforce are permitted to be in the United States. Universities are trained only to be aware of extraordinary occurrences that could be security risks. Forte added that he is sensitive to these issues, but he agreed with Katz that a university cannot restrict freedom of academic research. If universities act like police forces, they will lose valuable research faculty and research dollars; a different mechanism to expose risks in sensitive work would be useful—perhaps via the IC or intermediaries.

INTELLIGENCE COMMUNITY NEEDS AND FEDERAL R&D AGENCIES

Opening the final session of the workshop, Gerald Epstein, distinguished research fellow, Center for the Study of Weapons of Mass Destruction, National Defense University, mentioned the role of government funders to link academia and the IC but also highlighted the government’s function as a significant performer of research. F. Fleming Crim, chief operating officer, NSF, provided an overview of the structure of NSF as well as how the organization connects to the IC. The mission of NSF is to “promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense; and for other purposes.” He explained that NSF supports academic basic research, providing 27 percent of the total federal support for all science and engineering fields through seven directorates—Biological Sciences; Engineering; Mathematical and Physical Sciences; Computer and Information Science and Engineering; Geosciences and Polar Programs; Education and Human Resources; and Social, Behavioral, and Economic Sciences—and two important units—the Office of Integrative Activities and the Office of International Science and Engineering. This year, he continued, NSF is an $8.5 billion agency, and this funding reaches more than 300,000 people. Crim described three modes of NSF connection to the IC: (1) program collaborations; (2) interagency committees; and (3) international issues, particularly in relation to science and security. One example of a program collaboration is that between the NSF Division of Mathematical Sciences (DMS) and NGA, which is funding “Algorithms for Threat Detection” to develop the next generation of mathematical and statistical algorithms to deal with large spatiotemporal data sets and to better understand artificial neural networks. In this case, DMS and NGA jointly

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×

prepared solicitations in areas of mutual interest, and NSF reviewed the proposals on the basis of intellectual merit and broader impact. He stated that Industry–University Cooperative Research Centers (IUCRCs) provide great potential for NSF and IC partnerships. NSF provides funding to stand up and administer the IUCRCs, and the participating academic institutions, industries, and government agencies fund and determine the direction of the research. There are nearly 80 IUCRCs, three of which have significant participation from NSA. The University of Pittsburgh is the lead academic institution for the Space, High-Performance, and Resilience Computing IUCRC,17 which has more than 30 participants, including government organizations, academic laboratories, and industrial laboratories. In 2020, this IUCRC funded 11 projects. The Accelerated Real-Time Analytics IUCRC,18 led by North Carolina State University, focuses on wrangling rapidly changing real-time data. The Cloud and Autonomic Computing19 IUCRC is led jointly by Texas Tech and the University of Arizona, with six projects in 2020. Another way in which the IC organizations connect to NSF is by providing supplementary funding for NSF projects (e.g., on trustworthy deep neural networks and on research on cyber-physical systems). He emphasized that all of the aforementioned research is unclassified.

Crim remarked that NSF is also connected to the IC via interagency collaborations. For example, the Climate Security Advisory Council is an interagency committee of 20 agencies, including members of the IC, DoD, and the federal science agencies. The purpose of this council is to anticipate climate change effects on national security interests and inform senior policy makers about risks and opportunities. He also mentioned several Office of Science and Technology Policy (OSTP) and National Science and Technology Council (NSTC) groups that have IC participation, such as National Security Council (NSC)–OSTP, Bioeconomy; NSC–OSTP, Critical and Emerging Technologies; NSTC Subcommittees on Machine Learning and Artificial Intelligence, Future Advanced Computing Ecosystem, Quantum Information Science, and Networking and Information Technology R&D; and an Interagency Working Group on Special Cyber Operations Research and Engineering. Other collaborations focus on international issues related to science and security. For example, NSF programs such as the Rubin Observatory, the Antarctic and Arctic Program, and the Seismology Networks do work that connects to the concerns of the IC, as do the NSTC Subcommittee on Research Security; the Interagency Working Group on Research Security and Training; and various outreach events, such as university–FBI meetings.

Responding to Crim’s overview of NSF, Chris Hassell, acting principal deputy assistant secretary and senior science advisor, Office of the Assistant Secretary for Preparedness and Response (ASPR), HHS, explained that when the FBI wanted to explore the foundations of forensic science, it partnered with NSF to initiate programs. Hassell also agreed that IUCRCs provide enormous value for collaborating, moving research forward, and recruiting workforce.

Offering another federal government perspective, Hassell indicated that ASPR has two major areas of focus: (1) deployable medical teams that respond to public health crises in the nation; and (2) medical countermeasure development, via the Biomedical Advanced Research and Development Authority (BARDA). BARDA worked with DoD on Operation Warp Speed during the COVID-19 pandemic to distribute vaccines, therapeutics, and diagnostics. BARDA also works closely with NIH in a systems engineering role to scale up to commercialization and engage with the Strategic National Stockpile. The U.S. Food and Drug Administration and the Centers for Disease Control and Prevention are other important partners for ASPR. Hassell noted that multiple federal agencies would collaborate in the event of a release of a biological agent in the United States. It might first be perceived as a public health situation, but ASPR would coordinate with the IC and law enforcement to better understand whether the release was actually a deliberate attack and address the situation. He emphasized that ASPR, and HHS more broadly, relies on the IC for accurate and timely information to understand the complete landscape of threat determination and analyses.

Hassell remarked that agency collaborators consider what investments should be made to produce more effective medical countermeasures—for example, work on the mRNA platform for

__________________

17 See the Space, High-Performance, and Resilience Computing website at https://iucrc.org/node/7661.

18 See the Accelerated Real-Time Analytics website at https://iucrc.nsf.gov/centers/center-for-accelerated-real-time-analytics.

19 See the Cloud and Autonomic Computing website at https://iucrc.nsf.gov/centers/cloud-and-autonomic-computing.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×

the Pfizer and Moderna vaccines began at DARPA and was acquired by NIH, enabling a swift start for COVID-19 vaccine development. It is crucial to understand what countermeasures need to be readily available for the future, whether the disease agent is naturally occurring, modified, or intentionally produced. He stated that for intentional threats—especially those created by genomic engineering, synthetic biology, or advanced biotechnology—the best defenses are the technologies themselves. HHS works with its IC colleagues to better understand and evaluate those technologies to prepare for rapid response. He asserted that data science, automation, additive manufacturing, and nanotechnology are other important technologies that could be brought to bear by adversaries and thus should be understood and applied in the United States. He highlighted an IARPA program, Finding Engineering-Linked Indicators (FELIX), that links advanced genomic sequencing technology with data analytical technology for rapid analysis to determine whether an organism has been intentionally modified for increased potency. Researchers discovered that the same technology could be useful for disease surveillance situations, and HHS is considering how this construct could be leveraged for the public health mission. Hassell emphasized that the FELIX performers came from academia and industry, with funding from the IC.

Epstein observed significant technical overlap and great potential for common purpose between the IC mission and the research being pursued by the science and federal mission agencies. Although there are clear differences in the way security-focused agencies operate (e.g., with a focus on information control and counterintelligence) as compared to science agencies, both types of agencies have widespread international engagement. He wondered if companies in other countries view engagement with the IC more cautiously than that with other U.S. government agencies and asked whether overt engagement with the IC has the potential to interfere with international engagement activities. Crim responded that NSF’s conversations with NSA are no more difficult to carry forward than those with NIH, for example, because missions and restrictions still have to be aligned, and all NSF research is unclassified and transparent. This approach helps to ease the concerns of international partners. He added that the burden of being perceived to be working with the IC is actually greater in the domestic academic landscape than in the international landscape. Hassell explained that during his tenure with the FBI, when the organization engaged with the academic community, it assumed that institutions would have preconceived notions about dealing with “the men in black,” but that was either not the case or the misconception was quickly dispelled. During FBI-sponsored workshops, several institutions admitted that they did not know what to do when they suspected inappropriate activity in their laboratories. The FBI was able to provide guidelines about who to call and when. The FBI’s high level of engagement reinforced the notion that universities and law enforcement agencies are not segregated. In cases in which academic institutions are hesitant to work with a large agency, Hassell noted that it is important to recognize other opportunities—for example, researchers can be engaged with LANL’s mission without having to work on the nuclear weapons program. Hassell and Crim championed transparency and improved conversations between academia and federal agencies to better understand one another and to reduce barriers to collaboration.

Chang alluded to editorials opining on whether the United States has declined in science and engineering. He wondered whether NSF systematically monitors the global S&T landscape to understand if, in fact, the United States is falling behind. He asked about the best way to measure this, and whether any action would be taken by NSF in response. Crim replied that NSF houses the National Center for Science and Engineering Indicators, which monitors this issue by collecting and analyzing survey data. Each year, in coordination with the National Science Board, these data are published online. Crim noted that the metrics vary. For example, 10 countries currently dedicate a larger fraction of their gross domestic product (GDP) to R&D than the United States; two decades ago, only one country dedicated a larger fraction than the United States. In terms of the fraction of the GDP that is allocated for basic research, the U.S. federal government only funds one-third now instead of the two-thirds of two decades ago. Crim noted that the United States now competes in a global scientific environment. Chang asked what worries the agencies most about this issue. Crim remarked that if the United States is not vigorous in doing science and creating scientists for the future, it will not be healthy as a nation. He fears the United States failing to exploit the intellectual drive that makes it successful, not only in terms of international competition but also in terms of national quality of life. Chang questioned how NSF would respond if a U.S. adversary started to double its initiatives in a particular field. Given that NSF is likely aware of the technologies for which other countries are increasing funding, Gosler wondered if it would share that insight with the IC, which could then use its capabilities to reach ground truth and help the United

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×

States to determine its next investment. Crim said that NSF holds workshops about emerging technologies to better understand scientific opportunities, but it does not systematically take the steps that Gosler mentioned. Crim explained that there are proposals for legislation to direct NSF to compete in certain areas. But, currently, NSF funds one out of every four proposals that it receives, which means that a few billion dollars for highly rated research is being left on the table. He commented that many researchers believe that NSF’s awards are too small or too short in duration, and that NSF is not providing enough of them. Yet even small increases in any of these areas would require a doubling of the entire NSF budget. He reiterated that NSF receives a signal about emerging areas of competition based on the proposals it receives, and it makes decisions about which proposals to fund based on intellectual merit and broader impacts. NSF tries to move where the intellectual opportunities exist, while maintaining the health of its entire scientific enterprise.

Schiffer pointed out that highly skilled people in demand can make a significant amount of money in the corporate sector, without the constraints typical of government work. He asked how HHS recruits the best workforce to identify emerging threats and how NSF encourages people to pursue government service. Hassell emphasized how important it is to get students interested in science at the middle and high school levels. HHS has science, technology, engineering, and mathematics workshops for students, and it provides education on the variety of career trajectories available for federal employees (e.g., FBI agents who are chemists). HHS actively recruits not only from universities but also from pharmaceutical companies and corporations. For purposes of recruitment and retention, HHS tries to create opportunities similar to those in the private sector (e.g., sabbaticals, industrial interactions). Crim agreed with Hassell that concerted efforts are needed to underline the service opportunities of government work. He discouraged recruiting solely from the traditional demographic of science and encouraged increasing broader participation. The other key, Crim continued, is to create a workforce at all levels—that is, scientists with Ph.D. and B.S. degrees as well as skilled technical workers.

Marletta asked what NSF could do better to engage the IC and its mission challenges with the academic and research community in fundable projects. Crim responded that the IC’s provision of supplementary funding is an effective model. A memorandum of understanding (MOU) with a joint solicitation is also effective because it focuses on areas of importance to both organizations. MOUs can be problematic, however, in that each one is customized. He suspected that cumbersome processes (as well as other bureaucratic complications and IP concerns) could discourage the IC from pursuing partnership with NSF—it is important to identify better ways to communicate and to build relationships so that the communities can synchronize. Regarding the issue with MOUs and other bureaucratic obstacles, Hassell suggested intermediate actions—for example, holding HHS program reviews that include government employees from agencies other than HHS. This activity protects IP and confidentiality but promotes exposure. He stressed that it is crucial to find the right people to nurture such efforts. Hassell also tries to involve HHS staff in external program reviews to observe how other agencies function and to gather ideas. Another option is to detail people temporarily to other agencies, which serves to break down cultural barriers and increase awareness. He advised that this activity would have to be done carefully within authorities. He cautioned against letting policy barriers prevent progress—policies can be changed. He added that venues such as the National Academies are valuable in bridging opportunities; when people participate in meetings sponsored by other agencies, they bring ideas back to their own organizations.

Meyerson pointed out that with the rate and pace of technology today, it is not feasible to operate as in the past and be successful. Creating a cross-agency organization that is small and highly empowered could be a viable solution. At IBM, he put together a cross-sectional team of 20 people with excellent networks who had the delegated authority to make decisions within 48 hours (instead of 6 months) about foundational changes in the direction of a program. He added that the dynamic capability to “re-aim” a program’s focus is invaluable—the limitations of lengthy requirements lead to the death of innovation. He wondered if a cross-sectional approach has been attempted yet at the federal level. Hassell explained that a few similar interagency groups exist, such as the Public Health Emergency Medical Countermeasures Enterprise. However, owing to government constraints, this group still has to make its recommendation at the senior bureaucratic level.

Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×

PLANNING COMMITTEE MEMBERS Frederick R. Chang (Chair), Southern Methodist University; Michael A. Marletta (Vice Chair), University of California, Berkeley; Lilian Alessa, University of Idaho; Tomas Diaz de la Rubia, University of Oklahoma; Vishva M. Dixit, Genentech; Donald D. Duncan, Johns Hopkins University Applied Physics Laboratory; Gerald L. Epstein, National Defense University; Kathleen Fisher, Tufts University; James R. Gosler, Johns Hopkins University Applied Physics Laboratory; Laura M. Haas, University of Massachusetts Amherst; Robert F. Hale, Center for a New American Security; Daniel E. Hastings, Massachusetts Institute of Technology; Frances S. Ligler, Texas A&M University; Willie E. May, Morgan State University; Bernard S. Meyerson, IBM Corporation; Lisa J. Porter, LogiQ; Peter Schiffer, Yale University; Anthony J. Vinci, Center for a New American Security; Michael S. Witherell, Lawrence Berkeley, National Laboratory.

STAFF Dionna Ali, Associate Program Officer; Anita Eisenstadt, Program Officer; Shenae Bradley, Administrative Assistant; Anthony Fainberg, Senior Program Officer; Michael Niles, Senior Program Officer; Nia Johnson, Program Officer; Alan H. Shaw, Director.

DISCLAIMER This Proceedings of Workshop—in Brief was prepared by Linda Casola as a factual summary of what occurred at the workshop. The statements made are those of the rapporteur or individual workshop participants and do not necessarily represent the views of all workshop participants; the planning committee; or the National Academies of Sciences, Engineering, and Medicine.

REVIEWERS To ensure that it meets institutional standards for quality and objectivity, this Proceedings of a Workshop—in Brief was reviewed by Dewey Murdick, Georgetown University, Eliahu Niewood, The MITRE Corporation, and Peter Schiffer, Yale University.

SPONSORS This workshop was supported by the Office of the Director of National Intelligence.

SUGGESTED CITATION National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. https://doi.org/10.17226/26605.

Division on Engineering and Physical Sciences

Copyright 2022 by the National Academy of Sciences. All rights reserved.

images
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 1
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 2
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 3
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 4
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 5
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 6
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 7
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 8
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 9
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 10
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 11
Suggested Citation:"Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop - in Brief." National Academies of Sciences, Engineering, and Medicine. 2022. Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. doi: 10.17226/26605.
×
Page 12
Leveraging the Future Research and Development Ecosystem for the Intelligence Community by the U.S. Research Community: Proceedings of a Workshop—in Brief Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

On April 19, 2021, the Intelligence Community Studies Board of the National Academies of Sciences, Engineering, and Medicine convened a virtual workshop on behalf of the Office of the Director of National Intelligence. This unclassified workshop was the first of two data-gathering workshops.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!