At the request of the U.S. Army, on August 5-7, 2019, the National Academies of Sciences, Engineering, and Medicine’s Panel on Review of Extramural Basic Research at the Army Research Laboratory met to review the programs of the Physical Sciences Directorate (PSD) of the Army Research Office (ARO), which is an organizational unit within the Army Research Laboratory (ARL). The meeting was held at the ARO headquarters in Durham, North Carolina.
The panel’s review was guided by the following statement of task provided by the National Academies:
An ad hoc committee to be named the Panel on Review of Extramural Basic Research at the Army Research Laboratory, to be overseen by the Laboratory Assessments Board (LAB) of the Division on Engineering and Physical Sciences, will be appointed to provide annual assessments of the Army Research Office (ARO) programs. Each year one of the ARO’s three divisions (Information Sciences, Physical Sciences, and Engineering Sciences) will be assessed by a separately appointed panel. These assessments will address criteria to be defined by the ARO. Each year the panel will provide a report summarizing its findings, conclusions, and recommendations. The panel’s report will be made available to the public on the National Academies Press website and will be disseminated in accordance with National Academies policies.
The current report summarizes the 2019 findings of the Panel on Review of Extramural Basic Research at the Army Research Laboratory, which reviewed the programs at the ARO PSD. This is the first time that the National Academies is reviewing ARO’s PSD. In 2020, the National Academies is scheduled to conduct a review of the ARO Engineering Sciences Directorate programs in mechanical sciences, electronics, materials science, and earth sciences. In 2018, the National Academies conducted a review of the ARO Information Sciences Directorate programs in computing science, network science, and mathematical sciences.
The Army Research Laboratory’s ARO describes its mission as:1
To serve as the Army’s principal extramural basic research agency in the engineering, physical, information and life sciences; developing and exploiting innovative advances to ensure the Nation’s technological superiority. Basic research proposals from educational institutions, nonprofit organizations, and private industry are competitively selected and funded. ARO’s research mission represents the most long-range Army view for changes in its technology. ARO’s research represents the most long-range Army view, with system applications often 20-30 years away. ARO priorities
fully integrate Army-wide, long-range planning for research, development, and acquisition. ARO executes its mission through conduct of an aggressive basic science research program on behalf of the Army so that cutting-edge scientific discoveries and the general store of scientific knowledge will be optimally used to develop and improve weapons systems that establish land force dominance. The ARO research program consists principally of extramural academic research efforts consisting of single investigator efforts, university-affiliated research centers, and specially tailored outreach programs. Each approach has its own objectives and set of advantages. Programs are formulated in consultation with the Army Research Laboratory Directorates; the U.S. Army Combat Capabilities Development Command (CCDC) Research Centers; the Army Medical Research and Materiel Command; the Army Corps of Engineers; and the Army Research Institute for the Behavioral and Social Sciences. The programs are also jointly coordinated and planned through the Defense Science and Technology Reliance process under the Basic Research Panel.
Research in the physical sciences is focused on basic research to discover, understand, and exploit physical, chemical, and biological phenomena. This research is of a fundamental nature; however, in the long term, discoveries in this area are expected to lead to revolutionary capabilities in sensing, communications, protection, wound healing, power/energy storage and generation, and materials that extend the performance of Army systems well beyond current limits.2 The PSD’s programs are organized in three divisions: Physics (fiscal year [FY] funding of $26.7 million), Chemical Sciences (FY funding of $54.6 million), and Life Sciences (FY funding of about $82.3 million). Across the three divisions, in FY 2018, PSD funded 472 projects with a budget of $156.7 million—$26.2 million core funding and $130.5 million leveraged funding from sources that include the Multidisciplinary University Research Initiative (MURI) programs, the Presidential Early Career Awards for Scientists and Engineers, the Defense University Research Instrumentation Program (DURIP), the Minority Institutions Program, the Small Business Innovation Research (SBIR) program, the Small Business Technology Transfer (STTR) program, and other Army and Department of Defense (DoD) sources.
In general, the PSD’s metrics are strong, with 2,120 peer-reviewed publications in the FY 2016 to 2018 period, and funding for 1,034 graduate students and 511 postdoctoral researchers during the FY 2017 to 2018 period. However, most impressive for PSD focused on outcomes for the Army was the number of successful transitions from bench to application. There were 104 transitions reported for the 3-year period from FY 2016 to 2018, including the development of several commercial products and start-ups based on the science and technology supported by PSD. The transition of fundamental physical science research funded by ARO to applications developed in the ARL intramural laboratories is another good indicator of the success of PSD.
The panel consisted of 25 leading scientists and engineers whose expertise matched the programs at the ARO’s PSD that were reviewed. All panel members were volunteers who participated without compensation. The panel members’ independence is ensured by the National Academies using its rigorous vetting and approval process for appointment to its panels. The entire panel attended overview presentations by, and held discussions with, the directors of ARL, ARO, and PSD. The panel members then divided into three teams that separately attended presentations by and discussions with program managers (PMs) in the three PSD divisions (Physics, Chemical Sciences, and Life Sciences). The presentations and discussions occurred over a 2-day period. On the third day of the meeting, the panel assembled to share findings from the team reviews, develop impressions common across the team reviews, and prepare the panel’s report draft. On the afternoon of the third day, the panel met with ARO staff for wrap-up discussions to seek clarification of factual and contextual understandings.
The panel members prepared written summaries of their findings, conclusions, and recommendations, which were iteratively reviewed by the panel and formed the basis for the draft report that was subsequently developed under the guidance of the National Academies Army Research Laboratory Technical Assessment Board (ARLTAB), which focused particularly on the panel’s approach to the review and the report’s recommendations. ARLTAB consists of the chairs of the panels that review the scientific and technical work of all ARL directorates, including those at ARO.
After the panel addressed the comments offered by ARLTAB, the report was edited by professional editors at the National Academies and submitted to the National Academies Report Review Committee (RRC). The RRC appointed a team of reviewers to examine the report, considering such factors as the scope of the panel’s task, the reasonableness of the panel’s recommendations, and the clarity of the panel’s expression. Once the RRC reviewers’ comments were adequately addressed, the report was submitted to the Army for security review. After the report cleared the security review, it was publicly posted on the National Academies Press website (www.nap.edu).
The panel applied a largely qualitative rather than quantitative approach to the assessment. The approach of the panel relied on the experience, technical knowledge, and expertise of its members, whose backgrounds were carefully matched to the core technical competency areas in which ARL and ARO activities are conducted. The panel reviewed selected examples of the scientific and technological research programs at the ARO’s PSD because it was not possible to review all PSD programs and projects exhaustively in the time allotted. ARO selected the programs and projects as representative examples in its portfolio that were presented for review. Given the necessarily nonexhaustive nature of the review process, the omission of mention of any particular program or project should not be interpreted as a negative reflection on that program or project. Similarly, recommendations for some programs but none for others should not be read to imply that those programs are of lower quality or have more operational challenges than the other programs. Thus, some of the report chapters of the PSD divisions may have recommendations but not others.
The panel’s goal was to provide an overall impression of the ARO programs in physical sciences while preserving useful mention of suggestions specific to programs that the panel considered to be of special note within the set of those examined. Therefore, the panel strove to identify and report salient examples that supported discussion of accomplishments and opportunities for further improvement with respect to the PSD’s programs.
The panel was charged to apply the following criteria during the review:
- Overall scientific quality and degree of innovation. Was there a clear and cogent strategy regarding how each of the program managers’ major objectives are likely to make substantial and unique progress in advancing scientific frontiers of their discipline? Is the research novel, leading the field in an important area, and does it have the appropriate level of risk and payoff? Was related research being sponsored by other major players adequately summarized in terms of approach and goals? Were there areas of duplication?
- Scientific opportunity. Is there some reasonable basis (e.g., incipient breakthrough, new understanding, novel theory, etc.) to believe that the scientific objectives might be met? Have the highest priority objectives been selected?
- Accomplishments. Did the accomplishments represent significant scientific advances? If not, what is the potential that the accomplishments will lead to significant scientific advances? How do the accomplishments map to the stated program goals? Do the accomplishments reflect productivity and ingenuity on the part of the performers?
- Relevance/transitions. Is the potential, long-term Army application of the research significant? Were there appropriate examples of significant transitions, or anticipated transitions of research,
to follow on applied research or exploratory development either within industry or within an Army or DoD laboratory?
- What were the particular strengths in the program, and what were the weaknesses, if any?
- If there were notable weaknesses, what are suggestions for improvements in these areas?
- Are there any high-priority missed opportunities/areas?
- If so, what lower priority area(s) should be reduced or eliminated to accommodate the new area?
The panel was instructed that the following items are outside the scope of the panel’s charge and that these items should not be considered in the assessment:
- Other divisions or offices within ARO and ARL. The panel is charged in 2019 only to assess the Physical Sciences Division of ARO.
- Organizational changes. The ARO organizational structure is not subject to the assessment.
- Employee morale or motivation. The assessment panel does not conduct scientific surveys nor analyze the data required to assess morale, and is not asked to do so.
- Funding. The panel is not asked to assess or recommend the amount or sources of ARO funding.
This chapter discusses the process used to conduct the assessment and report the resulting findings, conclusions, and recommendations. Chapters 2 through 4 provide assessments of the programs within each of the PSD divisions (Physics, Chemical Sciences, and Life Sciences). Chapter 5 presents findings common across two or more of the divisions.