11

Introduction

At the request of the U.S. Army, the National Academies of Sciences, Engineering, and Medicine formed the Panel on Engineering Sciences at the Army Research Office, which met virtually on August 3-5, 2020, to review the programs of the Engineering Sciences Directorate (ESD) of the Army Research Office (ARO), which is an organizational unit within the Combat Capabilities Development Command (CCDC) Army Research Laboratory (ARL) of the U.S. Army Futures Command (AFC).

The panel’s review was guided by the following statement of task provided by the National Academies:

This part of the report summarizes the 2020 findings of the Panel on Engineering Sciences at the Army Research Office, which reviewed the programs at the ARO’s ESD. This is the first time that the National Academies is reviewing ARO’s ESD programs in electronics, materials science, and mechanical sciences. In 2019, the National Academies conducted a review of the ARO’s Physical Sciences Directorate’s programs in physics, chemical sciences, and life sciences. In 2018, the National Academies conducted a review of the ARO’s Information Sciences Directorate’s programs in computing sciences, network sciences, and mathematical sciences.

PROGRAMS WITHIN THE ENGINEERING SCIENCES DIRECTORATE

The Army Research Laboratory’s ARO describes its mission as follows:¹

___________________

¹2019 ARO in Review, U.S. Army, Combat Capabilities Development Command (CCDC)—Army Research Laboratory, Army Research Office (ARO), Research Triangle Park, North Carolina.

The Engineering Sciences Directorate (ESD) is focused on basic research to harness high-risk discoveries in electronics, materials science, mechanical sciences, and earth sciences. In the long term, fundamental discoveries in these areas are expected to initiate unprecedented and disruptive capabilities in protection, mobility, sensing, computing, propulsion, networks, manufacturing and sustainment to ensure the future technological superiority of our warfighters and Army.³ ESD’s programs are organized into three divisions: Electronics (fiscal year [FY] 2019 funding of $32.3 million), Materials Science (FY 2019 funding of $32.2 million), and Mechanical Sciences (FY 2019 funding of about $21.9 million).

In general, ESD’s metrics are strong, with 2,038 peer-reviewed publications in the FY 2017 to FY 2019 period, and funding for 781 graduate students per year and 326 postdoctoral researchers per year during the FY 2017 to FY 2019 period. There were 133 transitions reported for the 3-year period from FY 2017 to FY 2019. The transition of fundamental physical science research funded by ARO to applications developed in the ARL intramural laboratories is another good indicator of the success of ESD.

APPROACH TO THE ASSESSMENT

The panel consisted of 21 leading scientists and engineers whose expertise matched the programs at the ARO’s ESD that were reviewed. All panel members were volunteers who participated without compensation. The panel members’ independence is ensured by the National Academies, using its rigorous vetting and approval process for appointment to its panels. The entire panel attended overview presentations by, and held discussions with, the directors of ARL, ARO, and ESD. The panel members then divided into three teams that separately attended presentations by and discussions with program managers (PMs) in the three ESD divisions (Electronics, Materials Science, and Mechanical Sciences). The presentations and discussions occurred over a 2-day period. On the third day of the meeting, the panel assembled to share findings from the team reviews, develop impressions common across the team reviews, and prepare the panel’s report draft. On the afternoon of the third day, the panel met with ARO staff for wrap-up discussions to seek clarification of factual and contextual understandings.

The panel members prepared written summaries of their findings, conclusions, and recommendations, which were iteratively reviewed by the panel and formed the basis for the draft report that was subsequently developed under the guidance of the National Academies Army Research Laboratory Technical Assessment Board (ARLTAB), which focused particularly on the panel’s approach to the review and the report’s recommendations. ARLTAB consists of the chairs of the panels that review the scientific and technical work of all ARL directorates, including those at ARO.

___________________

² Basic research is defined by the Department of Defense (DoD) as “systematic study directed toward greater knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications toward processes or products in mind,” while applied research “is a systematic expansion and application of knowledge to develop useful materials, devices, and systems or methods” (DoD 7000.14-R Volume 2B, Chapter 5, 2017). Basic research drives directed studies toward revolutionary discoveries that will lead (and have led) to groundbreaking new capabilities for the Army in the time frame of 30 years and beyond, whereas applied research focuses on the near-term realization of new or improved technologies to meet a specific need.

³ Army Research Laboratory, Army Research Office, https://www.arl.army.mil/who-we-are/aro/army-researchoffice-directorates/, accessed October 3, 2020.

After the panel addressed the comments offered by ARLTAB, the report was edited by professional editors at the National Academies and submitted to the National Academies Report Review Committee (RRC). The RRC appointed a team of reviewers to examine the report, considering such factors as the scope of the panel’s task, the reasonableness of the panel’s recommendations, and the clarity of the panel’s expression. Once the RRC reviewers’ comments were adequately addressed, the report was submitted to the Army for security review. After the report cleared the security review, it was publicly posted on the National Academies Press website (www.nap.edu).

The panel applied a largely qualitative rather than quantitative approach to the assessment. The approach of the panel relied on the experience, technical knowledge, and expertise of its members, whose backgrounds were carefully matched to the core technical competency areas in which ARL and ARO activities are conducted. The panel reviewed selected examples of the scientific and technological research programs at the ARO’s ESD because it was not possible to review all ESD programs and projects exhaustively in the time allotted. ARO selected the programs and projects as representative examples in its portfolio that were presented for review. Given the necessarily nonexhaustive nature of the review process, the omission of mention of any particular program or project should not be interpreted as a negative reflection on that program or project. Similarly, recommendations for some programs but none for others should not be read to imply that those programs are of lower quality or have more operational challenges than the other programs. Thus, some of the report chapters of the ESD divisions may have recommendations but not others.

The panel’s goal was to provide an overall impression of the ARO programs in engineering sciences while preserving useful mention of suggestions specific to programs that the panel considered to be of special note within the set of those examined. Therefore, the panel strove to identify and report salient examples that supported discussion of accomplishments and opportunities for further improvement with respect to the ESD’s programs.

ASSESSMENT CRITERIA

The panel was charged to apply the following criteria during the review:

1. Overall scientific quality and degree of innovation: What are the most effective aspects of each program manager’s planned strategy to make substantial and unique progress in advancing scientific frontiers? How could each program manager’s strategy be further enhanced? How effectively is each program manager executing his or her strategy? What are the most significant high-payoff scientific discoveries described?
2. Scientific opportunity: What are the most compelling opportunities (e.g., incipient breakthrough, new understanding, novel theory, etc.) for future novel high-payoff scientific discoveries? How are unique interdisciplinary opportunities being identified and supported? How agile is the program? What other areas or breakthroughs should the program manager should be monitoring? Are there other partnerships that could help expand the program manager’s network?

Additional possible assessment criteria include the following:

1. Significant accomplishments: How do the program accomplishments represent significant scientific advances? How did the program manager play a significant role? How has the program manager led or guided the scientific community? What is the level of productivity and ingenuity of the performers?
2. Partnerships and transitions: How well is the program manager leveraging coordination, collaborations, and partnerships with ARL in-house research, the Army, and the Federal Research Enterprise? What were the most significant examples of transitions, or anticipated transitions, of funded research?

3. Level of effort: How effectively is the program manager utilizing the available programs and funding sources to make significant progress toward his or her strategy?
4. Other: What were the particular strengths in the program and what were the weaknesses, if any? If there were notable weaknesses, what would be some suggestions for improvements in these areas? Specifically, are there any high-priority missed opportunities/areas that require new or additional funding? If so, what lower priority area(s) should be reduced or eliminated to accommodate the new area? Also, are there any efforts that are insufficient for any reason (e.g., marginal scientific quality, marginal degree of innovation, redundancy, insufficient partnerships, subcritical funding, etc.) and should be phased out?

Specifically excluded from the assessment criteria are the relevance to, and impact of, the scientific and technical work with respect to Army missions.

Also, the panel was instructed that the following items are outside the scope of the panel’s charge and that these items should not be considered in the assessment:

1. Other divisions or offices within ARO and ARL: The panel is charged in 2020 only to assess the Engineering Sciences Directorate of ARO.
2. Organizational changes: The ARO organizational structure is not subject to the assessment.
3. Employee morale or motivation: The assessment panel does not conduct scientific surveys nor analyze the data required to assess morale, and is not asked to do so.
4. Funding: The panel is not asked to assess or recommend the amount or sources of ARO funding.

PART III CONTENT

This chapter discusses the process used to conduct the assessment and report the resulting findings, conclusions, and recommendations. Part III Chapters 12 through 14 provide assessments of the programs within each of the ESD divisions (Electronics, Materials Science, and Mechanical Sciences). Chapter 15 presents findings common across two or more of the divisions.