Unclassified Report Summary

Operational test and evaluation (OT&E) is a critical part of U.S. military capability development and essential to the effective employment of new operational systems. OT&E ensures that every operational system meets its operational requirements in a realistic representation of the operational environment. In an era of near-peer strategic competition in the Western Pacific, conflict in Eastern Europe, emerging technologies, increasingly capable adversaries, new competitive operational domains (cyber and space), rapid, rigorous, and realistic OT&E is essential to ensuring competitive advantage for the Joint Force and preserving a credible deterrence.

New battlefield capabilities—in particular, autonomous systems, artificial intelligence (AI), machine learning (ML), cyber weapons, electronic warfare (EW) systems, directed energy weapons, space-based platforms, and hypersonic aircraft and missiles—will define the next near-peer conflict. Preparing the U.S. military will require extensive and realistic operational testing that not only validates new operational systems for the future operating environment, but can also catalyze the Services and Joint Force to assess how well they adapt doctrine, tactics, and training to new paradigms of warfare.

Understanding the critical role of OT&E, the Office of the Director of Operational Test and Evalutation (DOT&E) tasked the National Academies of Sciences, Engineering, and Medicine with conducting two studies, one unclassified (referred to as Phase 1) and one classified (referred to as Phase 2). The goal of the combined effort of two committees was to investigate and assess the state of Department of Defense (DoD) ranges, their capacity to support OT&E in the coming decades, and recommend actions to mitigate and overcome any identified deficiencies. The Phase 1 report committee focused its efforts on the technical and physical infrastructure of DoD ranges. That committee and report,¹ while limited by classification, addressed a wide spectrum of range needs and recommended key steps that DOT&E and the OT&E community writ large can take to prepare to support future operational test (OT). This Phase 2 report picks up where the Phase 1 report ends—expanding on several areas and incorporating threat modeling and replication/representation in OT&E based on briefings, reports, and discussions at the Secret level.

This report is organized around five (5) key themes and recommendations that will prepare the test community to effectively support OT&E in the coming decades.

• Theme 1: Improve threat modeling and prototyping to maintain pace with adversary developments.
• Theme 2: Address gaps in testing of emerging capabilities driven by new technologies.
• Theme 3: Test as You Fight—Testing to operational capability versus program requirements.
• Theme 4: Formalize a live, virtual, and constructive (LVC) test range capability for joint multi-domain OT based on digital engineering and LVC technologies.
• Theme 5: Test at the speed of operational needs.

__________________

The report structure consists of six core chapters detailing the findings, conclusions, and recommendations of the committee.

Chapter 1 summarizes and expands on the Phase 1 report, noting its key themes and recommendations and describes how it lays the foundation for Phase 2. The Phase 1 study examined the physical and technical infrastructure of DoD ranges and infrastructure in an unclassified capacity. The report concluded that the challenges facing the OT&E community are driven by limited test capacity, the age of test infrastructure, the capability to test advanced technologies, and encroachment, which hinder the ability to inform integrated system performance and slow the overall pace of testing. The Phase 1 report emphasized three fundamental themes that included (l) a vision of future combat that will rely on connected kill chains in a joint all-domain operations (JADO) environment; (2) the role of digital technologies in dramatically reshaping the nature, practice, and infrastructure of OT; and (3) the use of “speed-to-field” as today’s measure of operational relevance.

To address these challenges, the Phase 1 study developed conclusions and recommendations around the following five major categories: (1) Develop the “Range of the Future” to test complete kill chains in JADO environments. (2) Restructure the range capability requirements process for continuous modernization and sustainment. (3) Bootstrap a new range operating system for ubiquitous modeling and simulation (M&S) throughout the operational system’s development and test life cycle. (4) Create the test-development-operations (TestDevOps) digital infrastructure for future operational test and seamless range enterprise interoperability. (5) Reinvent the range enterprise funding model for responsiveness, effectiveness, and flexibility.

This Phase 2 report carries forward a number of themes from the Phase 1 report, namely the following: future combat will be built on connected system-of-systems kill chains in JADO environments; digital technologies are dramatically reshaping the nature and infrastructure of testing; and “speed-to-field” is the measure of operational relevance in today’s competitive near-peer environment.

Finally, the Phase 1 report introduced the concept of the Range of the Future that this Phase 2 report builds on to develop its core recommendation—namely, that the future of OT&E will rely on geographically distributed, LVC systems. These systems will need to be capable of testing at scale in environments representative of future JADO.

Chapter 2 examines and assesses the changing threat environment in 2035. It details how scientific and technical intelligence (S&TI) supports OT&E through modeling and threat replication. The committee reviewed and assessed the threat replication capacity across the OT&E community and found that core challenges stemmed primarily from the S&TI process, the lack of consistent resourcing for threat model development, and the long timeframe for threat model release to the OT&E community. The committee’s key recommendations are to adopt threat agnostic models to support OT&E, consistent resourcing of threat model development, and continued S&TI support after Acquisition Milestone C or at key points in alternative acquisition pathways such as rapid prototyping acquisition.

Chapter 3 investigates several emerging technologies and technology areas that will define future conflicts and the unique challenges they pose to OT&E. These technologies include hypersonic systems, space systems, directed energy, 6th-generation aircraft, advanced electronic warfare systems, advanced acoustic and non-acoustic systems for undersea warfare, AI, autonomous systems, and cyber capabilities. The committee divided the technologies into the following two main categories: (l) relatively mature technology and primarily physical systems that will require additional OT&E resources to keep pace with threat conditions and (2) emerging software-based technologies that are cross-cutting, rapidly evolving, and will require OT&E to engage early in the life cycle to adapt their test strategies as the technologies continue to mature. Chapter 3 includes specific findings, conclusions, and recommendations for each technology area.

Chapter 4 focuses on a system-of-systems approach to T&E and the need for the OT&E community to adhere to an oft heard mantra by the committee, “test as you fight,” which is to provide OT plans that align as closely as possible to realistic warfighting. The overwhelmingly consistent message heard by the committee was the challenge of testing to the scale of the future fight. Current and emerging

DoD warfighting doctrines emphasize the need for the Joint Force to operate as a multi-domain or all-domain capable force. The development of mechanized forces in the early 20th century, the emergence of new warfighting domains in the air, undersea, space, and cyberspace throughout the 20th and early 21st centuries, and the rapid development and advancement of communications technologies over the past century have driven integration of warfighting domains and their respective domain-centric platforms into a system-of-systems architecture that fuses warfighting elements into a coherent Joint Force operating in all domains.

This system-of-systems approach and all-domain character of future warfare are not represented in OT&E today. Chapter 4 details this changing character of warfare, the shift it demands of OT, and pockets of excellence found within the Services.

Chapter 5 details the Range of the Future and expands on it in light of new information and subsequent analysis by the Phase 2 committee. This future reality must be accounted for by OT&E in a way that realistically replicates all-domain operations at scale. Notable challenges include the time and resources necessary to conduct physical testing at realistic scales, the potential for adversary intelligence collection on OT, and the need to keep pace with changing doctrine. The solution to this challenge is the development, adoption, and implementation of geographically distributed, but connected, LVC systems.

The committee identifies a number of LVC systems in various states of development and operation, most notably the Joint Simulation Environment developed for the F-35 program, which can serve as the building blocks for the Range of the Future. This encourages the further development of these systems and their proliferation across the Services.

Chapter 5 also identifies key attributes of the LVC range of the future and core considerations that must be taken into account during its development and operation. The DOT&E will play a critical role in championing the development of these systems and in ensuring their capability to connect across the Services and domains to realistically replicate the future near-peer operating environment.

Chapter 6 discusses the need to keep up the pace of innovation to ensure the U.S. military does not lose ground to its adversaries’ technology development curves. U.S. military capability development is an interconnected process moving from R&D, to acquisition programs of record, to developmental testing, to operational testing, and then on through training and fielding and sustainment. The interconnected relationship and reciprocal feedback loops between all of these stages means that a slowdown in one area can affect the progress in development. The committee strongly believes it is absolutely critical that OT&E serves as an accelerator in this process. Through the use of LVC systems, changes in process (detailed in the report), emphasis on improved information sharing, and the adoption of key emerging technologies such as AI, the DOT&E can serve as a key enabler of rapid capability development in this dynamic environment.

The success of OT&E, much like operational military forces, hinges on adaptability and continuous improvement based on the feedback and lessons learned from experience. A future vision of operational testing based on a system-of-systems approach to replicate the realistic employment of the of operational systems across their life cycles can only be achieved through continuously updated threat models, engineering assumptions, and applied real-world experience.

Additional details, including the study’s findings, conclusions, and recommendation, are available in the classified study report.

This page intentionally left blank.