National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×

Executive Summary

Rigorous operational testing (OT) of weapon systems procured by the U.S. Department of Defense (DoD) is fundamental to ensuring that these sophisticated systems not only meet their stated requirements, but also perform under realistic operational conditions when faced by determined adversaries employing their own highly capable offensive and defensive weaponry. Without adequate OT, operational commanders would be unable to make the most effective use of their capability and warfighters would lack confidence in the weapons they bring to the fight or, worse, may inadvertently put themselves in harm’s way because they do not have a fundamental understanding of their weapons’ capabilities and limitations. DoD’s test and training range enterprise provides the geography, infrastructure, technology, expertise, processes, and management that make safe, secure, and comprehensive OT possible. However, the range enterprise, along with the talented and committed range workforce that makes the system function, is under great stress. Unless prompt action is taken to address both longstanding and emerging challenges, including test capacity, modernization, digital infrastructure, encroachment, and resources, DoD’s ranges will be unable to support timely or adequate OT in the future.

The challenges facing the nation’s range infrastructure are both increasing and accelerating. Limited test capacity in physical resources and workforce, the age of test infrastructure, the capability to test advanced technologies, and encroachment impact the ability to inform system performance, integrated system performance, and the overall pace of testing. Investments in the U.S. test infrastructure and changes in test

Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×

and evaluation (T&E) methodologies and handling of data are necessary to inform the delivery of lethal, survivable, reliable, and affordable weapon systems to the field at a speed that is relevant to the operational need. This study draws on testimony from senior military officers and officials from operational, acquisition, and test backgrounds as well as on test and training experts, leading technologists, leaders from relevant commercial enterprises, and individuals with deep experience in DoD and congressional budget processes. The study committee conducted virtual and physical site visits to a representative sample of test ranges; collected test range inputs on modernization, sustainment, operations, and resource challenges; and reviewed prior studies and reports from the office of the Director of Operational Test and Evaluation (DOT&E), the military service test organizations, and the Test Resource Management Center (TRMC). This report makes a set of interdependent recommendations that the committee believes will put the DoD range enterprise on a modernization trajectory to meet the needs of OT in the years ahead.

The report emphasizes the following three fundamental themes:

  1. Future combat will demand connected kill chains in a Joint All-Domain Operations (JADO) environment. It is critical that DoD architects, specifies, develops, and tests systems to ensure that they are highly effective when fielded in this new reality. DoD acquisition processes, organizational stovepipes, test methods, and range infrastructure that were optimized for the testing of individual weapon systems in single domains will be inadequate to test future integrated weapon systems in the way that they will be operated in machine-speed warfare that crosses all combat domains, including land, sea, air, space, and cyberspace.
  2. Digital technologies are dramatically reshaping the nature, practice, and infrastructure of test. The weapon systems of today and tomorrow are fundamentally enabled by data and software, and DoD test ranges will be no different. The rapidly increasing importance of autonomy, artificial intelligence (AI), and machine learning across defense systems is creating novel challenges for OT. Furthermore, the advent of digital twins and high-performance modeling and simulation (M&S) is enabling new ways of testing, even as combinations of new domains and operational constraints increasingly make virtual testing the only practical approach for certain applications.
  3. Speed-to-field is today’s measure of operational relevance, which is in turn a continuously moving target. Enabled by the global proliferation of many digital, software, and communications-based technologies, U.S. adversaries are rapidly and continuously
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×

    deploying new generations of weapons designed to negate U.S. warfighting advantages. At the same time, new weapon systems are employing never-fielded technologies, which are also evolving at a pace enabled by Moore’s Law. Usable weapon systems are fielded promptly, but there is a need for continuous testing and assessment.

To address the challenges tied to these themes, the committee developed conclusions and recommendations that fall into the following five broad categories:

  1. Develop the “range of the future” to test complete kill chains in JADO environments. It is essential that the range enterprise accommodate new concepts of operation and new test approaches for realistic operational testing, which includes enabling infrastructure for system-of-systems integrated testing and the interoperability of multiple ranges across diverse domains. [Recommendation 3-1]
  2. Restructure the range capability requirements process for continuous modernization and sustainment. Enabling speed-to-field while maintaining the rigor of operational test and evaluation will require rapid range modernization for new weapons technologies and new threats. At the same time, key capabilities need to be sustained and even augmented to ensure required test capacity and throughput, while mitigating the issues caused by encroachment both in the physical and radio frequency environments. [Recommendations 3-2, 3-3, 3-4, 3-5]
  3. Bootstrap a new range operating system for ubiquitous M&S throughout the weapon system development and test life cycle. Many of today’s DoD programs cannot be tested effectively in live testing alone. High-fidelity virtual testing can improve readiness and the likelihood of success for actual hardware testing and may be the only environment to do certain types of tests. However, widespread and standardized use of M&S for operational test will depend on a new M&S infrastructure, significant cultural changes within the test community, and new approaches to the validation of M&S in an ever-changing threat and technology environment. [Recommendation 4-1]
  4. Create the “TestDevOps” digital infrastructure for future operational test and seamless range enterprise interoperability. Redefine the enterprise-supported core digital standards and capabilities for TRMC and test ranges to take advantage of DoD’s scale for software, data, networks, AI, cybersecurity, and M&S.
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×

    Make model-based engineering, the unbroken digital thread, and continuous integration/continuous delivery software practices the foundation for range agility, rapid test evolution, and speed-to-field. Ultra-high-bandwidth information flows must become frictionless, on-demand, and secure. [Recommendations 4-2, 4-3]

  1. Reinvent the range enterprise funding model for responsiveness, effectiveness, and flexibility. The resource needs of today and tomorrow reflect the reality of rapidly changing technology and threats; sustained capital investment for creation, upgrade, and maintenance of long-life range systems; and increasing demands for cross-domain system-of-systems testing with seamlessly integrated M&S. Including DOT&E earlier and continuously in the requirements development and acquisition processes will better establish and certify the timeliness and adequacy of range investments. [Recommendations 5-1, 5-2; Conclusions 5-1, 5-2]

Table ES.1 maps the report recommendations and key conclusions to the themes laid out above.

TABLE ES.1 Report Recommendations and Key Conclusions Mapped to Themes

Theme Recommendations and Key Conclusions
Develop the “range of the future” to test complete kill chains in JADO environments Conclusion 3-1: The lack of a Department of Defense or joint publication set of definitions for multi-domain operations and cyber-physical systems can result in different operational use cases.

Conclusion 3-2: Testing ranges are not optimized for testing end-to-end kill chains; they were not designed for collaborations with other ranges, and they lack the framework and infrastructure to test concurrent and connected kill chains.

Recommendation 3-1: To enable a range of the future that is capable of testing kill chains and multi-domain operations that can integrate effects across National Defense Strategy modernization areas, the Secretary of Defense should address the need to enable Department of Defense ranges to provide regular venues to “test as we fight” for acquisition and prototyping programs in a joint multi-domain battlespace of integrated systems.
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×
Theme Recommendations and Key Conclusions
Restructure the range capability requirements process for continuous modernization and sustainment Conclusion 3-3: Encroachment leads to the inability to demonstrate mission capability and identify deficiencies due to lack of access to the physical and electromagnetic spectrum space with which to conduct test and evaluation. This creates operational risk as DoD will have to field weapon systems that have not been tested against certain threats.

Recommendation 3-2: To ensure the ability to validate the survivability of Department of Defense (DoD) weapon systems against a realistic operational threat environment across air, sea, land, space, and cyberspace domains, DoD should identify and prioritize bands that cover U.S. military operational and test requirements which should be protected from sell-off to preserve these capabilities.

Recommendation 3-3: The Test Resource Management Center (TRMC) should assess current and projected commercial radio frequency communications technologies and spectrum allocations for secure, agile, high-bandwidth operational test needs. In addition, TRMC should determine the feasibility of developing new large-scale enclosed testing facilities combined with expanded modeling and simulation to support electromagnetic spectrum activities not suitable for open-air testing.

Recommendation 3-4: The Department of Defense should broaden the authority of the Test Resource Management Center to address issues of internal encroachment by reviewing internal range policies and actions to ensure that the test groups retain adequate mission space and prevent the placement of equipment or infrastructure that could potentially interfere with test operations. The Director of Defense Research and Engineering for Advanced Capabilities should be granted the authority to mitigate disputes arising over internal encroachment concerns and provided additional funding to manage internal encroachment.

Recommendation 3-5: The Test Resource Management Center should develop a strategy that assesses the use of and potential investment in suitable allied resources for open-air testing. This strategy should include criteria for the usage of allied resources and areas of potential investment to include range space available, data collection, security risks, and support facilities.
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×
Theme Recommendations and Key Conclusions
Bootstrap a new range operating system for ubiquitous M&S throughout the weapon system development and test life cycle Recommendation 4-1: A Department of Defense joint program office should establish a shared, accessible, and secure modeling and simulation (M&S) and data ecosystem to drive development and testing across the life cycles of multiple supporting programs. M&S should be planned from early concept development to support the entire life cycle of the system, from requirements generation, through design development, integration and test, and sustainment. Uncertainty quantification should be employed to identify the primary sources of uncertainty in the understanding of the system being developed and to define an integrated testing and simulation activity to reduce those uncertainties to an acceptable level.
Create the “TestDevOps” digital infrastructure for future operational test and seamless range enterprise interoperability Recommendation 4-2: A Department of Defense joint program office should adopt and promulgate modern approaches for standardization, architectural design, and security efforts to address data interoperability, sharing, and transmission challenges posed by the complexity of next-generation systems. The joint mission office should determine how to develop and maintain a protected data analysis tool and model repository for testing, increase the interconnectivity of test ranges, and ensure the development of data protocols for the real-time transfer of data at multiple classification levels.

Recommendation 4-3: The Test Resource Management Center should continue monitoring and supporting the Assured Development and Operation of Autonomous Systems (ADAS) Project, and prioritize efforts to develop a common set of standards, measurement approaches, and operational scenarios from which to evaluate the performance of artificial intelligence (AI) and autonomous systems, while recognizing that testing approaches may differ between AI and autonomous systems.
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×
Theme Recommendations and Key Conclusions
Reinvent the range enterprise funding model for responsiveness, effectiveness, and flexibility Recommendation 5-1: The Joint Requirements Oversight Council (JROC) should consult regularly with the Director of Operational Test and Evaluation (who is an advisor to the JROC) about the test requirements for systems considered by the JROC. This consultation should include an evaluation of current testing capabilities, facilities shortfalls, and plans to address these shortfalls.

Recommendation 5-2: The Office of the Secretary of Defense should either allow an exemption or set shallower expenditure benchmarks for the first 2 years of test modernization programs. This will reflect realistic expense curves for the technologies and projects needed to test next-generation programs and complex integration.

Conclusion 5-1: New mechanisms and funding limits for applying minor military construction are necessary for responsive test and evaluation activities.

Conclusion 5-2: There exists a need for the Department of Defense to pilot new process and authorities for funding ranges and infrastructure to make them simpler, more responsive, and more effective.
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×
Page 1
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×
Page 2
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×
Page 3
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×
Page 4
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×
Page 5
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×
Page 6
Suggested Citation:"Executive Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight. Washington, DC: The National Academies Press. doi: 10.17226/26181.
×
Page 7
Next: 1 Introduction »
Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight Get This Book
×
 Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems: Testing for the Future Fight
Buy Paperback | $35.00 Buy Ebook | $28.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Rigorous operational testing (OT) of weapon systems procured by the U.S. Department of Defense (DoD) is fundamental to ensuring that these sophisticated systems not only meet their stated requirements, but also perform under realistic operational conditions when faced by determined adversaries employing their own highly capable offensive and defensive weaponry. DoD's test and training range enterprise provides the geography, infrastructure, technology, expertise, processes, and management that make safe, secure, and comprehensive OT possible. The challenges facing the nation's range infrastructure are both increasing and accelerating. Limited test capacity in physical resources and workforce, the age of test infrastructure, the capability to test advanced technologies, and encroachment impact the ability to inform system performance, integrated system performance and the overall pace of testing.

Necessary DoD Range Capabilities to Ensure Operational Superiority of U.S. Defense Systems assesses the physical and technical suitability of DoD test and evaluation ranges, infrastructure, and tools for determining the operational effectiveness, suitability, survivability, and lethality of military systems. This report explores modernization, sustainment, operations, and resource challenges for test and evaluation ranges, and makes recommendations to put the DoD range enterprise on a modernization trajectory to meet the needs of OT in the years ahead.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!