5
Barriers to Virtual Design and Manufacturing in DoD Acquisition
The DoD acquisition process is designed to support the programmatic and operational needs for a broad and rapidly evolving material infrastructure, which in turn supports national defense and the warfighter. The DoD acquisition programs cover platforms, weapons, and command, control, communications, computers, intelligence, surveillance, and reconnaissance (C4ISR). Utilization of modeling and simulation (M&S) through all phases of the acquisition process can:
-
enhance the DoD's ability to field transformational capabilities;
-
reduce concept to product cycle time to field a military capacity;
-
archive technology investments for support of current programs and facilitate rapid response to new application requirements;
-
enable broad component, system, and system-of-system optimization via large design parameter space during technology exploration and concept development.
NEED FOR DEFINITION AND MANAGEMENT OF REQUIREMENTS
In April of 2003 Secretary of Defense Rumsfeld issued the "Transformation Planning Guidance" document describing the goals of transforming the DoD into a faster and more flexible force capable of meeting the challenges of fighting new enemies and engaging new targets in new ways.1 Transformation of capabilities within the DoD therefore requires a faster and more flexible capability-based acquisition system. Modeling and simulation provide a key enabler to integrating the research and development community, defense contractors, DoD acquisition offices, and the testing community to achieve the transformational changes envisioned in Secretary Rumsfeld's future DoD.
Therefore, effective utilization of modeling and simulation through all phases of the DoD acquisition process encompasses a compelling opportunity to embrace the simulation-based acquisition (SBA) policy, adopted by the DoD since its articulation by the Acquisition Council of the Executive Council on Modeling and Simulation in 1997, and the Simulation Test and Evaluation Program (STEP) program issued by the DoD Director of Test, System Engineering and Evaluation in the same year.2 The various phases of acquisition are:
1 |
U.S. Department of Defense, Donald H. Rumsfeld, Secretary of Defense, "Transformational Planning Guidance," April, 2003. Available at: http://www.oft.osd.mil/index.cfm. Accessed May 2004. |
2 |
A.L. Hillegas and J.F. O'Bryon, "Modeling and Simulation in the Acquisition Process," ITEA Journal, March/April, |
-
preconcept design and technology development;
-
manufacturing and supply system design;
-
requirement definition and analysis;
-
system development and demonstration (including system integration);
-
live-fire test and evaluation (LFT&E);
-
effectiveness testing and analysis;
-
system logistics;
-
system interoperability;
-
production;
-
deployment;
-
sustainment;
-
upgrades and refitting;
-
end of life—decommissioning and disposal.
The multiple steps of the acquisition process are schematically shown in Figure 5-1. Several DoD-sponsored studies within the last few years have examined these factors that limit more effective and timely utilization of modeling and simulation in defense acquisition. The 2002 NRC study Modeling and Simulation in Manufacturing and Defense Systems Acquisition—Pathways to Success shares a particularly significant synergy with the focus of this report.3 The recommendations of that study in regard to modeling and simulation in acquisition identified the importance of (1) "building the right thing," (2) development of guidelines concerning model, simulation, algorithm, and data ownership to enhance collaboration and facilitate reuse, (3) effort to define how modeling and simulation is to be integrated into DoD acquisition including use of simulation support plans, (4) incentives for managers to adopt best practices for the use of modeling and simulation, and (5) development of pilot efforts sponsored by the OSD to advance the use of modeling and simulation.
The findings of the current report are based upon extensive insights from DoD programmatic experts and industrial leaders4-7 and show a strong correlation with the recommendations of the 2002 NRC study.8 Several of the topics identified in the current study directly mirror those of the 2002 NRC study, specifically the importance of developing and implementing simulation support plans (SSPs) to assist requirement definition. The development of incentives for program managers to adopt SBA, SSPs, and utilization of
|
pp. 25-29, 2001; E.A. Seglie, "The Future of Test and Evaluation," ITEA Journal, September/October, pp. 21-28, 2002. |
modeling and simulation throughout the acquisition life-cycle process remains a recommendation that the current study also strongly endorses.
The rapidly evolving field of virtual modeling and simulation has the potential to improve the DoD acquisition process significantly. Performance, cost, and product quality are identified in this report as central to any design, manufacture, and life-cycle support dominated by early program requirement definition and analysis. Timely establishment of firm technical program scope and mission requirements not only is crucial to avoiding unnecessary acquisition costs and reducing risk, but also is the crucial step preceding the follow-on nested stages of design, testing, and manufacturing necessary to effectively address all the phases of acquisition. The importance of timely requirements definition cannot be overstated. The use of modeling and simulation to guide the establishment of realistic requirements that all the relevant communities—design, manufacturing, performance—can agree on is essential to minimizing requirements drift and reducing both production cost and time to deployment. A recent example of the effects of failure to establish firm requirements definitions was given by the Assistant Secretary of the Air Force for Acquisition, Marvin R. Sambur, in regard to the F/A-22 Raptor
Program Review.9 He stated regarding the review that "unstable requirements, faulty cost estimates, lack of test community buy-in, inadequate systems engineering, and unstable funding" were linked to the Raptor program's problems with meeting cost and schedule targets.
NEED FOR BUILDING LINKAGES ACROSS ALL PHASES OF DOD ACQUISITION
Realization of a new acquisition paradigm for future DoD procurements that will optimize performance, cost-effectiveness, and time to actual military capability (which is not the same as a formal declaration of an initial operational capability or deployment) requires the use of modeling and simulation in each of the sequential phases of acquisition. These must encompass concept initiation, technology demonstration, subsystem design, manufacture and supply chain design trade-offs, subsystem performance analysis, subsystem performance assessment, system integration and logistics, live-fire test and evaluation (LFT&E), interoperability, manufacture, system deployment, maintenance, upgrading and refitting, and finally, decommissioning and disposal. Each stage in this sequence plays a key role in enabling military systems to meet performance goals, cost and schedule goals, and, therefore, the warfighters' needs. Each acquisition stage builds on and is dependent on the foundation of the acquisition step and technical analysis and risk assessment of the stage directly preceding it to expand design options while simultaneously decreasing time to field capability and to reduce the risk of implementation. Figure 5-2 reflects this hierarchy and the additive nature of the types of modeling and simulation critical to making the technical decisions to support development and thereafter acquisition.
As reflected in this schematic pyramid diagram, modeling and simulation tools utilized in the development of a new military system span many levels of fidelity. At one end are the physics-based codes used to design trade-off studies and CADCAM tools for assessing the manufacturability of the proposed designs. At the top of the pyramid are simulation tools essential to establishment of military worth (system effectiveness) and determination of how the system will be used by the warfighter (concept of operations). Across the range of modeling and simulation embodied in the pyramid structure are not only different tools but also different communities that develop each level's tools. Traditional barriers between levels and lack of transmissibility of data between levels remain one of the chief barriers to more robust utilization of modeling and simulation to positively affect DoD acquisition. While there are natural linkages between each of these levels, two principal barriers exist. First, fundamental modeling requirements are often lost in enthusiasm over information technology (IT) advances; i.e., you cannot develop and implement predictive models of processes and physics if you don't understand the fundamentals at the lowest level of representation in the pyramid scheme. Second, realization of SBA relies on efficient and timely connectivity between each of the hierarchical levels of the total DoD life-cycle process; a missing linkage between any of the levels represents a "bridge-to-nowhere" situation, which must be crossed. Herein lies a chief barrier to more effective utilization of modeling and simulation in all the steps of the acquisition process (see Figure 5-1).
Virtual modeling and simulation possesses a unique ability to integrate the technical decision points up this pyramid to establish the military worth of the technology under development and thereafter assess impact logistics and examine system-level interactions.
Lack of Modeling and Simulation Plans in DoD Acquisition Programs
Early implementation of modeling and simulation support plans has positively impacted
acquisition and has helped the program managers (PMs) who use them to better justify the funding for program execution.10 Nevertheless, these support plans are not widely used. Planning for a project's modeling and simulation needs can provide a more realistic assessment of development costs for the necessary resources; can foster more accurate quantification of the benefits of modeling and simulation; and can provide a timely vision of which tools, databases, and validation, verification, and accreditation (VV&A) are needed during each portion of the acquisition process.
Effective plans for modeling and simulation, and timely returns from their utilization, are not free, however. Focused emphasis on these plans is best achieved through integrated product teams (IPTs) and dedicated staff assigned to the acquisition program.11 The benefits of adopting modeling and simulation plans can impact programs in both the short term and the long term, as well as having a broader effect on the DoD. Development of proactive plans allows the PM to engage the DoD service laboratories to leverage existing expertise within the DoD, as well as that available through the Department of Energy (DOE), and also to engage the academic science and technology base. Planning that looks beyond the end of the immediate program can also facilitate reuse of modeling and simulation tools, whether these tools were developed by the prime contractor, within the DoD or DOE technology laboratories, or at universities. Reuse of the data and tools developed for a particular project could provide strong growth of simulation-based acquisition within the DoD.
Lack of Ownership of Models, Simulations, Data, and Databases
In DoD acquisition programs surveyed during the "meet MASTER" exercise, over half of the models and databases in use were developed by industry, and industry retained 40 percent of this total following program completion.12 The high rate of retention was directly linked to industry viewing these models, data, and databases as proprietary. This opinion is in part a reflection of corporate investments in internal research and development (IR&D) in a spectrum of design, performance, manufacturing, and life-cycle areas including the following:
-
CADCAM coupled models
-
Manufacturing automation and control models
-
Performance models and simulations
-
Physics-based performance models and their associated databases
-
Subsystem integration and assembly models
-
Verification and validation test problems
-
System analysis and performance models
-
Hardware-in-the-loop (HITL)
-
Man-in-the-loop (MITL)
-
System interoperability
Need for an Integrated Perspective on Modeling and Simulation
As illustrated in Figure 5-2, a nested set of modeling and simulation capabilities is required to accomplish SBA from concept, to manufacture, to system performance, to
deployment. As the level of complexity of a program increases, modeling and simulation offer a practical means to examine, quantify, and exercise all portions of a system in an organized manner and allow assessment of its capabilities to meet mission objectives. Assessment of the system of systems and the interoperability of systems deployed is an area that particularly benefits from using virtual modeling and simulation due to the severe financial and operational issues of live testing and validation of these two issues in actual battlefield situations. Nevertheless, integration of modeling and simulation tools to allow bridges from one stage of acquisition to the next remains one of the largest challenges of SBA.
As an illustration, let us examine the development of a munitions system. Physics-based tools, including CAD and CAM, used to examine the design trade-offs in concert with the ability to manufacture the components, reflect in many cases state of the art technology readily available to the DoD contractor infrastructure.Emerging physics and phenomenological modeling approaches are pushing abilities to integrate the previous design and manufacturing knowledge at the subsystem level with subsystem performance metrics in some instances. However, weaponization through buildup of the subsystems to produce a full system with the performance capability to meet field objectives now rests at the fringes of a contractor's modeling and simulation capability in many instances due to the lack of interoperability among tools for creating models and simulations, and the insufficiency of the physics-based tools to accurately assess system integration, analysis, and logistics.13 Due to this limited connectivity and lack of robust, validated tools and databases, further promulgation of the design tools to the
level of accurate assessment of system lethality, collateral damage, and mission planning is greatly hindered if not impossible.
Interoperability is one of the "Grand Challenges" of the DoD acquisition requirements. With ever increasingly sophisticated programs and complex platforms deployed within a mission scenario, use of models and simulations offers one of the few avenues, barring full theater testing and evaluation, to assess program interoperability and performance. Interoperability will continue to increase in importance as connectivity is extended beyond system-of-systems considerations. For example, since September 11, 2001, evolving DoD requirements embrace fully functional interoperability with civilian police and indigenous military personnel as demonstrated in the war in Iraq.
Recommendation 7. Defense Acquisition Processes: The Department of Defense should define best practices for government ownership rights to models, simulations, and data developed during system acquisitions.
Formal guidelines and best practices for transferring models, simulations, and data between the government and its contractors are essential for competitive procurement. Instituting common model access, common model databases, and common document controls will ensure that information generated under government funding is available to multiple program managers.
Incentives for program managers to develop integrated design and manufacturing tools can make simulation-based acquisition become a reality for DoD programs. Well-defined metrics for integration of design and manufacturing can help the program managers use simulation-based acquisition. Metrics that are compatible with different acquisition programs will allow these investments to be leveraged in the future. Also, specifying the modeling and simulation techniques that will be used in the proposal evaluation process, especially the cost structure analysis and affordability models, will facilitate simulation-based acquisition. Integrating the concept-of-operations definition into the modeling and simulation program plans can bring end users into the acquisition process and thus foster a more successful transition to military capability.