1
Summary of Workshop Presentations and Discussions
ADVANCING THE FUTURE STATE OF DIGITAL MODEL-CENTRIC ENGINEERING WITHIN THE DEPARTMENT OF DEFENSE
Kristen Baldwin, Principal Deputy,
Deputy Assistant Secretary of Defense for Systems Engineering,
Department of Defense
To frame her presentation, Kristen Baldwin shared that her goal was to connect the way the Department of Defense (DoD) views advanced technologies in manufacturing and in materials science with the way that complex systems are engineered. She planned to discuss DoD’s current implementation of systems engineering as well as their future interests.
Baldwin presented the Office of the Undersecretary of Defense (OUSD) Acquisition, Technology, and Logistics (AT&L) Systems Engineering (SE) organization chart to show the depth of the workforce. As of 2015, DoD employs 108,000 military uniformed and civilian engineers, making it one of the world’s largest engineering organizations. Robert Gold is responsible for the engineering enterprise and promotes and advances policy, guidance, best practices, and specialty disciplines, especially manufacturing. Baldwin emphasized the importance of system producibility and emerging disciplines such as security, anti-tamper and anti-counterfeit practices, value engineering, and engineering for affordability. James Thompson provides the program support and engagement for this organization, which includes approximately 180 major defense acquisition programs.
According to Baldwin, this structure allows DoD to bring to bear engineering and technical expertise to its defense acquisition programs. Programs are assisted on an individual basis, but the structure of the department also allows systemic root cause analysis: Baldwin explained that program-wide challenges can be identified more readily, and relevant policies, guidance, and best practices can be implemented accordingly. Further, engineering tools and environments can be implemented into program support assessment. This “full-circle” approach continues to modernize DoD’s engineering capacity, Baldwin asserted.
According to Baldwin, the primary goal of DoD SE is to advance engineering tools and environments in order to best use modern technology. Phil Zimmerman, Deputy Director of Engineering Tools and Environments, focuses on three areas of the digital approach to engineering: digital engineering design, Engineered Resilient Systems (ERS), and modular open systems approaches. Baldwin explained that many questions are currently under discussion regarding the development of modular open systems approaches, including the following: How does DoD contract for these projects? How does DoD acquire the data rights to enable the projects? The first plan of attack, she said, is to build a technical standard baseline and then to adopt those standards for interfaces and definitions for what the terms “modular” and “open” mean.
As DoD moves toward embracing model-based systems engineering, Baldwin explained, it is important to understand the myths and misconceptions about the area. First, Baldwin noted, the concept of using models is not new: Models are now simply evolving from a secondary role to a primary focus. Second, documents will still be used, but they will now be generated directly from models or influenced by models. Third, engineers need to be able to understand and use many types of models (e.g., algorithm models, data models, system models, etc.). Last, she emphasized that Systems Modeling Language (SysML) is only one of many ways to represent model content, and DoD does not promote a certain type of model or a certain language to discuss the model.
Baldwin stated that DoD is interested in establishing better cross-disciplinary communication, which is an essential part of the systems perspective. The ultimate goal is to integrate the engineering design model with the operational model to better move across broader communities through a common understanding.
Baldwin explained that systems engineers must have a diverse skill set since they work across varied domains (e.g., land, naval, air, communication, reconnaissance, and business systems). DoD systems are meeting current needs but must become more adaptable to better meet future demands that are unknown and unpredictable. Affordability of new systems is an essential parameter, and Baldwin noted that systems engineers must be mindful that these systems are not operating alone and should consider environmental drivers. It is important for DoD engineers to balance agility of systems design with analytic, deep process rigor, she said.
Systems engineers today work in a data-intensive environment. The benefits of such an environment, Baldwin noted, are that more data are available and that greater opportunities to use models to capture and translate that data exist. This enhances integration of technical and nontechnical disciplines, for instance. The drawbacks to such a data-intensive environment are that enemies of the United States have and will use U.S. models and data, which means that they could anticipate U.S. systems and plans. With this threat, Baldwin explained, a critical question arises: How can DoD remain open to performance enhancement while still protecting the systems and the information they contain? As a result, Baldwin noted, systems security engineering is an emerging discipline.
Baldwin shared that acquisition programs and engineering teams are now tasked with developing models and evolving these models over time. The teams consider how program officers, chief engineers, and members of other disciplines use these models. Baldwin described several initiatives that are in place to improve the integration of digital engineering, manufacturing, and modeling in both engineering and acquisition. The Systems Engineering Research Center (SERC)1 is a University-Affiliated Research Center that created a partnership among 22 university systems engineering departments across the country. The partnership has established a 5-year technical plan, which focuses on enterprise systems and systems of systems; trusted systems; systems engineering and systems management transformation (most relevant to the current workshop); and human capital development. SERC participants are given a problem from the DoD systems engineering department, and they research and identify solutions that could be implemented. There is also a digital engineering working group within DoD, led by Zimmerman, that has been tasked with developing a taxonomy to better integrate models and to adapt and develop standards to communicate across disciplines. According to Baldwin, the working group started building this foundation by considering what types of data the acquisition programs and the engineers each collect. Baldwin talked about two related programs: The Computational Research and Engineering Acquisition Tools and Environments (CREATE)2 program explores how to use physics-based modeling that can transition into DoD acquisition, while the ERS3 program works to fill some of the gaps identified in these initiatives.
DoD has additional partnerships outside the department itself, Baldwin explained. For example, the National Defense Industrial Association has a model-
___________________
1 For more information about SERC, see http://www.sercuarc.org/about-serc/about-the-serc/, accessed May 18, 2018.
2 For more information about CREATE, see https://www.computer.org/csdl/mags/cs/2016/01/mcs2016010010.pdf, accessed May 18, 2018.
3 For more information about ERS, see http://defenseinnovationmarketplace.mil/resources/ERS_Overview_20130228.pdf, accessed May 18, 2018.
ing and simulation working group that is helping DoD to develop leading indicators for its own modeling processes. There is also an interagency working group (National Institute of Standards and Technology, National Aeronautics and Space Administration, Department of Homeland Security, Federal Aviation Administration, and National Science Foundation) that meets once per quarter to discuss challenges in complex engineering system design, such as model-based systems engineering and digital modeling.
Currently, DoD has many stovepiped data sources: requirements, science, systems, subsystems, assemblies, test, and operations, for example. Each group generates and maintains its own data, Baldwin explained. Though it will certainly be a challenge, DoD wants to create a digital model-centric environment in which all data are collected in a structured manner, facilitated by a taxonomy, and then the data are matured over time throughout the life cycle. Instead of throwing away data at each step, data would be collected, kept at each phase, and made available to stakeholders through analysis, risk reduction, manufacturing, deployment, and operations phases. According to Baldwin, this will strengthen DoD’s ability to revisit former decisions and make better decisions for the future.
The Reliance 21 framework4 establishes various communities of interest that focus on mission, capability, or technology. Baldwin explained that her group works most closely with the ERS community because of their shared interest in digital modeling. The key goal is to better integrate advanced technologies into the engineering process. Issues of consideration while on the path to achieving this goal center around the use of open architecture, better life cycle intelligence, greater knowledge over time, retained data, and protected security. For Baldwin, the goal is for engineers to optimize: with this new focus on analytic rigor, they should be able to trade materials properties instead of just trading major systems or components.
Despite the fact that there are good modeling practices already in use today, challenges remain for DoD because these modeling practices have not yet been deployed across the whole enterprise. To do this, Baldwin continued, all participants need to be aware of the technical capabilities that exist across DoD and better work across communities. A second challenge is in the acquisition system: How, where, and when does DoD integrate expanded knowledge? A third challenge is balancing two cultures: cost and schedule versus performance. How can cultural barriers and processes of these distinct communities’ objectives be overcome? A fourth challenge relates to the tools that each program uses: Can they be tailored
___________________
4 Reliance 21 is the overarching framework of the Department of Defense (DoD) Science and Technology (S&T) joint planning and coordination process. The goal of Reliance 21 is to ensure that the DoD S&T community provides solutions and advice to the department’s senior-level decision makers, warfighters, Congress, and other stakeholders in the most effective and efficient manner possible.
for particular domains, and can the workforce be trained to use new tools? Last, DoD realizes that overselling methods, processes, and tools is ineffective. Instead, Baldwin said, DoD finds value in informed decision making about modernizing the state of practice and utilizing technology to make the most progress.
Baldwin highlighted DoD’s primary goals moving forward: to continue to leverage members of the diverse community to change the culture and transform to digital model-centric engineering; to make policy changes that will “set the emand signal” for change; and to work to address challenges and enable the workforce with new technologies and practices.
Discussion
To open the question and answer session, Paul Kern, The Cohen Group, stated that the operational test community is resistant to using models and simulation. He explained that this community’s perspective is important because operational testing relates to how people actually use the systems. Baldwin responded that one of the most prominent barriers is trusting in the data. The test community, she explained, wants to demonstrate through live-fire testing, so DoD has created integrated developmental tests and operational tests. So, Baldwin suggested, the question becomes, can time, effort, and cost in testing decrease by relying on upfront test data once models have been validated and data can be trusted? Baldwin elaborated that this also relates to DoD’s goal of reducing the cost and time for the life cycle. The Air Force has already reduced life cycle operations and sustainment costs with better utilization of developmental testing. Maintenance cycles could also be optimized with a greater understanding of early life cycle performance data, Baldwin noted. This would allow groups to be more predictive, especially in the instance of aircraft repair, which could improve cost effectiveness.
Michael F. McGrath, chair of the Defense Materials Manufacturing and Infrastructure standing committee, noted that there is an additional, related barrier: testing and certification of new materials being used in system design. The Defense Advanced Research Projects Agency (DARPA) has a program called Open Manufacturing5 that qualifies processes from a modeling and database perspective so that materials and components can be more easily certified and used. DARPA also has a program called Materials Development for Platforms6 that tries to close the gap between systems engineering and development of the material properties for
___________________
5 For more information about the DARPA Open Manufacturing Program, see http://www.darpa.mil/program/open-manufacturing, accessed May 18, 2018.
6 For more information about the DARPA Materials Development for Platforms Program, see http://www.darpa.mil/program/materials-development-for-platforms, accessed May 18, 2018.
use in the platform. McGrath emphasized that testing at varied scales is relevant to model-based engineering.
Johan de Kleer, Palo Alto Research Center, expressed support of modeling and developing taxonomies, but he cautioned that product development processes often do not keep pace with evolving technologies. De Kleer reinforced that understanding how people fit into this process is essential to this discussion. Baldwin agreed, noting that DoD is working with its engineering centers to establish engineering environments that house varied knowledge management capabilities. For example, the U.S. Army Tank Automotive Research, Development, and Engineering Center7 has developed a more systems-level approach by incorporating tradespace analysis tools with their requirements analysis tools. This allows a focus on the knowledge needed to proceed to the next step in the development process, as well as a generic maturation life cycle to make environments more adaptable. Baldwin stated that DoD realizes the opportunities that exist at the engineering level to promote adaptability.
Denise Swink, private consultant, pointed out that the communities who would benefit most from a systems-level approach will not adapt unless they are convinced that the benefits outweigh the risks and costs. Providing tools to these communities is not enough. Swink asked the following questions: How can these communities be convinced? Will DoD conduct any use cases to show the improvements of using integrated sets of tools and teams? Baldwin responded in the affirmative, stating that it is important to demonstrate the difference that is possible (e.g., decreased cost and time and increased knowledge) before people will be willing to change their processes. DoD has reached out to each of the military services, particularly to programs early in the life cycle and programs that are at critical design review stages, to show how the life cycle can be improved by embedding a more digital approach.
Mike Yukish, Pennsylvania State University Applied Research Laboratory, said he heard a story about the F-117 aircraft shape being defined by the modeling limitations. He asked if there were other examples of this. Baldwin provided an example of a member from her own team who studied analysis of alternative processes where engineers were asked to create nine options. This number has driven many of their future decisions, Baldwin explained, because engineers reside in a product line and have a difficult time seeing beyond it. It is desirable for the engineering community to see things more conceptually instead of focusing too narrowly, such as on only the nine options from nine existing product lines. With such a community, Baldwin noted, innovation is possible.
___________________
7 For more information about TARDEC, see http://www.army.mil/tardec, accessed May 18, 2018.
