National Academies Press: OpenBook
« Previous: 6 Emerging AI Technologies and Future T&E Implications
Suggested Citation:"7 Concluding Thoughts." National Academies of Sciences, Engineering, and Medicine. 2023. Test and Evaluation Challenges in Artificial Intelligence-Enabled Systems for the Department of the Air Force. Washington, DC: The National Academies Press. doi: 10.17226/27092.
×

7

Concluding Thoughts

The committee was tasked with the following questions:

  1. Evaluate and contrast current testing and assessment methods employed by the Department of the Air Force (DAF) and in commercial industry.
  2. Consider examples of artificial intelligence (AI) corruption under operational conditions and against malicious cyberattacks.
  3. Recommend promising areas of science and technology that may lead to improved detection and mitigation of AI corruption.

What was uncovered through investigating these questions was a significant overlap between the answers. For example, examples of AI corruption (question 2) were a common topic of promising areas of science and technology (question 3) and reflected as an issue in the current testing and assessment methods employed by the DAF (question 1).

In structuring this report, the committee organized chapters such that each question was primarily addressed in a specific chapter in the report, and thus the recommendations were primarily reported in those sections. However, each chapter of the report contains relevant findings and recommendations for each of the questions addressed by the committee, so in practice, it was not possible to isolate the questions to individual chapters.

Task 1, “Evaluate and contrast current testing and assessment methods employed by the Department of the Air Force and in commercial industry,” is primarily answered in Chapters 3 and 4. In these chapters, the current testing and

Suggested Citation:"7 Concluding Thoughts." National Academies of Sciences, Engineering, and Medicine. 2023. Test and Evaluation Challenges in Artificial Intelligence-Enabled Systems for the Department of the Air Force. Washington, DC: The National Academies Press. doi: 10.17226/27092.
×

assessment methods found by the committee are described and referenced, and a comparison to best practices in commercial industry are directly covered. However, to fully understand this contrast, findings and recommendations from Chapters 2 and 5 must be also considered.

Task 2, “Consider examples of AI corruption under operational conditions and against malicious cyberattacks,” is primarily addressed in Chapter 5. However, the topic of AI corruption is a primary challenge throughout all of the test and evaluation (T&E) of AI-enabled systems and thus is mentioned throughout the study, especially in Chapter 6.

Task 3, “Recommend promising areas of science and technology that may lead to improved detection and mitigation of AI corruption,” is primarily addressed in Chapter 6. However, AI corruption is under active S&T research and in the spirit of DevSecOps and AIOps, solutions are being deployed as rapidly as they are discovered. Thus, many of the approaches in Chapters 3, 4, and 5 also support findings and recommendations to mitigate AI corruption.

In short, the complexity, interconnection, and coupling of issues throughout T&E with AI-enabled systems will require a reassessment of all T&E policies, processes, and procedures to assure that validation and verification of all systems, not just the AI components, will support the necessities of a dynamic and risky deployment and operational environment. In this case, the committee concluded that while the questions appeared quite straightforward on an initial reading, in fact the DAF has now caught the AI tiger by its tail. Taming that tiger will be challenging, especially as AI-enabled components become commonplace in all platforms and MDAPs, but it is not at all an insurmountable problem. It requires vision, hands-on leadership, prioritization, and a shared commitment to an AI-enabled future DAF.

Suggested Citation:"7 Concluding Thoughts." National Academies of Sciences, Engineering, and Medicine. 2023. Test and Evaluation Challenges in Artificial Intelligence-Enabled Systems for the Department of the Air Force. Washington, DC: The National Academies Press. doi: 10.17226/27092.
×
Page 137
Suggested Citation:"7 Concluding Thoughts." National Academies of Sciences, Engineering, and Medicine. 2023. Test and Evaluation Challenges in Artificial Intelligence-Enabled Systems for the Department of the Air Force. Washington, DC: The National Academies Press. doi: 10.17226/27092.
×
Page 138
Next: Appendixes »
Test and Evaluation Challenges in Artificial Intelligence-Enabled Systems for the Department of the Air Force Get This Book
×
 Test and Evaluation Challenges in Artificial Intelligence-Enabled Systems for the Department of the Air Force
Buy Paperback | $42.00 Buy Ebook | $33.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Department of the Air Force (DAF) is in the early stages of incorporating modern artificial intelligence (AI) technologies into its systems and operations. The integration of AI-enabled capabilities across the DAF will accelerate over the next few years.

At the request of DAF Air and Space Forces, this report examines the Air Force Test Center technical capabilities and capacity to conduct rigorous and objective tests, evaluations, and assessments of AI-enabled systems under operational conditions and against realistic threats. This report explores both the opportunities and challenges inherent in integrating AI at speed and at scale across the DAF.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!