The Intelligence Community Studies Board (ICSB) of the National Academies of Sciences, Engineering, and Medicine convened a workshop on August 9-10, 2017, to examine challenges in machine generation of analytic products1 from multi-source data. With funding from the Office of the Director of National Intelligence (ODNI), the ICSB established a planning committee (see Planning Committee on The Intelligence Machine Analytics Workshop and Appendix A) to develop the workshop agenda (see Appendix B). The workshop statement of task is shown in Appendix C. More than 300 people registered to participate in the workshop either in person or online.
Workshop speakers and participants discussed research challenges related to machine-based methods for generating analytic products and for automating the evaluation of these products, with special attention to learning from small data,2 using multi-source data, adversarial learning,3 and understanding the human–machine relationship. During the presentations and discussion sessions, attendees were asked to address the following questions, with particular emphasis on their role for the intelligence community:
- What are the primary issues?
- What is the research knowledge base?
- What are the current and “next level” of key performance metrics?
- What is the “level after next” of expected research and development performance?
- What are the requisite enabling technologies?
- How can the government best prepare the scientific workforce to enhance discovery in this area?
- What are the technical objectives and metrics needed for success?
1 For the purposes of this report, ODNI defines “machine generation of analytic products” as the use of computer algorithms to quickly and accurately discover, interpret, and relay actionable information or insights to a consumer.
2 Small data are data in low volumes, structured varieties, and batch velocities (See IBM Big Data and Analytics Hub, Infographics & Animations, “Taming Big Data: Small Data vs. Big Data,” http://www.ibmbigdatahub.com/infographic/taming-big-data-small-data-vs-big-data, accessed August 9, 2017).
3 Adversarial learning refers to the intersection of computer security and machine learning ((See IBM Big Data and Analytics Hub, Infographics & Animations, “Taming Big Data: Small Data vs. Big Data,” http://www.ibmbigdatahub.com/infographic/taming-big-data-small-data-vs-big-data, accessed August 9, 2017).
This proceedings is a factual summary of what occurred at the workshop. The planning committee’s role was limited to organizing and convening the workshop. The views contained in this proceedings are those of the individual workshop participants and do not necessarily represent the views of the participants as a whole, the planning committee, or the National Academies of Sciences, Engineering, and Medicine. In addition to the summary provided here, materials related to the workshop, including videos of presentation and discussion sessions, can be found at http://sites.nationalacademies.org/DEPS/icsb/DEPS_180349 or https://vimeo.com/album/4730093.