National Academies Press: OpenBook
« Previous: 5 Overcoming Barriers to Wider Use of Automated Research Workflows
Suggested Citation:"6 Conclusion." National Academies of Sciences, Engineering, and Medicine. 2022. Automated Research Workflows For Accelerated Discovery: Closing the Knowledge Discovery Loop. Washington, DC: The National Academies Press. doi: 10.17226/26532.
×
Page136
Suggested Citation:"6 Conclusion." National Academies of Sciences, Engineering, and Medicine. 2022. Automated Research Workflows For Accelerated Discovery: Closing the Knowledge Discovery Loop. Washington, DC: The National Academies Press. doi: 10.17226/26532.
×
Page137
Suggested Citation:"6 Conclusion." National Academies of Sciences, Engineering, and Medicine. 2022. Automated Research Workflows For Accelerated Discovery: Closing the Knowledge Discovery Loop. Washington, DC: The National Academies Press. doi: 10.17226/26532.
×
Page138
Suggested Citation:"6 Conclusion." National Academies of Sciences, Engineering, and Medicine. 2022. Automated Research Workflows For Accelerated Discovery: Closing the Knowledge Discovery Loop. Washington, DC: The National Academies Press. doi: 10.17226/26532.
×
Page139
Suggested Citation:"6 Conclusion." National Academies of Sciences, Engineering, and Medicine. 2022. Automated Research Workflows For Accelerated Discovery: Closing the Knowledge Discovery Loop. Washington, DC: The National Academies Press. doi: 10.17226/26532.
×
Page140
Suggested Citation:"6 Conclusion." National Academies of Sciences, Engineering, and Medicine. 2022. Automated Research Workflows For Accelerated Discovery: Closing the Knowledge Discovery Loop. Washington, DC: The National Academies Press. doi: 10.17226/26532.
×
Page141
Suggested Citation:"6 Conclusion." National Academies of Sciences, Engineering, and Medicine. 2022. Automated Research Workflows For Accelerated Discovery: Closing the Knowledge Discovery Loop. Washington, DC: The National Academies Press. doi: 10.17226/26532.
×
Page142
Suggested Citation:"6 Conclusion." National Academies of Sciences, Engineering, and Medicine. 2022. Automated Research Workflows For Accelerated Discovery: Closing the Knowledge Discovery Loop. Washington, DC: The National Academies Press. doi: 10.17226/26532.
×
Page143

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6 Conclusion The tools and techniques being developed under the large umbrella of automated research workflows (ARWs) promise to collapse the centuries-old serial method of research investigation into processes where thousands or even millions of simulations or experiments are iterated rapidly in closed loops, with the analysis of data and even the design of experiments or controlled observations being assisted by machine learning (ML) or optimization techniques. Simultaneously, ARWs provide a way to satisfy pressing demands across fields to increase interoperability, reproducibility, replicability, and trustworthiness by better tracking results, recording data, establishing provenance, and creating more consistent metadata than even the most dedicated researchers can provide themselves. The committee’s exploration of ARWs illustrates that the research enterprise stands at an important inflection point. The scientific revolution of the 17th century ushered in an unprecedented era of human progress, leading directly to discoveries and innovations that have transformed tasks requiring the application of muscle or simple technologies into services performed by ever more effective machines. The research enterprise will need to develop new approaches and tools as it enters an era in which core elements of knowledge discovery itself can be automated and accelerated. In important ways, this emerging process of innovation and adaptation represents a continuation of the long-standing trend of computational power being harnessed to perform a variety of research tasks. Yet new twists will need to be considered and addressed. Concerns about privacy, ethics, and trust arising in many domains of human activity become even more 136 PREPUBLICATION COPY—Uncorrected Proofs

relevant to the entire research enterprise as we increase use of artificial intelligence (AI)-based technologies. As illustrated in the use cases examined in Chapter 3, different disciplines of research have very different usage patterns relative to ARWs—in terms of specific tools and platforms and, more generally, propensity to incorporate workflows into their processes in the first place. Costs for equipment, software, staffing, and training may vary by discipline, but the broad need for domain researchers to incorporate new methods and approaches holds across the use cases. In addition, additional specialized expertise in areas such as software engineering, algorithm development, and data science will be required in a number of fields. Further, several lines of thought that emerged from the March 2020 workshop are germane not just to the task at hand, but more broadly across the scientific enterprise. These themes include the need to break down academic silos, provide incentives for greater collaboration among researchers, ensure greater interoperability across technologies, foster sharing of a broader range of research outputs, and address issues such as striking an appropriate balance between access to and protection of data. The committee’s findings and recommendations point to promising areas of focus for the research enterprise in facilitating the effective implementation of ARWs. The use cases and supporting literature described in Chapter 3 support all of the recommendations, with Findings A and B and Recommendation 1 in particular flowing directly from examples drawn from a variety of domains. Finding C and Recommendations 2, 3, and 4 are supported in Chapters 4 and 5, which draw on presentations from the March 2020 workshop and other cited literature. Finding C and Recommendation 5 are also supported mainly in Chapter 5, again, with points drawn from the use cases. 137 PREPUBLICATION COPY—Uncorrected Proofs

FINDINGS AND RECOMMENDATIONS Finding A: Accelerating Discovery In many disciplines, the emergence of automated research workflows (ARWs), built upon contemporary cyberinfrastructure, is demonstrating the potential to vastly increase the speed and efficiency of a range of research activities. These include designing and conducting experiments, analyzing data, and observing natural phenomena. These improvements can be realized at scale by implementing infrastructure and practices that facilitate the application of artificial intelligence and machine learning and related technologies to research. Realizing the potential of ARWs could accelerate the pace of scientific discovery by orders of magnitude and thereby expand the research enterprise’s contribution to society. Finding B: Additional Benefits In addition to increasing the speed and efficiency of research, the effective development and implementation of the technical and human infrastructure for automated research workflows (ARWs) will contribute to strengthening the research process in other ways. For example, the greater transparency and repeatability made possible by automating and capturing specific steps in the research process—advances that underlie the development of ARWs—can foster reproducibility, replicability, and responsibility in research. Adoption of common and interoperable tools and platforms—which could be accelerated by the advance of ARWs but depends on other developments as well—can facilitate international and interdisciplinary research collaboration. Broader access to research workflows and results and the enhanced ability to uncover and correct errors can contribute to greater confidence in research findings and 138 PREPUBLICATION COPY—Uncorrected Proofs

the research enterprise and reduce redundancy among research efforts. To be sure, issues such as dealing with large amounts of streaming data and complex computational approaches will continue to pose technical challenges to the design and implementation of ARWs. In addition, incorporating emerging principles and guidelines for responsible artificial intelligence and machine learning advocated by various organizations, such as building in human review of algorithms, uncovering and addressing bias, and supporting transparency and reproducibility, will also help to secure the benefits of ARWs. Recommendation 1: Design Principles Organizations that fund, perform, and disseminate research, along with scientific societies, should support and enable automated research workflows (ARWs) that embody the following design principles: ● ARWs and the systems, tools, and platforms that comprise them should facilitate openness, reproducibility, and transparency. ● ARWs should facilitate the effective use of artificial intelligence (AI) and machine learning (ML) as research tools and incorporate principles of responsible AI and ML to mitigate the risks from various human and technological deficiencies, such as confirmation and sampling biases, inappropriate application of statistics, and challenges to interpretability of results and quantification of confidence and uncertainties when drawing inferences from ML analyses. 139 PREPUBLICATION COPY—Uncorrected Proofs

● The associated research objects (data, code, even entire workflows) for ARWs should be FAIR (Findable, Accessible, Interoperable, and Reusable), not only by humans but also by machines, to facilitate automated reuse and collaboration. ● ARWs should prioritize reuse and sustainability of existing tools and systems when possible and appropriate, reducing costly duplication efforts and facilitating the extension of capabilities through integration or federation of systems, and agreement on standards. Designs should allow for specialization into specific domains, but avoid unnecessary rebuilding. ● While proprietary services and components can enhance the utility of ARWs, key ARW infrastructure should be controlled by and be accessible to the research community itself, with the community developing standards and practices to facilitate this. Finding C: Research Enterprise Realizing the potential of automated research workflows (ARWs) will require modification of the research enterprise, including sustainable funding for the necessary hardware, software, and human resources, educating the scientific workforce, reporting and sharing research results, and structuring researcher rewards and incentives. Multidisciplinary, multirole collaboration is essential to realize the potential of ARWs. Recommendation 2: Infrastructure, Code, and Data Sustainability Research funders, working with other stakeholders such as societies, research institutions, and publishers, should place greater priority on approaches to ensuring the creation and 140 PREPUBLICATION COPY—Uncorrected Proofs

sustainability of key systems, tools, platforms, and data archives for automated research workflows (ARWs). Priorities include: ● Funding support for efforts by research institutions and societies to link disciplines so they can share and benefit from the expertise in statistics, machine learning, or data science, and engineering and computer science that is required to build and maintain sustainable infrastructure for ARWs. ● Funders and research communities structuring funding for cyberinfrastructure projects such as large scientific instruments so as to maximize the potential for innovation in ARWs and the reuse of data and other outputs. ● Funders and research institutions supporting open data standards and open interfaces for scientific instruments. ● Funders and research institutions enabling reuse, reproducibility, and long-term sharing of FAIR data and software resources through support of repositories that make archival and updated versions of these resources available within and across disciplines, and providing approaches to sustain those repositories. ● Publishers updating their data-sharing requirements by directly associating articles to data in FAIR repositories. Recommendation 3: Human Resources Research funders, higher education, research institutions, and scientific and professional societies should support the development and implementation of educational programs and career pathways aimed at building the workforce needed to develop and utilize automated 141 PREPUBLICATION COPY—Uncorrected Proofs

research workflows (ARWs), including the creation of career tracks that support ARW capabilities. Examples of what is needed include: ● Programs that foster integration of domain expertise with data science and software engineering skills. ● Programs that inculcate data literacy and computational analytical skills in all areas of research. ● Developing the human resources needed to build, maintain, and operate ARW hardware and software, including hardware and software engineers who build, maintain, and operate automated laboratories and the software needed to learn from data and to design experiments. ● Fostering collaborative research that aims at developing and using ARWs and that facilitates sharing workflows, code, data, and data products in ways that respect and protect privacy considerations. Recommendation 4: Culture and Incentives Research funders, research institutions, and disciplines should work to create an automated research workflow (ARW)-friendly culture by making changes in incentive and reward structures aimed at encouraging behaviors that are central to realizing the potential of ARWs. These include: ● Encouraging team science and multidisciplinary teams. 142 PREPUBLICATION COPY—Uncorrected Proofs

● Using funding support and provisions for data management plans to encourage development and curation of FAIR, responsible, and good-quality data resources. ● Developing, improving, and sharing software resources. ● Reporting reproducible results. ● Helping others adopt ARW practices. ● Pursuing international collaboration when possible in order to accelerate progress toward implementing the above changes at scale. Finding D: Legal and Policy Issues In addition to barriers to progress that exist within the research process itself, there are legal and policy issues that affect implementation of automated research workflows in specific domains that will require international multistakeholder efforts to address. Recommendation 5: Preserving Privacy Research enterprise funders, performers, publishers, and beneficiaries should work with governments, data privacy experts, and other entities to address the legal, policy, and associated technical barriers to implementing automated research workflows in use-inspired applications in specific domains and explore solutions to make the outputs available through privacy-preserving algorithms, federated learning approaches to using data, and other methods. 143 PREPUBLICATION COPY—Uncorrected Proofs

Next: References »
Automated Research Workflows For Accelerated Discovery: Closing the Knowledge Discovery Loop Get This Book
×
Buy Paperback | $45.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The needs and demands placed on science to address a range of urgent problems are growing. The world is faced with complex, interrelated challenges in which the way forward lies hidden or dispersed across disciplines and organizations. For centuries, scientific research has progressed through iteration of a workflow built on experimentation or observation and analysis of the resulting data. While computers and automation technologies have played a central role in research workflows for decades to acquire, process, and analyze data, these same computing and automation technologies can now also control the acquisition of data, for example, through the design of new experiments or decision making about new observations.

The term automated research workflow (ARW) describes scientific research processes that are emerging across a variety of disciplines and fields. ARWs integrate computation, laboratory automation, and tools from artificial intelligence in the performance of tasks that make up the research process, such as designing experiments, observations, and simulations; collecting and analyzing data; and learning from the results to inform further experiments, observations, and simulations. The common goal of researchers implementing ARWs is to accelerate scientific knowledge generation, potentially by orders of magnitude, while achieving greater control and reproducibility in the scientific process.

Automated Research Workflows for Accelerated Discovery: Closing the Knowledge Discovery Loop examines current efforts to develop advanced and automated workflows to accelerate research progress, including wider use of artificial intelligence. This report identifies research needs and priorities in the use of advanced and automated workflows for scientific research. Automated Research Workflows for Accelerated Discovery is intended to create awareness, momentum, and synergies to realize the potential of ARWs in scholarly discovery.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!