National Academies Press: OpenBook
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR1
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR2
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR3
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR4
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR5
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR6
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR7
Page viii Cite
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR8
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR9
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR10
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR11
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. doi: 10.17226/26507.
×
PageR12

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Prepublication Copy – Subject to Further Editorial Correction FOSTERING RESPONSIBLE COMPUTING RESEARCH FOUNDATIONS AND PRACTICES Committee on Responsible Computing Research: Ethics and Governance of Computing Research and Its Applications Computer Science and Telecommunications Board Division on Engineering and Physical Sciences A Consensus Study Report of PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION

THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001 This activity was supported by the National Science Foundation Grant No. CNS-1937181. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project. International Standard Book Number-13: 978-0-309-XXXXX-X International Standard Book Number-10: 0-309-XXXXX-X Digital Object Identifier: https://doi.org/10.17226/26507 This publication is available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu. Copyright 2022 by the National Academy of Sciences. National Academies of Sciences, Engineering, and Medicine and National Academies Press and the graphical logos for each are all trademarks of the National Academy of Sciences. All rights reserved. Printed in the United States of America Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. https://doi.org/10.17226/26507. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION

The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president. The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president. The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president. The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine. Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION

Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task. Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies. Rapid Expert Consultations published by the National Academies of Sciences, Engineering, and Medicine are authored by subject-matter experts on narrowly focused topics that can be supported by a body of evidence. The discussions contained in rapid expert consultations are considered those of the authors and do not contain policy recommendations. Rapid expert consultations are reviewed by the institution before release. For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION

COMMITTEE ON RESPONSIBLE COMPUTING RESEARCH: ETHICS AND GOVERNANCE OF COMPUTING RESEARCH AND ITS APPLICATIONS BARBARA J. GROSZ, NAE,1 Harvard University, Chair MARK ACKERMAN, University of Michigan, Ann Arbor STEVE M. BELLOVIN, NAE, Columbia University MARIANO-FLORENTINO CUÉLLAR, Carnegie Endowment for International Peace DAVID DANKS, University of California, San Diego MEGAN FINN, University of Washington MARY L. GRAY, Microsoft Research JOHN L. HENNESSY, NAS2/NAE, Stanford University and Alphabet, Inc. AYANNA M. HOWARD, Ohio State University JON M. KLEINBERG, NAS/NAE, Cornell University SETH LAZAR, Australian National University JAMES MANYIKA, McKinsey Global Institute and Google, Inc. JAMES MICKENS, Harvard University AMANDA STENT, Colby College Staff JON EISENBERG, Senior Board Director, Study Director KATIRIA ORTIZ, Associate Program Officer SHENAE BRADLEY, Administrative Assistant 1 Member, National Academy of Engineering. 2 Member, National Academy of Sciences. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION v

COMPUTER SCIENCE AND TELECOMMUNICATIONS BOARD LAURA HAAS, NAE,1 University of Massachusetts Amherst, Chair DAVID CULLER, NAE, University of California, Berkeley ERIC HORVITZ, NAE, Microsoft Research CHARLES ISBELL, Georgia Institute of Technology ELIZABETH MYNATT, Georgia Institute of Technology CRAIG PARTRIDGE, Colorado State University DANIELA RUS, NAE, Massachusetts Institute of Technology MARGO SELTZER, NAE, University of British Columbia NAMBIRAJAN SESHADRI, NAE, University of California, San Diego MOSHE Y. VARDI, NAS2/NAE, Rice University Staff JON K. EISENBERG, Senior Board Director SHENAE A. BRADLEY, Administrative Assistant RENEE HAWKINS, Finance Business Partner THƠ NGUYỄN, Senior Program Officer KATIRIA ORTIZ, Associate Program Officer BRENDAN ROACH, Program Officer 1 Member, National Academy of Engineering. 2 Member, National Academy of Sciences. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION vi

Preface Computing technology is increasingly woven into our personal and professional lives, physical infrastructure, and societal fabric. With this rise in computing’s impact comes an interest in ensuring that its use contributes to human flourishing, thriving societies, and a healthy planet and an interest in addressing ethical and societal impact concerns that arise when computing technologies lead to such undesirable outcomes as an erosion of personal privacy, the spread of false information and propaganda, biased or unfair decision making, disparate socioeconomic impacts, or diminished human agency. It has become increasingly apparent that it is vital for the computing research community to increase its capacity to address these concerns. Accordingly, the National Science Foundation requested that the National Academies of Sciences, Engineering, and Medicine examine best practices that research sponsors, research-performing institutions, and individual researchers can use to formulate, conduct, and evaluate computing research and associated activities in a responsible manner (see Box P.1). To carry out the study, the National Academies appointed the Committee on Responsible Computing Research (see Appendix A). The study committee comprised expertise across many areas of computer science and engineering, information science, computing technology development, social sciences, philosophy, and law. Within computer science and engineering, the committee included expertise in an array of subfields: theory, systems, artificial intelligence, human-computer interaction, cybersecurity, and robotics. Based on the statement of task, the committee has focused on practical approaches based on scholarship in ethics and in scholarship on sociotechnical systems together with approaches from computer science and engineering, information science, and related fields such as design. Several members of the committee changed their primary professional affiliations during the course of this study. Mariano-Florentino Cuéllar, previously justice of the Supreme Court of California and Herman Phleger Visiting Professor at Stanford University, became president of the Carnegie Endowment for International Peace; David Danks, previously L.L. Thurstone Professor of Philosophy and Psychology at Carnegie Mellon University, became professor of data science and philosophy at the University of California, San Diego; James Manyika, previously chairman and director of the McKinsey Global Institute, became senior vice president of technology and society at Google, Inc.; and Amanda Stent, previously NLP Architect at Bloomberg LP, become director of the David Institute for AI at Colby College. Also, Alondra Nelson, Harold F. Linder Professor at the Institute for Advanced Study, stepped down from the committee in January 2021 when she was appointed as deputy director for science and society at the White House Office of Science and Technology Policy. In order to explore ethical and societal impact issues in context, the committee convened public meetings with experts in criminal and civil justice, public governance, work and labor, and healthcare and with research managers from the computing industry and federal sponsors of computing research (see Appendix B); the committee benefited greatly from the insights these experts contributed. The committee did not consider the distinctive trade-offs associated with the context of national security, cognizant that other groups with more focused expertise have extensively examined such matters.1 1 For example, National Security Commission on Artificial Intelligence, 2021, Final Report, https://www.nscai.gov/2021-final-report,/ and National Research Council and National Academy of Engineering, 2014, Emerging and Readily Available Technologies and National Security: A Framework for Addressing Ethical, Legal, and Societal Issues, The National Academies Press, Washington, DC. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION vii

Early on in its deliberations, the committee considered the questions in the statement of task about governance and regulatory regimes and quickly realized that few if any of these are in place today. What does exist today are sets of principles aimed at guiding those engaged in developing and deploying computing technologies. These principles may be a useful starting point but, as is discussed in Chapter 2 of this report, they are insufficient in themselves as they are divorced from practice and do not provide sufficiently thorough explanations of their underlying assumptions. The primary aim of this report and its recommendations is to empower the computing research community to further develop and use these practical approaches and attain socially beneficial research practices. The committee believes that the adoption of such practices at the research stage will have significant downstream effects by serving as a model for those who develop and deploy computing technologies. Toward this end, the report considers needs for reshaping not only research practice but also computing education; the recommended changes in computing education will help ensure that future computing professionals across industry are better equipped to address ethical and societal concerns. Last, the recommendations also include measures that could help reshape incentives in the computing research ecosystem so that they are better aligned with the goal of responsible computing research. BOX P.1 Statement of Task A National Academies study will explore ethics and governance issues associated with the personal and social consequences of computing research and its applications. The study committee will gather input through at least one open meeting and a solicitation for written comments from relevant research communities and stakeholders. It will consider such topics as: (1) Guiding principles, tools, and practical approaches for identifying and addressing ethical issues; (2) The feasibility and likely performance of research governance frameworks and regulatory regimes, and related best practices that research funders, research-performing institutions, and individual researchers can leverage to formulate, conduct, and evaluate ethical research and associated activities; (3) Multidisciplinary approaches to understanding ethical issues in computing research; (4) How these approaches can empower the research community to develop and pursue socially productive practices; and (5) Ways to promulgate ethical principles and responsible practices and sustain attention to them in the computing research community, including through education and training The study will consider these issues across different subdomains or application areas of computing, such as medicine, autonomous vehicles, and elections. The study will not focus on ethical issues associated with the conduct of research itself except where these relate to the implications of research results. In carrying out this study, the committee will also consider related questions such as: (a) How do ethics and governance issues and needs present differently in different research contexts? Are there other ethics and governance issues that apply more broadly across many or most areas of computing research? (b) What set of research governance frameworks or regulatory regimes are feasible in each of these contexts? (c) How might research governance take place at different granularities and modalities of governance, such as community, organizational, local, regional, national, and international? (d) What empirical evidence exists for how these research governance frameworks or regulatory regimes might correspond to ethically desirable outcomes? (e) What is the current relative maturity level of ethics and governance concepts in different aspects of the computing research space? Which areas are the most advanced and can their relative maturity be leveraged into use elsewhere in computing? (f) What incentives or contextual changes would be effective in helping computing researchers, and those who develop subsequent applications, place more emphasis on ethical considerations? For which existing, and likely future, stakeholders are such changes compatible with current incentives? The committee will prepare a final report containing its analysis, findings, and (as appropriate) recommendations. The report will identify and (to the extent feasible) recommend practical steps that National Science Foundation- supported researchers and others in the computing research community can take to address ethics in all phases of their research from proposal to publication. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION viii

Acknowledgment of Reviewers This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the following individuals for their review of this report: Elizabeth Bradley, University of Colorado, Boulder, Kenneth Calvert, University of Kentucky, Deborah Crawford, University of Tennessee, Knoxville, Finale Doshi-Velez, Harvard University, Batya Friedman, University of Washington, Eric Horvitz, NAE,1 Microsoft, Charles Isbell, Georgia Institute of Technology, Craig Partridge, Colorado State University, Fernando Pereira, NAE, Google, Inc., Allison Stanger, Middlebury College, and Moshe Vardi, NAS2/NAE, Rice University. Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report nor did they see the final draft before its release. The review of this report was overseen by the monitor, Samuel H. Fuller, NAE, Analog Devices, Inc. He was responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring committee and the National Academies. 1 Member, National Academy of Engineering. 2 Member, National Academy of Sciences. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION ix

Contents SUMMARY 1 1 INTRODUCTION 5 1.1 The Nature of Computing and Computing Research, 5 1.2 The Nature of the Technology Innovation Ecosystem, 6 1.3 The Nature of the Computing Research Ecosystem, 7 1.4 The Roles of Ethics and Social Science in Computing, 8 1.5 Sources of Ethical and Societal Impact Challenges, 9 1.6 A Brief History of Concerns, 9 1.7 Characteristics of Responsible Computing in Light of the Ubiquity of Computing Technologies, 11 1.8 Study Approach, 12 2 THEORETICAL FOUNDATIONS FROM ETHICAL AND SOCIAL SCIENCE FRAMEWORKS 14 2.1 The Value and Scope of Ethics, 14 2.2 The Power of a Sociotechnical Perspective, 21 3 SOURCES OF ETHICAL CHALLENGES AND SOCIETAL CONCERNS FOR COMPUTING RESEARCH 29 3.1 Societal Contexts, 30 3.2 Limitations of Human Capabilities, 41 3.3 Societal Contexts and Design and Deployment Choices, 45 3.4 System Robustness, 59 3.5 Limits of a Purely Computing-Technical Approach, 66 4 CONCLUSIONS AND RECOMMENDATIONS 71 4.1 Reshape Computing Research, 74 4.2 Foster and Facilitate Responsible Computing Research, 75 4.3 Support the Development of the Expertise Needed to Integrate Social and Behavioral Science and Ethical Thinking into Computing Research, 77 4.4 Ensure That Researchers Have Access to the Knowledge and Expertise Needed to Assess the Ethical and Societal Implications of Their Work, 80 4.5 Integrate Ethical and Societal Considerations into Computing Research Sponsorship, 81 4.6 Integrate Ethical and Societal Considerations into Publication, 83 4.7 Adhere to Best Practices for Systems Design, Deployment, Oversight, and Monitoring, 86 4.8 Support Engagement with the Public and the Public Interest, 87 APPENDIXES A Committee Member Biographical Information 93 PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION xi

B Presentations to the Committee 98 C Federal Computing Research Programs Related to Ethical and Societal Impact Concerns 100 PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION xii

Next: Summary »
Fostering Responsible Computing Research: Foundations and Practices Get This Book
×
Buy Paperback | $40.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

With computing technologies increasingly woven into our society and infrastructure, it is vital for the computing research community to be able to address the ethical and societal challenges that can arise from the development of these technologies, from the erosion of personal privacy to the spread of false information.

Fostering Responsible Computing Research: Foundations and Practices presents best practices that funding agencies, academic organizations, and individual researchers can use to formulate and conduct computing research in a responsible manner. This report explores ethical issues in computing research as well as ways to promote responsible practices through education and training.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!