Fostering Responsible
Computing Research
Foundations and Practices
__________
Committee on Responsible Computing
Research: Ethics and Governance of
Computing Research and Its Applications
Computer Science and Telecommunications
Board
Division on Engineering and Physical Sciences
Consensus Study Report
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This activity was supported by the National Science Foundation Grant No. CNS-1937181. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.
International Standard Book Number-13: 978-0-309-29527-7
International Standard Book Number-10: 0-309-29527-0
Digital Object Identifier: https://doi.org/10.17226/26507
This publication is available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2022 by the National Academy of Sciences. National Academies of Sciences, Engineering, and Medicine and National Academies Press and the graphical logos for each are all trademarks of the National Academy of Sciences. All rights reserved.
Printed in the United States of America.
Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2022. Fostering Responsible Computing Research: Foundations and Practices. Washington, DC: The National Academies Press. https://doi.org/10.17226/26507.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org.
Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task.
Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies.
Rapid Expert Consultations published by the National Academies of Sciences, Engineering, and Medicine are authored by subject-matter experts on narrowly focused topics that can be supported by a body of evidence. The discussions contained in rapid expert consultations are considered those of the authors and do not contain policy recommendations. Rapid expert consultations are reviewed by the institution before release.
For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo.
COMMITTEE ON RESPONSIBLE COMPUTING RESEARCH: ETHICS AND GOVERNANCE OF COMPUTING RESEARCH AND ITS APPLICATIONS
BARBARA J. GROSZ, NAE,1 Harvard University, Chair
MARK ACKERMAN, University of Michigan, Ann Arbor
STEVEN M. BELLOVIN, NAE, Columbia University
MARIANO-FLORENTINO CUÉLLAR, Carnegie Endowment for International Peace
DAVID DANKS, University of California, San Diego
MEGAN FINN, University of Washington
MARY L. GRAY, Microsoft Research
JOHN L. HENNESSY, NAS2/NAE, Stanford University and Alphabet, Inc.
AYANNA M. HOWARD, The Ohio State University
JON M. KLEINBERG, NAS/NAE, Cornell University
SETH LAZAR, Australian National University
JAMES MANYIKA, McKinsey Global Institute and Google, Inc.
JAMES MICKENS, Harvard University
AMANDA STENT, Colby College
Staff
JON K. EISENBERG, Senior Board Director, Study Director
KATIRIA ORTIZ, Associate Program Officer
SHENAE A. BRADLEY, Administrative Assistant
___________________
1 Member, National Academy of Engineering.
2 Member, National Academy of Sciences.
COMPUTER SCIENCE AND TELECOMMUNICATIONS BOARD
LAURA HAAS, NAE,1 University of Massachusetts Amherst, Chair
DAVID CULLER, NAE, University of California, Berkeley
ERIC HORVITZ, NAE, Microsoft Research
CHARLES ISBELL, Georgia Institute of Technology
ELIZABETH MYNATT, Georgia Institute of Technology
CRAIG PARTRIDGE, Colorado State University
DANIELA RUS, NAE, Massachusetts Institute of Technology
MARGO SELTZER, NAE, University of British Columbia
NAMBIRAJAN SESHADRI, NAE, University of California, San Diego
MOSHE Y. VARDI, NAS2/NAE, Rice University
Staff
JON K. EISENBERG, Senior Board Director
SHENAE A. BRADLEY, Administrative Assistant
RENEE HAWKINS, Finance Business Partner
THƠ NGUYỄN, Senior Program Officer
KATIRIA ORTIZ, Associate Program Officer
BRENDAN ROACH, Program Officer
___________________
1 Member, National Academy of Engineering.
2 Member, National Academy of Sciences.
Preface
Computing technology is increasingly woven into our personal and professional lives, physical infrastructure, and societal fabric. With this rise in computing’s impact comes an interest in ensuring that its use contributes to human flourishing; thriving societies; a healthy planet; and an interest in addressing ethical and societal impact concerns that arise when computing technologies lead to such undesirable outcomes as an erosion of personal privacy, the spread of false information and propaganda, biased or unfair decision-making, disparate socioeconomic impacts, or diminished human agency.
It has become increasingly apparent that it is vital for the computing research community to increase its capacity to address these concerns. Accordingly, the National Science Foundation requested that the National Academies of Sciences, Engineering, and Medicine examine best practices that research sponsors, research-performing institutions, and individual researchers can use to formulate, conduct, and evaluate computing research and associated activities in a responsible manner (see Box P.1).
To carry out the study, the National Academies appointed the Committee on Responsible Computing Research (see Appendix A). The study committee comprised expertise across many areas of computer science and engineering, information science, computing technology development, social sciences, philosophy, and law. Within computer science and engineering, the committee included expertise in an array of subfields: theory, systems, artificial intelligence, human–computer interaction, cybersecurity, and robotics. Based on the Statement of Task, the committee has focused on practical approaches based on scholarship in ethics and in scholarship on sociotechnical systems together with approaches from computer science and engineering, information science, and related fields such as design.
Several members of the committee changed their primary professional affiliations during the course of this study. Mariano-Florentino Cuéllar, previously justice of the Supreme Court of California and Herman Phleger Visiting Professor at Stanford University, became president of the Carnegie Endowment for International Peace; David Danks, previously L.L. Thurstone Professor of Philosophy and Psychology at Carnegie Mellon University, became professor of data science and philosophy at the University of California, San Diego; James Manyika, previously chairman and director of the McKinsey Global Institute, became senior vice president of technology and society at Google, Inc.; and Amanda Stent, previously NLP Architect at Bloomberg LP, became director of the Davis Institute for Artificial Intelligence at Colby College. Also, Alondra Nelson, Harold F. Linder Professor at the Institute for Advanced Study, stepped down from the committee in January 2021 when she was appointed as deputy director for science and society at the White House Office of Science and Technology Policy.
In order to explore ethical and societal impact issues in context, the committee convened public meetings with experts in criminal and civil justice, public governance, work and labor, and health care and with research managers from the computing industry and federal sponsors of computing research (see Appendix B); the committee benefited greatly from the insights these experts contributed. The committee did not consider the distinctive trade-offs associated with the context of national security, cognizant that other groups with more focused expertise have extensively examined such matters.1
Early on in its deliberations, the committee considered the questions in the Statement of Task about governance and regulatory regimes and quickly realized that few if any of these are in place today. What does exist today are sets of principles aimed at guiding those engaged in developing and deploying computing technologies. These principles may be a useful starting point but, as is discussed in Chapter 2 of this report, they are insufficient in themselves as they are divorced from practice and do not provide sufficiently thorough explanations of their underlying assumptions.
The primary aim of this report and its recommendations is to empower the computing research community to further develop and use these practical approaches and attain socially beneficial research practices. The committee believes that the adoption of such practices at the research stage will have significant downstream effects by serving as a model for those who develop and deploy computing technologies. Toward this end,
___________________
1 For example, National Security Commission on Artificial Intelligence, 2021, Final Report, https://www.nscai.gov/2021-final-report, and National Research Council and National Academy of Engineering, 2014, Emerging and Readily Available Technologies and National Security: A Framework for Addressing Ethical, Legal, and Societal Issues, The National Academies Press, Washington, DC.
the report considers needs for reshaping not only research practice but also computing education; the recommended changes in computing education will help ensure that future computing professionals across the industry are better equipped to address ethical and societal concerns. Last, the recommendations also include measures that could help reshape incentives in the computing research ecosystem so that they are better aligned with the goal of responsible computing research.
Acknowledgment of Reviewers
This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
We thank the following individuals for their review of this report:
___________________
1 Member, National Academy of Engineering.
Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report nor did they see the final draft before its release. The review of this report was overseen by the monitor, Samuel H. Fuller, NAE, Analog Devices, Inc. He was responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring committee and the National Academies.
___________________
2 Member, National Academy of Sciences.
Contents
1.1 The Nature of Computing and Computing Research
1.2 The Nature of the Technology Innovation Ecosystem
1.3 The Nature of the Computing Research Ecosystem
1.4 The Roles of Ethics and Social Science in Computing
1.5 Sources of Ethical and Societal Impact Challenges
1.6 A Brief History of Concerns
1.7 Characteristics of Responsible Computing in Light of the Ubiquity of Computing Technologies
2 THEORETICAL FOUNDATIONS FROM ETHICAL AND SOCIAL SCIENCE FRAMEWORKS
2.1 The Value and Scope of Ethics
2.2 The Power of a Sociotechnical Perspective
3 SOURCES OF ETHICAL CHALLENGES AND SOCIETAL CONCERNS FOR COMPUTING RESEARCH
3.2 Limitations of Human Capabilities
3.3 Societal Contexts and Design and Deployment Choices
3.5 Limits of a Purely Computing-Technical Approach
4 CONCLUSIONS AND RECOMMENDATIONS
4.1 Reshape Computing Research
4.5 Integrate Ethical and Societal Considerations into Computing Research Sponsorship
4.6 Integrate Ethical and Societal Considerations into Publication
4.7 Adhere to Best Practices for Systems Design, Deployment, Oversight, and Monitoring
4.8 Support Engagement with the Public and the Public Interest
A Committee Member Biographical Information
B Presentations to the Committee
C Federal Computing Research Programs Related to Ethical and Societal Impact Concerns
