4

Conclusions and Recommendations

The technologies that computing research yields have transformed every sector of the economy, enhanced economic welfare through reduced costs and enhanced productivity, and improved our quality of life in many ways. As computing technology has become more pervasive, however, concerns have mounted that some technologies and uses have had harmful effects on individuals, groups of individuals, and society at large, leading to calls for greater attention to ethical and societal considerations in computing research. (The term “computing research” is used in this report to include research in computer science and engineering, information science, and related fields.)

Chapter 2 of this report describes a set of core ethical concepts (Section 2.1) and fundamental ideas from social and behavioral sciences (Section 2.2) for determining ways those ethical concepts play out in everyday life. Chapter 3 describes a variety of generators of ethical and societal challenges that arise in computing research itself and in the further development and integration of computing research outcomes into deployed technologies. As the discussions in its subsections illustrate, some subfields of computer science and engineering—such as artificial intelligence (AI) and machine learning, computer security, and human–computer interaction—have encountered these issues earlier than others, and their experiences are instructive for the entire computing research community.

This chapter recommends initial practical steps toward ensuring that ethical and societal consequences of computing research are more fully considered and addressed and its obligations to support human flourishing, thriving societies, and a healthy planet are fulfilled. Its recommendations assign responsibilities to actors across the full

spectrum of the computing research ecosystem: computer researchers, the computing research community, the scientific and professional societies in which they participate, other scholarly publishers, the public- and private-sector agencies and organizations that sponsor computing research, and the public- and private-sector institutions in which computing research is performed. These practical steps do not relieve public and private organizations from considering other ways to take account of ethical and societal consequences.

Importantly, the recommendations are aimed not just at academic researchers and government research funders even though this study was sponsored by the National Science Foundation (NSF), and the committee could easily have focused its recommendations on NSF and the other federal agencies that sponsor computing research. Industry research is an important contributor to the overall information technology innovation ecosystem, and that ecosystem relies heavily on the exchanges of ideas and researchers between academia and industry. It quickly became apparent to the committee that measures aimed at making government-sponsored academic research more responsible could also be applied in industry settings and that their adoption might well foster better outcomes for the research-performing companies and the overall computing research enterprise as well as fostering societally desirable outcomes.

Accordingly, and with the recognition that the management and shareholders of each company must make their own decisions about what computing research they conduct and how they conduct it, this report adopts inclusive terminology in the recommendations that follow for the actors that sponsor and carry out research that encompasses academic, industrial, and government research organizations. This report adopts the term “researchers” as inclusive of researchers in academia, private-sector and government research institutes, and industry, and uses “research institutions” to encompass colleges, universities, and private- and public-sector research organizations.

Although the incentives of academic and other not-for-profit research institutions differ from those of industry, the recommendations are relevant to all these research environments. It is in the setting of incentives for adopting the recommendations and monitoring their use and outcomes that these types of institutional settings differ.

The term “research community” or “computing research community” is used with respect to issues or recommendations that may not necessarily apply to each individual researcher but that do apply to the community as a whole. The term “research sponsors” is used to include those funding computing research in academia, private-sector and government research institutions, and industry. The term “proposal” includes requests for computing research project funding from any source including government, academic institutions, private philanthropy, and industry and both external and internal support.

The recommendations vary in the particular actors in the computing research ecosystem, individually and in combination, to which they assign particular obligations. Recommendation 1 assigns to researchers and the research community the reshaping of computing research to encompass also the ethics and social and behavioral science expertise needed for responsible computing research. Recommendation 2 calls on research sponsors and research institutions to support the research community in broadening this scope and in defining new kinds of projects and research partnerships to carry out this broader program. Recommendation 3 focuses on education, indicating the need for academic institutions to reshape their curricula in various ways and for scientific and professional societies as well as research sponsors to provide training for computing researchers in practices needed for responsible computing research, both carrying it out and evaluating it. Recommendation 4 complements Recommendation 3 by identifying ways computing research institutions along with research sponsors and scientific and professional societies can provide computing researchers with access to scholars and scholarship in ethics and social and behavioral sciences. Recommendations 5 and 6 focus on the two key actors in the computing ecosystem who vet computing research and can assess whether particular efforts adequately address ethical and societal impacts, namely research sponsors and scholarly publishers. Recommendation 7 addresses computing researchers who develop systems, focusing on the need for them to follow best practices. Recommendation 8 asks all actors in the computing research ecosystem to work together to support better public understanding of computing research and its outcomes.

There is as yet little if any empirical data on the effectiveness of these recommended interventions to draw on. Thus, these recommendations were formulated primarily by considering the actors and leverage points in the computing research ecosystem, drawing on the expertise among the study committee members and presenters to the committee, and reviewing some promising early efforts. As with all innovation in science and engineering, the innovations called for in these recommendations require ongoing assessment and revision to determine what works best—see in particular Recommendations 3.5 (build the capacity to evaluate different approaches for researchers) and 5.4 (evaluate the effectiveness of this report’s recommendations for research sponsorship) below.

The subsection “Advancing Diversity, Equity, and Inclusion” in Chapter 3 discusses the importance to responsible computing research of considering diversity, equity, and inclusion (DEI). This report considers these issues throughout many of its recommendations, which it does rather than offering separate recommendations because the need to pay attention to them permeates the challenge of responsibility in computing research. In particular, these issues are reflected in recommendations addressing the potential

negative impacts of different technologies on underrepresented groups, which is a DEI consideration, and those addressing the value of diversifying the set of stakeholders who inform technological design choices. Much work has been done in the scientific community on DEI challenges in general by other groups with expertise across DEI areas. For example, the National Academies of Sciences, Engineering, and Medicine have compiled a set of reports on diversity and inclusion in science, technology, education, mathematics, and medicine.¹ These concerns are important, and the committee urges all parties engaged in computing research endeavors to act on the recommendations from these reports and those of other groups.

The recommendations provide practical ways for computing researchers and the computing research community to address the situations, conditions, and computing practices discussed in Chapter 3 that have potential to raise ethical challenges and societal concerns. Of special note is the need to address these challenges and concerns at the societal level and not just the individual level. In particular, researchers have an ethical responsibility to consider that computing research can generate or exacerbate not only risks to autonomy, well-being, privacy, and other individual-level intrinsic and instrumental ethical values, but also extreme risks of safety and security breakdowns that could physically harm large numbers of people and society more generally.

Two conclusions from the analyses in Chapters 2 and 3 underlie the recommendations that follow. These conclusions make clear that computing research needs fundamentally to broaden the scope and set of issues it needs to take into account. There may be costs associated with the changes called for in the recommendations below, but they are costs necessary for achieving responsible computing research.

Conclusion 1. To be responsible, computing research needs to expand to include consideration of ethical and societal impact concerns and determination of effective ways to address them.

Conclusion 2. To be responsible, computing research needs to engage the full spectrum of stakeholders and deploy rigorous methodologies and frameworks that have proven effective for identifying the complicated social dynamics that are relevant to these ethical and societal impact concerns.

In keeping with the study’s Statement of Task, the report’s recommendations also do not directly address government regulation of computing technologies including corporate computing research. However, they do discuss ways that the computing research

___________________

¹ See National Academies of Sciences, Engineering, and Medicine, “Diversity and Inclusion in STEMM Collection,” https://www.nap.edu/collection/81/diversity-and-inclusion-in-stemm.

community can help inform government action in this space. As discussed in Chapter 1, the design and deployment of computing technologies that are raising societal and ethical concerns are shaped by a combination of corporate decision-making, incentives set by the market and government regulation, and decisions made by organizations in acquiring the technologies. These factors are the proper realm of societies, who determine norms, and of governments, which institute mechanisms to enforce those norms. Nevertheless, computing researchers have responsibilities related to such societal and ethical concerns. These responsibilities include fully disclosing the capabilities and limitations of their research results and advising the public and governments on areas where adverse impacts may occur and government regulation or changes to corporate governance may be needed. This analysis supports a third conclusion that underlies the recommendations below:

Conclusion 3. For computing technologies to be used responsibly, governments need to establish policies and regulations to protect against adverse ethical and societal impacts. Computing researchers can assist by revealing limitations of their research results and identifying possible adverse impacts and needs for government intervention.

The recommendations are listed in a logical order, not by priority. They differ in the resources, time, and energy they will require. They are intended to work together to enable the computing research community to conduct responsible research. Some of the recommendations are aimed at proactively promoting good while others aim to mitigate or minimize potential harms.

They aim to help the computing research community be more proactive in anticipating and avoiding potential harms rather than, as is generally the case today, only reacting when bad things happen. By taking these steps at the research stage, two kinds of important downstream effects are possible. First, as responsible computing research is taken up by other researchers and technology developers and deployers, it will serve as a model for them to be responsible as well. Second, the recommendations for changes in computing education will help ensure that future computing professionals across industry, including those in product groups and leadership and governance positions, not just research, are better equipped to address ethical and societal concerns.

As noted in the introduction, the recommendations speak to all computing researchers. The recommendations also speak to those whose scholarship and expertise is in disciplines that have studied moral reasoning and those whose scholarship examines the place of science and technology in the world—particularly in philosophical ethics and

the theory of sociotechnical systems. The multidisciplinary efforts recommended will involve much more than straightforwardly applying existing theory. They will require deep engagement of such scholars and computing researchers (Chapter 2).

4.1 RESHAPE COMPUTING RESEARCH

Recommendation 1. The computing research community should reshape the ways computing research is formulated and undertaken to ensure that ethical and societal consequences are considered and addressed appropriately from the start.

Just as the cost of addressing problems after the fact is often much higher than the cost of addressing them during the initial design of a system, so too in research it is much easier to address oversights, unanticipated consequences, and unexpected outcomes if more thought is given to these issues at the outset. (An analogous claim has been made with respect to computer security: that it must be considered at the outset rather than bolted on later.) To promote more positive outcomes and to avoid, mitigate, or otherwise address the potential negative consequences of computing research, computing researchers need to draw on expertise from a variety of domains including those in the humanities and social and behavioral sciences; integrate into their research plans engagement with the various populations who are affected by the outcomes of the research (e.g., conclusions, predictive models, or artifacts); and be transparent about the limitations of such outcomes. This reshaping must be an ongoing process with repeated engagements with experts in a variety of domains.

Recommendation 1.1. Research projects should incorporate applicable and relevant expertise in social and behavioral sciences, ethics, and any domains of application the project includes.

The humanities and social and behavioral sciences provide methods and intellectual approaches that are relevant to anticipating and understanding the effects that technologies are likely to have on people, institutions, and society. Researchers in other fields have expertise that can help computing researchers to determine ways their research can have the impact they desire. To incorporate such knowledge, computing researchers should draw on such expertise. For this collaborative endeavor to succeed, computing research and scholars with expertise in these other fields must each acquire sufficient familiarity with each other’s approaches and methods to have meaningful conversations.

Such familiarity can be acquired through either course work or independent study. Recommendation 3.1 addresses steps that universities can take. For research projects aimed at applications in particular domains (e.g., health care or education), it is likewise crucial to have the participation of experts knowledgeable about those domains.

Recommendation 1.2. Projects that include among their aims the achievement of societally relevant outcomes should engage stakeholders from the start of design through deployment, testing, and redesign processes, and employ design teams that include domain experts and social and behavioral scientists to help ensure that the project is solving the right problems.

If they do not engage the relevant stakeholders, computing researchers run the risk of building systems that may work for themselves (or their friends and colleagues) but may not perform equally well for other populations including ones not originally targeted by the developer. Put another way, builders of computing technology cannot approach design in an abstract manner but must consider the context of where and how the system will be used, who will use the system, and the risks and benefits to different groups of both users and others who will be affected by it. It is important for such projects also to include expertise in the well-established methods, such as participatory design, for accounting for these diverse factors.

Recommendation 1.3. Research projects that produce artifacts or research results likely to be adopted in other research should consider and report on possible limitations such as potential biases or risks of applying them to other problems or in other contexts.

Many research projects produce artifacts, algorithms, or other methods that have the potential to be widely used by other researchers or by industry. Examples include open-source software, data sets for machine learning, and models (including models such as large language models, which are trained on sufficiently broad data and of a sufficient scale that they are adaptable to a wide range of tasks). Release of such artifacts magnifies their impact and fosters reproducibility yet release of very early research outputs, including code, data sets, and tools, poses potential risks. Any research output can potentially be misused or applied inappropriately, in a context that differs significantly from the one envisioned by the original researchers. Potential forms of documentation include warning statements, user guides, data sheets, model cards, use parameter specifications, and recommended evaluation metrics.

4.2 FOSTER AND FACILITATE RESPONSIBLE COMPUTING RESEARCH

Recommendation 2. The computing research community should initiate projects that foster responsible computing research, including research that leads to societal benefits and ethical societal impact and research that helps avoid or mitigate negative outcomes and harms. Both research sponsors and research institutions should encourage and support the pursuit of such projects.

Responsible computing research involves both efforts to mitigate potential harms arising from the use of computing research (preventive) and research activities that have direct positive social good (proactive) with consideration to the scale and impact of those potential impacts. The multidisciplinary nature of both types of work means that they are often harder to fund; both the disciplines and the majority of funding opportunities are siloed. Both types also involve the engagement of experts in the domain(s) of use, which requires additional investment.

Recommendation 2.1. Research sponsors should develop programs aimed at approaches and tools for reducing or mitigating societally harmful characteristics of computing technologies.

Examples of research to mitigate potential harms include the following:

• Tools for mitigating bias and negative privacy impacts.
• Better approaches to system validation, inspectable models, and techniques for auditing system performance.
• Methods for anticipating and assessing the extent of, as well as approaches for addressing potential large or even extreme risks. A current example is assessing the consequences of potential biases or flaws in large-scale machine learning models when they are used as the basis for deployed AI-based systems affecting labor markets, physical processes, or financial practices.
• Research that deepens our understanding of the essential properties of responsible computing.
• Research that helps identify the limits of purely computing technical approach to societally harmful characteristics of computing technologies.

Recommendation 2.2. The computing research community should pursue, and research sponsors should invest in, computing research that would benefit the social good.

Examples of such research include the following: computing research aimed at such goals as sustainability, vaccine allocation, haptic feedback for robotic and telesurgery, or disaster planning and community resilience. Although there may be broad agreement on certain goals (e.g., sustainability or equity), the determination of social good inherently involves values and trade-offs and ultimately entails political decisions. It is the role of government and civil society to decide here as in many other circumstances on the social or public goods that they deem to be important. Research sponsors should look to neutral outside advisory groups for guidance from government, civil society, and economic actors about priorities. For NSF, the National Science Board could be the source of high-level guidance.

Recommendation 2.3. Computing research sponsors should foster and provide support for new kinds of projects that would help facilitate responsible computing research, including the multidisciplinary projects called for in Recommendation 1.

Potential new kinds of projects include the following:

• Government–computing research collaboratives that bring computing researchers (including students and other early career researchers) in direct contact with government agencies and civic and community organizations so that researchers better understand the contexts in which research results will be used and, potentially, contribute to the organization’s technology needs and uses.
• Large-scale computing and data resources (e.g., the proposed National AI Research Resource) that would allow academic researchers to investigate capabilities that otherwise would only be available to large corporations. Such work must, of course, adhere to responsible computing guidelines.
• Public–private partnerships that support academic access to industry data in support of responsible computing research objectives. These partnerships can make possible research that neither academia nor industry could perform on their own. Such activities should include appropriate safeguards so that funding cannot be terminated, or publication restricted without adequate cause.

Recommendation 2.4. Research sponsors and universities should explore new partnerships with companies, nonprofits, and philanthropies to provide financial or in-kind support for responsible computing research with due attention to academic freedom.

One natural focus for such partnerships would be in addressing the sorts of challenges described in Recommendation 2.1. Possible models for such partnerships include the following:

• Research programs partially supported by one or more companies and managed by a federal funding agency, with such key elements as proposal selection and assessment performed by the federal agency. Another possibility is joint funding with a nonprofit organization, such as NSF’s 2018 Early-Concept Grants for Exploratory Research on societal challenges arising from AI technology that was jointly supported by NSF and the Partnership on AI.² NSF has a significant track record with such partnerships.
• Arrangements by industry to provide researchers with access to such artifacts as open data sets.
• Industry–university collaborations that support industry sharing of ethical and societal impact challenges they have encountered and the processes and strategies they have used successfully to address them.
• Industry support for university-based research centers that emphasize responsible computing research and methods.

Recommendation 2.5. To enable carrying out Recommendation 1, research sponsors and research proposers should ensure that, where appropriate, awards provide sufficient resources to include researchers from fields outside computer science and to engage stakeholder groups and outside experts with societal, ethical, or domain expertise.

Recommendation 2.6. To enable carrying out Recommendation 1 and Recommendations 2.1 to 2.3, academic tenure and promotion committees and industry performance reviews should recognize the importance and value of scholarship, both disciplinary and multidisciplinary, that investigates the ethical and societal impacts of computing research.

___________________

² See National Science Foundation, 2018, “Dear Colleague Letter: Early-Concept Grants for Exploratory Research on Artificial Intelligence (AI) and Society—Supported Jointly with the Partnership on AI,” NSF 19-018, https://www.nsf.gov/pubs/2019/nsf19018/nsf19018.jsp.

Responsible computing research requires multidisciplinary research, and the participation of researchers at all career stages in all relevant disciplines. The needed multidisciplinary research starts from an engagement with responsible computing issues and then identifies the concepts and reasoning from a relevant field that can be used to approach resolving them (e.g., “engaged ethics” as described in Section 2.1.1). If this research is to attract leading scholars and scientists, it must count in performance reviews. Just as tenure and promotion committees had to accommodate conference publications,³ here too they need to adjust to accommodate the challenges of assessing the contributions of research that includes these highly multidisciplinary theorems.

4.3 SUPPORT THE DEVELOPMENT OF THE EXPERTISE NEEDED TO INTEGRATE SOCIAL AND BEHAVIORAL SCIENCE AND ETHICAL THINKING INTO COMPUTING RESEARCH

Recommendation 3. Universities, scientific and professional societies, and research and education sponsors should support the development of the expertise needed to integrate social and behavioral science and ethical thinking into computing research.

Responsible computing research requires that all participants in computing research possess a broader scope of expertise than is typical of most undergraduate majors or graduate programs. Responsible computing research will not significantly advance unless educational and training programs adapt and change.

Recommendation 3.1. Universities should enhance (1) teaching and learning in computer science and engineering, information science, and other computing-related fields to ensure that the next generation is better equipped to understand and address ethical issues and potential societal impacts of computing and (2) humanities and social and behavioral science education to ensure that students in those fields are equipped to participate in informed discussions of potential impacts of

___________________

³ D. Patterson, L. Snyder, and J. Ullman, 1999, “Evaluating Computer Scientists and Engineers for Promotion and Tenure,” Computing Research News (September), http://archive2.cra.org/uploads/documents/resources/bpmemos/tenure_review.pdf.

computing research and technologies. Research sponsors should support such activities.

It is critical that these enhancements provide students with appropriate skills and experiences, rather than simply requiring students to take an introductory course in the other discipline(s). The goal here should be to teach students to understand ideas, theories, and results from the other discipline(s) and use them productively, rather than conduct or produce work in the other discipline(s).

The computing ethics projects described in the subsection “Integrating Ethical and Societal Issues into Training” in Chapter 3 are being carried out in diverse types of higher education settings, including small colleges, public and private universities, several of which serve populations underrepresented in computing fields. Their varying innovative approaches to integrating the teaching of ethics and responsible computing are leading indicators that academic institutions of all types can shape programs that address these needs. This variation also illustrates how each institution must, drawing on the various emerging models, make its own decisions about which approaches best fit their particular context.

Specific possible actions by universities include the following:

• More tightly integrate relevant ethics and social and behavioral knowledge into existing undergraduate and graduate computing courses and into curricular degree requirements. Sources of work that could be adopted or contributed to include the Online Ethics Center for Engineering and Science,⁴ which has many resources for teachers that sometimes address sociotechnical systems and the Responsible Computer Science Challenge,⁵ which has yielded a range of teaching materials, including curricular modules and other resources for integrating ethics and social sciences into the computer science curriculum,^6,7 the Computing Ethics Narratives project,⁸ and the Massachusetts Institute of Technology’s Case Studies in Social and Ethical Responsibilities of Computing.⁹ Develop and share additional corpora of case studies, including examples encountered in research projects.

___________________

⁴ See Online Ethics Center for Engineering and Science, “Homepage,” https://onlineethics.org.

⁵ A program sponsored by Omidyar Network, Mozilla, Schmidt Futures, and Craig Newmark Philanthropies.

⁶ Mozilla, “Teaching Responsible Computing Playbook,” https://foundation.mozilla.org/en/what-we-fund/awards/teaching-responsible-computing-playbook/teaching-materials.

⁷ See Computing Ethics Narratives, “Homepage,” https://www.computingnarratives.com.

⁸ Ibid.

⁹ D. Kaiser and J. Shah, 2021, Case Studies in Social and Ethical Responsibilities of Computing, Massachusetts Institute of Technology: MIT OpenCourseWare, https://ocw.mit.edu/resources/res-tll-007-case-studies-in-socialand-ethical-responsibilities-of-computing-fall-2021.

• Integrate a deeper knowledge of the powers and limitations of computing into undergraduate and graduate curricula in the social and behavioral sciences and humanities, thus enabling students to understand better ways in which their field can contribute to public understanding.
• Ensure that in particular those engaged in graduate training in one of the non-computing fields relevant to progressing responsible computing have the necessary technical expertise to subsequently collaborate and engage with technical computing disciplines.
• Provide opportunities for students to engage with computing research and its application in real-world social contexts; for example, through work with federal, state, and local governments.
• Design undergraduate majors, undergraduate- and master’s-level certificates, or degrees in computing ethics for computer scientists and in computing for social and behavioral scientists and humanists. Existing examples include the minor in Societal and Human Impacts of Future Technologies at Carnegie Mellon University and the Master’s in AI Ethics program at Cambridge University.
• Develop incentives—such as reduced course loads and support for course development—for faculty in both computer science departments and relevant social and behavioral sciences and humanities departments to collaborate in integrating the teaching of ethics and consideration of societal impact into the computer science curriculum.

Specific possible actions by sponsors of computing research and education include the following:

• Support the development and sharing of curricular materials by multidisciplinary teams and
• Explore ways to enable and encourage principal investigators (PIs) to expand the scope of educational programs to provide students with opportunities to participate in multidisciplinary research, whether through programs such as NSF’s Research Experiences for Undergraduates or PI-directed educational activities. This may require new forms of such programs that can accommodate multiple kinds of faculty expertise in a grant.

Recommendation 3.2. Organizers of computing research conferences, workshops, and meetings of principal investigators should convene sessions or events at their meetings to share best practices and otherwise

promote responsible computing research, both disciplinary and multidisciplinary. Research sponsors should support such activities.

Specific actions in support of this recommendation include the following:

• Research sponsors should support academic, scientific, and professional organizations in hosting meetings of PIs and other computing researchers that expose them to effective and best practices for enabling deeply multidisciplinary research.
• Research sponsors should support substantive opportunities (e.g., summer intensive workshops) that bring together early career humanists, social and behavioral scientists, and computing researchers with established mentors in their respective disciplines focused on developing cohorts with deeper connections and literacies across the disciplines that need to cooperate on sociotechnical problems.

Recommendation 3.3. Conference organizers, journals, and research sponsors should provide computing researchers with guidelines and training opportunities on the appropriate ways to review ethical and societal issues in papers and proposals.

This guidance is particularly critical as researchers are increasingly required (e.g., by conferences and funding programs—see Recommendations 5.1, 5.3, and 6) to engage with ethical and societal issues, but typically lack the experience or training to appropriately meet these requirements. This recommendation focuses on the groups who are best positioned to provide much-needed training and education to researchers.

It is well established that peer review for multidisciplinary proposals faces particular challenges. Organizations that rely on peer review to assess academic quality should try to recruit scholars with experience of multidisciplinary work to review such proposals, rather than relying on experts in only one of the constituent disciplines.

Recommendation 3.4. Universities and computing research sponsors should, through their education and research activities, develop programs that help create the knowledge, expertise, and talent pool that public- and private-sector organizations will need to make knowledgeable decisions about their acquisition of computing technologies.

Decisions about acquisition and deployment are often made by individuals in government and the private sector who lack the training to assess the quality of a proposed acquisition with respect to its suitability and its potential impacts on all affected parties. This recommendation is meant to develop students with the expertise to realize Recommendation 8, below. Without a recommendation such as this one, computing research might become significantly more ethical and responsible even as computing use continues on its current problematic trajectory.

Recommendation 3.5. Research sponsors should support the development of techniques and capacity for evaluating the effectiveness of different approaches to enabling researchers to address ethical and societal implications of computing research.

The computing research community has not yet been able to establish best practices for responsible computing research. Many ideas and proposals have been made, but there has been little practical implementation or empirical validation of these ideas. Moreover, research on methodological issues has historically not rewarded systematic empirical tests or validations. Hence, it is up to research sponsors to step into this gap to ensure that the research community is learning in a rigorous, informed manner how to do things better.

A specific opportunity for research sponsors is to support empirical tests of educational and workflow interventions to produce researchers and research teams who conduct more responsible computing research—for example, research to systematically examine the effectiveness and other implications of different models for ethics review of research projects and scholarly publications.

Research sponsors, institutions participating in research, and in some cases external civil society organizations can all contribute by developing the capacity to assess, and potentially audit, the extent to which these ethical and societal concerns are being taken seriously in research over time.

4.4 ENSURE THAT RESEARCHERS HAVE ACCESS TO THE KNOWLEDGE AND EXPERTISE NEEDED TO ASSESS THE ETHICAL AND SOCIETAL IMPLICATIONS OF THEIR WORK

Recommendation 4. Computing research organizations—working with scientific and professional societies and research sponsors—should ensure that their computing faculty, students, and research staff have

access to scholars with the expertise to advise them in examining potential ethical and societal implications of proposed and ongoing research activities, including ways to engage relevant groups of stakeholders. Computing researchers should seek out such advice.

Many of the recommendations in this report (notably Recommendations 1, 2, and 5) assume that computing researchers can access the expertise in ethics and social and behavioral sciences they need to design and carry out responsible research projects. This recommendation addresses several ways that such expertise can be made available in a variety of institutional settings.

Computing researchers should seek out such advice within their own institutions and from scientific and professional organizations, suitable expertise on the ethical and societal implications of their research, and if such expertise is not immediately available, work with their institutions and networks to develop access to it.

Recommendation 4.1. Computing research organizations should identify in-house experts who can be consulted by computing faculty, students, and research on how to address ethical considerations and societal impacts of their research. As scholars from most of the disciplines that computing researchers will need to draw on would need funding to collaborate, universities should work with the various stakeholders to help identify resources that can enable this collaboration.

Importantly, this recommendation does not call for creating another mechanism for centralized institutional review of research project designs but rather to create the capacity for ongoing consultation by researchers as projects evolve. Such an approach mirrors practices emerging in industry to bring early consultation into earlier stages of research and design. There is a useful analogy to statistical consulting centers that many universities have. An important goal for such centers is to provide opportunities for researchers to consult experts in statistics in the early phases of their research project ideation, when larger changes are most readily made. This analogy also suggests potential funding models, such as a mix of institutional support for shorter-term or smaller-scale engagements and support from research project budgets for larger engagements.

Recommendation 4.2. Scientific and professional societies of computing researchers should help their members identify experts whom they can consult when developing proposals and carrying out research

projects. Such connections can be particularly valuable for researchers who do not have extensive access to in-house expertise at their own institutions.

Smaller institutions may not have the capacity to provide in-house expertise in responsible computing research. Early career stage computing professionals are least likely to know experts in relevant non-computing fields, and they might be more comfortable engaging with experts outside their institution who are familiar with their own field or subfield of computing research.

Recommendation 4.3. Research sponsors and scientific and professional societies should support the development and sharing of tutorials, best practice descriptions, class materials, and the like to provide concrete examples of solutions that have worked previously for researchers.

The best way to speed progress is to share best practices and other knowledge as widely as possible throughout the research community.

4.5 INTEGRATE ETHICAL AND SOCIETAL CONSIDERATIONS INTO COMPUTING RESEARCH SPONSORSHIP

Recommendation 5. Sponsors of computing research should require that ethical and societal considerations be interwoven into research proposals, evaluated in proposal review, and included in project reports.

This recommendation recognizes the sociotechnical nature of computing research. It is designed to ensure that engagement with ethical and social responsibility becomes a routine component of computing research throughout its lifecycle, starting with research ideation and design, continuing with the challenges encountered and changes to project plans that occur as research projects evolve, and ending with reporting of lessons learned from the research.

Note that this recommendation and the subrecommendations under it call for something distinct from and complementary to existing institutions, rules, and practices designed to protect human subjects (e.g., institutional review boards and informed

consent rules) or to ensure the ethical conduct of research (e.g., the National Science Foundation’s requirements for responsible and ethical conduct of research¹⁰).

Recommendation 5.1. Computing research proposals should describe in an integrated fashion the ethical and societal considerations associated with the proposed work and ways the work will address those considerations. Research sponsors should avoid requiring a freestanding top-level section focused on ethics because such a separation from the main contents of the proposal would treat ethics as an add-on rather than integral to the proposed work.

Specifically:

• The content of computing research proposals should describe the ethical and societal considerations associated with the proposed work and how they will be addressed in the proposed research (e.g., participation of researchers from other disciplines or involvement of domain experts and stakeholders).
• Ethical and societal considerations should be integrated into the main body of the proposal (e.g., addressed explicitly in sections describing the proposed work, its intellectual merit, and its broader impacts) and not segregated into an “add-on” section.
• The proposed list of researchers who will participate in carrying out the research should include the full range of requisite disciplinary expertise, and the proposed project plan and budget should reflect their active participation.

Implementers of this recommendation can draw on the experiences of recent experiments with oversight of responsible computing research at the proposal stage. Two recent examples are the Stanford Institute for Human-Centered Artificial Intelligence’s Ethics and Society Review board¹¹ and the Microsoft Research Ethics Review Program.¹² There are other interesting examples of ethics oversight in the corporate sector that are in service of product development rather than research and are not addressed here.

___________________

¹⁰ National Science Foundation, “Responsible and Ethical Conduct of Research,” https://www.nsf.gov/od/recr.jsp.

¹¹ M.S. Bernstein, M. Levi, D. Magnus, B. ARajala, D. Satz, and C. Waeiss, 2021, “ESR: Ethics and Society Review of Artificial Intelligence Research,” arXiv, https://arxiv.org/abs/2106.11521.

¹² Microsoft, “Microsoft Research Ethics Review Program & IRB,” https://aka.ms/msrethicsreviewprogram.

Recommendation 5.2. Computing research sponsors should develop criteria for evaluating ethical and societal considerations and ensure that project review panels have the requisite expertise to conduct such evaluations.

Individual computer scientists may not currently possess the expertise to consistently evaluate research project designs for the adequacy of their presentation of ethical and societal impacts. Thus, a first step for the computing research community is to establish guidelines for evaluation of proposals to assess whether they appropriately address ethics concerns specific to the field of computer science. These criteria will also serve as a signal of what responsible computing research looks like to researchers developing proposals. Chapters 2 and 3 of this report are intended to serve as a useful source in developing such criteria.

Specifically:

• Funding agencies should develop and publicize transparent criteria for evaluation of ethical and societal impact considerations (e.g., in the descriptions of the research, its intellectual merit and broader impacts).
• To have the requisite expertise on review panels, the panels should (1) include researchers with relevant expertise in assessing how well a proposal takes into account ethical and societal impacts; (2) be diverse and inclusive (e.g., to increase the likelihood that impacts on all relevant communities are considered); and (3) include reviewers with significant experience participating in multidisciplinary projects.

Recommendation 5.3. Computing research sponsors should require that project reports address ethical and societal issues that arose.

Research is dynamic and emergent, and researchers often encounter unexpected roadblocks and results. In response, researchers adapt and adjust the course of their projects. As they do so, they need also to reassess and modify their plans for handling potential ethics and societal impact effects of the work. Researchers should periodically update sponsors about the shifts in this aspect of their projects. To avoid additional reporting requirements, the most appropriate place to address adaptations and updates to these changes as well as to the technical research plans is in existing reporting mechanisms.

A concern, particularly for young researchers or researchers who might feel vulnerable, is that such reports to program officers about challenges or changes in research design could have career-affecting negative consequences—for example, lead to exclusion from future funding opportunities. Thus, it will be important that any response to such reports be to provide non-judgmental guidance to researchers as they responsibly attempt to adjust their research plans.

Specifically:

• Research sponsors should require that interim project reports provide updates on societal or ethical issues encountered and describe the ways they were addressed.
• Research sponsors should require that final project reports discuss lessons learned about ethical and societal impact. In particular, they should include a summary of any unanticipated ethical or social consequences of the research and provide guidance regarding ethical considerations to future researchers and developers who might subsequently extend or use the results of the research.
• Research sponsors should develop ways to share high-level, salient lessons learned in an anonymized, aggregated manner.

Recommendation 5.4. Research sponsors should evaluate the impacts of their implementation of these recommendations after approximately 5 years (i.e., at least one grant cycle after new requirements are developed and issued) and periodically thereafter.

Recommendations 5.1 to 5.3 are, in the view of the committee, the plausible, practical steps that research sponsors could take to ensure that the computing research they support addresses ethical and societal impact challenges. The intent is that these recommendations will lead to deep and effective engagement with ethical and societal impacts by the research community yet not be overly burdensome to researchers or their sponsors. Because the proposed interventions are new, their effectiveness should be assessed, and the interventions adapted as needed to meet the intention of the recommendation.

4.6 INTEGRATE ETHICAL AND SOCIETAL CONSIDERATIONS INTO PUBLICATION

Recommendation 6. Scientific and professional societies and other publishers of computing research should take steps to ensure that ethical and societal considerations are appropriately addressed in publications. The computing research community should likewise take steps to ensure that these considerations are appropriately addressed in the public release of artifacts.

This recommendation is intended to be applied to all the ways in which computing research is published or otherwise released. In many areas of computing research, conference proceedings are at least as important as journals in publishing research results. Moreover, computing research also produces artifacts—code, models, and data—that play an important and complementary role to conferences and journals in disseminating research results. For example, software is often released with an open-source license and deposited in a repository such as GitHub. And like many other disciplines, computing researchers frequently deposit papers in non-reviewed repositories such as arXiv. The recommendations in this report call on computing researchers and organizations where computing research is conducted to address responsibility not only in conferences and journals but also when releasing artifacts and papers in repositories such as GitHub and arXiv.

Many subfields of computing research have robust traditions of releasing code and data as part of scientific publications, to ensure reproducibility and, in many cases, to foster the wider use of these artifacts. Some of these research artifacts develop large user bases over time, numbering in the tens of thousands. Standard conference and journal review processes often omit in-depth examination of accompanying artifacts; as such artifacts are critical to the field (and potentially to society), they should be treated as first-class objects in scientific publications.

Recommendation 6.1. Conferences and journals should include in their evaluation criteria and metrics an assessment of how well a paper addresses ethical issues and societal impacts associated with the research, approaches taken by the researchers to mitigate these issues, and potential approaches that future researchers or developers using these results should take to mitigate potential negative impacts.

Specifically:

• Publication venues should establish a clear set of responsible computing research guidelines for authors and reviewers. These guidelines should be created by the venue’s governing scientific or professional association¹³ or created by the venue after consultation with a diverse group of scientists. (These guidelines would be in addition to whatever they may require with respect to protecting human research subjects.)
• The guidelines adopted by the publication venue should be posted publicly and linked from the call for papers.

Recommendation 6.2. Conferences and journals should encourage researchers to report unanticipated ethical or social consequences of the research as well as guidance regarding ethical considerations to other researchers and developers who might use the research in the future.

Specifically:

• Publication venues should provide authors with space within the main paper page limits to discuss the ethical considerations of their research.
• Publication venues should provide authors and reviewers questions to consider, “model” ethical considerations discussions, or similar sets of examples and guidelines.¹⁴
• Publication venues should adopt procedures for paper withdrawal in cases where it has been reliably established that there is inadequate attention to ethical or societal consequences of the reported research.
• Publication venues should consider adopting procedures for post-publication discussion of papers’ ethical or societal impacts. A model is provided by journals, for example, in the behavioral sciences, that solicit and publish comments on papers.

___________________

¹³ For example: Linguistic Society of America, “Ethics,” https://www.linguisticsociety.org/resource/ethics; ACM Ethics, “2018 ACM Code,” https://ethics.acm.org/; IEEE, “IEEE Code of Ethics,” https://www.ieee.org/about/corporate/governance/p7-8.html; Association of Computational Linguistics, “ARR ACL Rolling Review,” https://aclrollingreview.org/responsibleNLPresearch/; Neural Information Processing Systems, “A Retrospective on the NeurIPS 2021 Ethics Review Process,” https://blog.neurips.cc/2021/12/03/a-retrospective-on-the-neurips-2021-ethics-review-process.

¹⁴ See P. Nanyakkara, J. Hullman, and N. Diakopoulos, 2021, “Unpacking the Expressed Consequences of AI Research in Broader Impact Statements.” Poster Paper Presentation, Pp. 795-806 in AIES ’21, May 19-21, 2021, Virtual Event, https://drive.google.com/file/d/1-sj-jP4VNYHCIajOLo0pSMk6HxYimh1E/view?usp=sharing. This paper analyzes broader impact statements in NeurIPS papers and suggests ways to better frame them.

Recommendation 6.3. Review committees should use these criteria and metrics and possess the multidisciplinary expertise needed to do so.

Specifically:

• Publication venues should consider whether to assign ethics review as a standard reviewer responsibility or to assign ethics review to a separate set of reviewers.
• If ethics review is a standard reviewer responsibility, reviewers need to be educated about the responsible computing research guidelines adopted by the venue.
• If ethics review is assigned to a separate set of reviewers, care should be taken to ensure the pool of ethics reviewers is diverse (e.g., geographically diverse) and informed about the guidelines adopted by the venue.

Recommendation 6.4. Scientific and professional societies should identify experts who are willing to be consulted by paper authors, paper reviewers, and program committees.

Analogous to Recommendations 4.1 and 4.2, which are concerned with performing research, preparing and reviewing publications may also require the involvement of people whose primary expertise is not in the field of the publication venue. For example, an AI venue may need to draw on the expertise of a medical ethicist for certain submissions.

Recommendation 6.5. Scientific and professional societies should, with participation and support from academia and industry research institutions, establish criteria for whether and how to release artifacts (hardware, code, models, or data sets) that may have harmful effects. Released artifacts should be accompanied by information about their intended uses, limitations, and potential harmful effects.

There are recently invigorated norms around making artifacts public, especially when they result from publicly funded research. One reason is reproducibility, which is a core scientific value, and in many cases, artifacts are critical to reproducibility. Another is to make the results of research available widely so as to fuel further innovation. However, full transparency must be balanced against other societal values, including but not limited to privacy (data that contains personally identifiable information, where the research

participants did not consent to data release), intellectual property (data, code, or models), and security (data, code, or models that have the potential to weaken national security or that expose significant vulnerabilities in widely used products).

With these points in mind, publication venues should encourage authors and reviewers to think critically and carefully about whether and how to release research artifacts. There are multiple options for releasing artifacts including the following:

• Unrestricted access and use,
• Restrictive licenses,
• Limits on access and monitoring of use, and
• Release of a weakened version of the artifact that mitigates its harmful effects while still allowing others to reproduce or build on the research results.

Code may be buggy; data may be biased or incomplete; models may be incorrect. In certain circumstances, authors may decide not to release artifacts described in their papers. Reviewers should not reject papers for the sole reason that they are not accompanied by research artifacts. Publication venues should consider deeper notions of reproducibility than merely the release of code and other artifacts.¹⁵

Publication venues that host or link to research artifacts other than papers (e.g., code, data) should:

• Consider creating “model” or template code readmes, data sheets, model cards, and so on;
• Encourage reviewers to examine and comment on artifacts that are submitted with papers, or consider separate research artifact review; and
• Define and adopt procedures for artifact withdrawal or update, should authors or others identify ethical failures or gaps.

Recommendation 6.6. Computing research conferences should adopt policies and principles governing when they accept sponsorship and make these policies publicly available.

___________________

¹⁵ Venues that want to encourage reproducibility can host reproducibility hackathons (F. Balsiger, A. Jungo, N.A. R J, J. Chen, I. Ezhov, S. Liu, J. Ma, et al., “The MICCAI Hackathon on Reproducibility, Diversity, and Selection of Papers at the MICCAI Conference,” arXiv, https://arxiv.org/abs/2103.05437); publish interesting negative results, “Journal of Interesting Negative Results in Natural Language Processing and Machine Learning,” http://jinr.site.uottawa.ca); and provide reproducibility checklists of authors and reviewers (Association for the Advancement of Artificial Intelligence, “Reproducibility Checklist,” https://aaai.org/Conferences/AAAI-21/reproducibility-checklist).

Many computing research conferences accept financial support from outside sponsors. Many of these conferences do not have a policy stating the conditions governing such support. Without such a policy, conference organizers handle controversies that arise on an ad hoc basis. By establishing a policy and making it public, conference organizers will save time and energy and the process will be (and be seen as) fairer. One example of such a policy is the “Sponsorship Policy of the ACM Conference on Fairness, Accountability, and Transparency.”¹⁶

4.7 ADHERE TO BEST PRACTICES FOR SYSTEMS DESIGN, DEPLOYMENT, OVERSIGHT, AND MONITORING

Recommendation 7. Computing researchers who are involved in the development or deployment of systems should adhere to established best practices in the computing community for system design, oversight, and monitoring.

Computer science and information science and engineering scholarship along with best practices developed in industry provide a wealth of information about such practices.

Recommendation 7.1. Researchers should follow all well-established best practices for system design.

There are many such best practices for designers and developers including the following:

• For algorithms that optimize, consider the need to optimize for human expertise or the complementarity of human and machine expertise rather than for machine expertise alone;
• Make systems accessible regardless of such differences in abilities as cognitive (including literacy), visual, motor, or hearing ones and regardless of cultural background (e.g., first language, dialect, or accent);
• Integrate technology with organizational practices;
• Make system behaviors and results transparent to the full range of their users and provide automated tooling (e.g., visualizations and performance analysis

___________________

¹⁶ See Association for Computing Machinery, “ACM FAccT Conference Sponsorship Policy,” https://facctconference.org/sponsorship.html.

• tools) so users can understand what a system is doing and how it is interacting with its environment;
• During the system design stage, conceptualize the success of the system not just in terms of quantitative metrics like “performance” or “accuracy,” but also with respect to broader issues involving ethical concerns and broader societal impacts, even if those factors admit only to qualitative, not quantitative assessment;
• Involve appropriately diverse expertise and stakeholders, ensuring that people and groups affected by computing systems are involved in their design, taking into account the global reach of computing technology;
• Ensure that the security and privacy of data is considered at design time, and that these considerations touch every aspect of the system; and
• Identify potential unanticipated uses and mitigate the harms they could cause.

Recommendation 7.2. Researchers producing artifacts (hardware, code, models, and data sets) should be sufficiently transparent as the artifact evolves during their research about the capabilities, maturity, limitations, and potential ethical and societal impacts of the artifacts so that researchers building systems and vendors building products incorporating the artifact and users of those products can adequately assess them. This transparency should be maintained as the artifact evolves.

The intention of this recommendation is that researchers provide enough information so that other researchers and vendors incorporating those artifacts can validate systems prior to and after deployment. Doing so is essential for vendors themselves to be transparent about their systems and allow for vendor-independent evaluation by regulators (where applicable), purchasers, and users. In doing so, it is important for all parties to be clear whether an artifact has been released at an early stage (i.e., not fully tested) in order to get feedback. One recent effort at such transparency can be found in a paper discussing risks of foundation models.¹⁷ Before a full public release, researchers should seriously consider engaging with a diverse group of potential users.

___________________

¹⁷ R. Bommasani, D.A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M.S. Bernstein, et al., 2021, “On the Opportunities and Risks of Foundation Models,” https://doi.org/10.48550/arxiv.2108.07258.

Recommendation 7.3. Researchers should recognize the potential lifetimes of computing systems that may be built based on their work. They should document their design assumptions and, if possible, build in safeguards for triggering reassessment of these design assumptions.

In practice, computing systems are evolving constructions that require adaptation in response to changing facts. Such adaptations should trigger a reconsideration of potential ethical and societal impacts.

4.8 SUPPORT ENGAGEMENT WITH THE PUBLIC AND THE PUBLIC INTEREST

Recommendation 8. Research sponsors, research institutions, and scientific and professional societies should encourage computing researchers to engage with the public and with the public interest and support them in doing so.

Individuals and societies as a whole are often affected by new technologies and would benefit from opportunities to better understand what’s going on “under the hood.” Note that public engagement (Recommendation 8.1) and transparency (Recommendation 8.2) are necessary but not sufficient: Recommendation 8.3 helps provide information that lawmakers, regulators, and other decision-makers need to make informed governance decisions.

Recommendation 8.1. Researchers should consider proposing, and research sponsors should consider supporting, public engagement activities at relevant stages of research projects to inform the public about emerging computing technologies.

Specific opportunities include the following:

• Include multidisciplinary expertise on research teams to help identify relevant publics and effective outreach strategies,
• Leverage existing institutional communications capabilities, and
• Support the involvement of members of the public where needed.

Recommendation 8.2. Computing researchers should develop and promulgate knowledge that supports better decision-making by acquirers of computing technologies, particularly governments. Research sponsors should consider supporting such work as well as research on effective ways of informing decision-makers and the public.

Past mistakes could potentially have been avoided if such information had been available to technology developers or the governments and other users that adopted the technology. For instance, government agencies adopted face recognition technology or systems for parole decision-making only to discover serious bias issues after deployment.

Potential opportunities include:

• Develop methodologies for creating effective “buyers guides” and “users guides” (a data and computing system equivalent of “good features of” nutrition and drug labels) for computing technologies;
• Advise on appropriate approaches for evaluating computing technologies applied in socially impactful contexts;
• Advise on governance gaps and challenges as well as potential approaches; and
• Better document parameters and conditions of applicability and appropriate use of research project results (algorithms, code, and systems) and make them available to decision-makers and users.

Recommendation 8.3. Universities, research funding agencies, and scientific and professional societies should provide opportunities for computing researchers (along with their collaborators in other disciplines and application domains) to advise the public about the limitations as well as the strengths of emerging computing technologies and provide settings in which researchers can learn how to serve effectively in advisory capacities.

Potential opportunities include programs such as the National Academies’ Jefferson Science Fellowship and the American Association for the Advancement of Science’s Science and Technology Fellowships, which bring scientists and engineers to work in federal agencies, and science communication programs of such organizations as the Kavli Foundation and the Alda Center for Communicating Science at Stony Brook University.

Recommendation 8.4. Computing researchers (along with their collaborators in other disciplines and application domains) should be encouraged by universities, research funding agencies, and scientific and professional societies to bring their knowledge of potential effects and consequences to governments and civil society organizations early in a technology’s development and as a technology is considered for or used in different contexts so that potential negative consequences of that technology can be understood and mitigated adequately.

This recommendation aims to provide public officials with greater insights as to the need for regulation of existing or new technologies. As new technologies are developed, some research sponsors may want to fund research on whether and where new regulations may be needed. Importantly, in addition to permitting “good regulation” it helps avoid “bad regulation,” including potential stifling of innovation. Disclosure allows for feedback from in-house and external technologists, avoiding bad regulation (and perhaps stifling of innovation) and permitting good regulation.

This page intentionally left blank.