Enhancing Public Access to the Results of Research Supported by the Department of Health and Human Services

Proceedings of a Workshop—in Brief

In 2022, the Office of Science and Technology Policy (OSTP) released a memorandum on Ensuring Free, Immediate, and Equitable Access to Federally Funded Research (the “Nelson Memo”), which calls upon federal agencies that fund research and development to update their public access¹ policies by the end of 2024, establish procedures to support scientific and research integrity, and ensure equitable delivery of federally funded research results and data (Nelson, 2022). To help inform the development of these new policies, the National Academies of Sciences, Engineering, and Medicine (the National Academies) held a public workshop from November 30 to December 1, 2023, sponsored by the National Institutes of Health (NIH), to provide a venue for stakeholders to discuss approaches that U.S. Department of Health and Human Services (DHHS) agencies, including NIH, Administration for Children and Families, Administration for Community Living, and the Food and Drug Administration, could consider to ensure changes to public access policies promote equity in publication opportunities for investigators, ways to improve accessibility² to publications by diverse communities of users, and considerations for increasing findability and transparency of research results. This Proceedings of a Workshop—in Brief summarizes the presentations and discussions expressed by workshop participants and should not be seen as a consensus of the workshop participants, the workshop planning committee,* or the National Academies.

IMPROVING PUBLIC ACCESS TO RESEARCH

Speakers Bonnie Swenor, Johns Hopkins Disability Health Research Center, and Carrie Wolinetz, Lewis-Burke Associates, discussed the policy background for federal guidance on public access to research results. Swenor explained that a core component of the Nelson Memo is that federally funded research belongs to the public and there is a duty to make research results accessible to all people. Wolinetz suggested that public access to federally funded research is part of a broader effort to democratize information.

Swenor explained that most research results are not designed to maximize accessibility for the end user, including researchers and study participants (NIH,

__________________

2022). She described barriers she has encountered as a researcher with a disability to accessing publications, data, databases, and dashboards. Similarly, study results are often inaccessible to participants with disabilities. People with disabilities should be empowered to act as co-researchers in studies, which can contribute valuable new ways of looking at information and interpreting data (NIH, 2022; NIMHD, 2023). This can involve information being made available in accessible formats and the acceptance of lived experience as a form of expertise.

Wolinetz pointed out that while federal agencies can have impact on the research ecosystem, they still must operate within statutory authority. She suggested policy makers can develop tools to help stakeholders adapt to policy change. For example, a series of questions could be informative and might include:

• Are there timelines to recognize?
• Are there resources that can be helpful?
• Are there implementation approaches that could help determine accessibility considerations for different communities?

RESEARCHER’S PERSPECTIVE ON OPEN ACCESS PUBLISHING

Phillip Sharp, Massachusetts Institute of Technology (MIT), shared a perspective on the impact of publishing practices on the progression of science based on a recent report, Access to Science and Scholarship: Key Questions About the Future of Research Publishing (Sharp et al., 2023). The paper offers a historical overview of open access (OA) publishing³ and lays out a list of questions to better “understand, measure, and best prepare for the impact of new policies related to open access in research.”

Sharp raised concerns about the sustainability of the peer review process. The increased number of manuscript submissions has put pressure on peer review volunteers, leading to a backlog in the review process and longer review times. Furthermore, the peer review process itself is often opaque and vulnerable to biases. Report co-author Amy Brand, MIT Press, shared that her organization is seeking ways to identify new peer reviewers to better address issues with conflict of interest and involve more of the community in an equitable way.

Sharp expressed concern about the trend of consolidation and vertical integration among publishers, which influences measures of scientific advancement and limits publication avenues. He predicted that technologies, including the influence of artificial intelligence (AI), preprint servers, and new data-sharing platforms, will continue to drive change.

BUSINESS MODELS FOR PUBLIC ACCESS

Speakers shared opportunities and barriers associated with different publishing models that support public access, the impacts of shifted publication costs in response to changes in public access policies, and considerations for reasonable publication costs.

Stefano Bertuzzi, American Society for Microbiology (ASM), spoke from the perspective of a scientific society about how public access fits into the new ASM strategic vision for open science. ASM is restructuring to shift focus from services (e.g., conferences, meetings) to a core concept of empowering the public and scientific communities to shape the future of the field based on their needs. An open publishing model is “absolutely essential” for this shift; ASM has embraced the Subscribe to Open (S2O) model.⁴ Bertuzzi said this model is a way to engage different stakeholders (e.g., libraries, funders, authors), share responsibility, and work together toward open science. He acknowledged that the model is not without implementation challenges, such as decreased revenue for ASM, but emphasized that the partnerships catalyzed by S2O will better serve the scientific community and meet the spirit of the Nelson Memo. Bertuzzi shared concerns with other open publishing models, such as Plan S⁵ and those associated with article processing fees (APCs). He suggested that the former is too directive and does not facilitate stakeholder

__________________

collaboration, and the latter is not equitable because it shifts publication fees from indirect to direct costs, which could potentially make publishing too expensive for authors with fewer resources.

Jessica Sebeok, Wiley, said that as an organization, Wiley recognizes that new models and forms of collaboration are required to meet the needs of the 21st century researcher. A holistic approach to open access acknowledges the interwoven challenges facing the scholarly publishing landscape as well as the potential for unintended consequences. The optimal approach is for stakeholders (e.g., publishers, agencies, authors, institutions) to come together to grapple realistically with how to promote more sharing, accessibility, and ability for everyone to participate in the knowledge economy. There is no single solution for every discipline, for every demographic, or for every publishing model. Sebeok explained that open or public access is not free—there are costs associated with peer review, sorting, editing, curation, overhead, and personnel salaries and stipends. For example, Wiley is the only major publisher to provide data to the Journal Comparison Service, a database that helps stakeholders gauge whether the fees they are paying are commensurate with the publication services they receive. Sebeok noted that determining what counts as a reasonable publication cost (like an APC) requires a more granular analysis relative to other considerations such as grant type, university overhead, equipment costs and conference fees.

Open public access to research does not necessarily lead to more diversity and inclusion, said Kimberley Bugg, Atlanta University Center Robert W. Woodruff Library. Speaking from the perspective of a historically black college and university (HBCU), she pointed out that costs associated with OA may be prohibitive. Of the 101 HBCUs, only 10 are designated R2 (having high research activity), and none are designated R1 (having highest research activity); many are small universities with limited financial resources or research infrastructure. Bugg noted that government grants that include support for publishing costs largely go to universities other than HBCUs. Bugg suggested that HBCUs could form alliances to collaboratively establish repositories and share costs, such as the HBCU Library Alliance.⁶ This multi-university project focuses on historical collections and archival material but could be adapted for research data and publications.

Heather Joseph, Scholarly Publishing and Academic Resources Coalition (SPARC), said that the concept of “open” access should not be the end goal, but instead a strategy to enable fast, broad, barrier-free sharing of knowledge. Joseph provided three suggestions for agencies that are updating their public access policies. First, agencies could consider ways to broaden the scope of acceptable publication models (e.g., preprints, high-impact journals, open repositories) to enhance more equitable outcomes in access, contribution, and social benefit. Second, agencies could introduce more flexibility in options for financial support for publishing and communication of research results (e.g., interagency collective investments in scientific communication infrastructure; direct arrangements with institutions or scientific societies to support publishing). Third, agencies could adjust expectations for how research is shared and communicated to better align with scientific and disciplinary communities.

Günter Waibel, California Digital Library, focused his remarks on the implementation of the Nelson Memo by agencies, publishers, and institutions. He described two predominant publication models in the United States. The first track is a mandatory green OA track, requiring a repository deposit without an embargo or delay. Most U.S. authors today own the copyright in their work, but simultaneously publishers have contractual interventions that aim to divorce the authors from their rights. Waibel suggested alleviating this confusion by invoking the preexisting federal purpose license in agency public access plans, which ensures that all federally funded authors have the prior right to deposit their articles in an agency repository regardless of any contractual agreement with publishers. The second track is a gold OA model, in which federal funds are used to pay for OA publishing fees, largely via inclusion of “reasonable

__________________

publication costs” in grant budgets. Waibel cautioned that this creates a risk of double payments to publishers. For example, a 2019 University of California (UC) review that found UC paid $40 million in publisher subscription contracts one year, while authors paid separately an additional $10 million in APCs the same year. To resolve the double payment issue, subsequent UC open access publisher agreements combined former subscription dollars and grant funds into a single capped payment to the publisher. Based on this example, Waibel suggested that funding agencies continue to emphasize the availability of grant funds for APC payments to authors.

Equity and Accessibility Considerations for Publishing

Speakers offered further discussion on equity and accessibility considerations for publishing in a conversation moderated by Adriene Lim, University of Maryland. Several speakers emphasized the importance of having flexibility to experiment with different publishing approaches. Bertuzzi, Bugg, and Joseph agreed that the APC-only model is unsustainable in the long term. Bertuzzi and Joseph explained that S2O allows libraries and publishers to reassess subscription costs annually. Sebeok rebutted that this approach can be unsustainable for smaller societies with thin margins, potentially diverting funding from other society activities. Bugg expressed uncertainty about whether any of the models discussed lend themselves to fully supporting diversity and inclusion.

Speakers considered how to determine a reasonable cost for publication. Bugg said that reasonableness is subjective and based on what is affordable. Waibel observed that publishing is a commercial marketplace, and that reasonable costs are tied to what people are willing to pay. One issue is that the market suffers from prestige distortion. Joseph said there should be urgency to separate journals with substantial name recognition (“prestige journals”) from the perceived quality of the research being communicated. Several speakers suggested that coalitions could enable more equitable distribution of publishing costs through shared power. As an example, Waibel shared that under the UC open access publisher agreements, the UC library pays the publishing costs for low-resource authors while better resourced authors are asked to contribute from grant funds.

Bertuzzi and Bugg agreed that prestige journal costs are an accessibility barrier. Bertuzzi suggested that funders could issue competitive grants for innovation in scientific publishing models. Sebeok added that Wiley is working to address accessibility along multiple axes: bias and lack of representation on editorial boards, involving local researchers to prevent exploitative practices, addressing language barriers, including gender- affirming language for authors, and implementing assistive technologies.

BEYOND EMBARGOS: ACCESSIBILITY CONSIDERATIONS

Speakers discussed barriers to public access of DHHS-funded research results, steps for improving equity in access and accessibility to publications by a diverse community of users, and ways to develop and sustain bridges to various communities so that many viewpoints are included in the process of improving public access to research publications.

John-Ross Rizzo, New York University (NYU) Grossman School of Medicine, shared an observation that regardless of OA publishing model, this only provides access to the manuscript, not the full content of research results. Accessing the full content requires optimizing digital accessibility. For example, an accessibility roadmap for publishers could include tools for the publication process, similar to the Microsoft 365 Accessibility Checker, and guidelines for accessibility accountability at all levels of the publishing process (e.g., writers, reviewers, editors). Drawing on lived experience of the barriers he faces as a blind editor and reviewer, Rizzo proposed a strategy for “full spectrum access and accessibility” in the peer review process. This approach could include best practices to help ensure the inclusive validity of the scientific publishing process, public reporting of journal accessibility metrics, and community engagement in study design and evaluation (Stern and O’Shea, 2019).

Jenny Peng, Oxford University Press (OUP), reported that 75 percent of OUP journals are society owned. The majority of OUP research articles are projected to be gold OA by the end of 2024. Peng emphasized that offering a variety in available business models (e.g., APC and non-APC OA revenue options, waivers for authors from lower and middle-income countries) is key to providing

flexibility for meeting the accessibility needs of specific communities and authors. OUP’s accessibility policies for their publishing platform include targeting level AA of the Web Content Accessibility Guidelines (WCAG) 2.1,⁷ complying with accessibility laws in the United Kingdom and Europe, providing articles in accessible formats, and search engine optimization to enhance discoverability. She noted the importance of partnering with stakeholders early in the policy development process, particularly when it comes to policies for access and accessibility.

Yvette Pearson, University of Texas at Dallas, said evaluating accessibility for the end products of research begins by asking who was involved in defining the initial research problems and who was engaged in the problem solving. Researchers may need to break down silos and think differently about what counts as quality dissemination of research findings and how to evaluate the impact of research results beyond traditional metrics like publication in high-impact journals. For example, Pearson engages communities to understand their needs and what information is useful based on insights from lived experience to inform study design. When the study is complete, she shares the research results with the community to inform downstream actions and decision making.

Lisa McCorkell, Patient-Led Research Collaborative (PLRC), spoke about accessibility considerations for new public access policies from the viewpoint of the chronic illness and disability communities. PLRC represents people with Long COVID and associated conditions such as chronic fatigue syndrome. Access to research about Long COVID can help patients better understand the condition, new treatments, and means to improve quality of life. McCorkell shared criteria for what “true access” to research results entails: being able to read, understand, use, share, and be a part of the research, all without causing harm, triggering symptoms, or causing financial difficulty. Existing barriers for patients and research participants to accessing and understanding the latest results include the time and energy needed for searching (especially for those with energy limitations or cognitive dysfunction), lack of understanding of where to search for article context, and lack of disability (e.g., WCAG 2.1) or physical (e.g., Internet access) digital accessibility standards. These barriers disproportionately affect those with lower socioeconomic status or education levels. To help address some of these issues, McCorkell suggested that public access policies and tools be developed in partnership with patient communities and people with disabilities. One example is developing plain-language summaries and dissemination guidance in partnership with patient communities. Another example is a scorecard to gauge patient integration into the research process.

E. Yvonne Lewis, Healthy Flint Research Coordinating Center (HFRCC), discussed her work following the Flint water crisis using community engagement and participatory research methods to expand the community’s knowledge of research and how that research could be useful. HFRCC works in close collaborations with local universities and seeks to ensure that studies in the community are being used at every level to make a difference. A key component of their work is translating and disseminating research results. Another role is ensuring that community members are involved in identifying the research problem, the methods to study the problem, the potential solutions, and how to share the results with the community. Lewis stressed that there should be ongoing conversations with community members, particularly those impacted by the focus of the research. Involving community members in the research process is also a way to combat historical mistrust of researchers. Lewis reviewed barriers that can hinder the dissemination process, including language, generation, communication, and culture. She concluded that there will be no one approach to meet all community needs; a variety of mechanisms is needed to make a difference.

Enhancing Public Access Through Stakeholder Engagement

Speakers considered ways to improve public access to research results through building capacity for meaningful engagement across stakeholder groups in a conversation moderated by Pamela Padilla, University of North Texas. Rizzo shared examples of NYU programs to

__________________

facilitate knowledge translation to the public, including a community-focused seminar series and support groups in multiple care domains. Pearson noted that university communication and media relations teams are resources for getting information to the mainstream public. McCorkell and Pearson suggested incorporating communication skills for a range of audiences, including the public, into scientific training.

McCorkell and Peng noted the importance of plain-language summaries for public access. Peng recounted working for an education journal that introduced plain-language summaries that successfully helped teachers engage with the research. She noted that those services are not without cost to publishers. Publishers want to move forward at a pace that is neither prohibitive cost-wise nor increases inequities; there is imperfection in the system, but the perfect should not be the enemy of the good.

Another consideration is the digital infrastructure that facilitates public access and engagement. Peng noted that the gold OA model, despite inequities in who can pay APCs, makes the version of record publicly available. However, availability is a secondary concern if someone lacks reliable Internet access; Padilla referenced barriers to Internet access for trainees during the COVID campus shutdowns, while McCorkell emphasized that public access policies should embrace digital equity. Peng and McCorkell observed the potential for AI to contribute to dissemination efforts, including plain-language summaries. Rizzo proposed that AI could help automate innovative new dissemination formats.

Several speakers noted that funding agencies can play a role in encouraging stakeholder engagement as an approach to public access and accessibility. Pearson shared a recent National Science Foundation (NSF) solicitation, Workplace Equity for Persons with Disability Solicitation (WEPDS) as an example. WEPDS provides funding to research equity barriers in STEM training workplaces and potential solutions. The solicitation requires inclusion of scientists with authentic disability experiences and a plan for knowledge dissemination to the public. Lewis added that there will be more efforts for public engagement if it is a requirement for funding. Padilla advised that another layer of compliance could add extra burdens to university offices and faculty, so agencies will need ways to ensure researchers are resourced, engaged, and see the importance of this topic.

RESEARCH RESULTS: FINDABILITY, TRANSPARENCY, AND ACCESSIBILITY

Speakers presented on ways to improve use of persistent identifiers (PIDs) and metadata, experiences with adoption of different identifiers, and possible benchmarks and metrics that could be used to assess and evaluate accessibility and findability for DHHS-funded research results.

Roger Schonfeld, ITHAKA, offered that the problem is not findability, but rather usability. The public is typically not the intended audience of scientific publications, which can be difficult to understand or assess for validity without specialized knowledge. Schonfeld suggested that a component of public access for medical research results could include regularly updated summaries that are designed for patients and their carers as the primary audience. He concluded by suggesting that new agency policy frameworks consider usability and the associated services as a core component of their responsibility to public access.

Chris Shillum, ORCID, focused his comments on PIDs and metadata as the building blocks of research infrastructure. It is not easy to alter systems and processes that have been embedded for decades, so ORCID and partner organizations need to be clear about changes being made to keep experimenting and driving innovation. PIDs and metadata can be used at all stages of the research life cycle. The goal in using this infrastructure is a more integrated research data ecosystem, with better tracking of outcomes, accurate attribution, relationships among types of entities, reduced administrative burden, and alignment with open research practices. Shillum noted that increased investment into the PID infrastructure by funders can give insights into the impacts of the research they fund, as well as assisting with making research data publicly available once grants are awarded (Chen et al., 2023).

ORCID is also taking steps to use the PID infrastructure to promote trust in science by highlighting the sources of individual assertions and embedding trust markers in review and PID assignment workflows, and in the future potentially in funding workflows.

Jamie Wittenberg, University of Colorado Boulder, highlighted how public access policies that facilitate open data sharing make reproducible, empirical research possible. While the technology exists to implement data-sharing requirements, there are not widely adopted, open, and transparent standards or metrics for how to evaluate and monitor the reuse of scientific data and data quality. Furthermore, requiring data sharing is different than enforcement, and how to integrate incentive structures for researchers into the system remains uncertain (Christensen et al., 2019). The consequences of this disconnect, according to Wittenberg, are that researchers are not citing their data, publishers are not enforcing data-sharing requirements, and agencies and universities do not have broad agreement about who is responsible for data policy compliance. She suggested that agencies could leverage advances in technology and infrastructure to establish and implement best practices for data sharing and metrics instead of relying on a slow uptake among researchers and publishers. An example is the Global Data Citation Corpus a new initiative that uses metadata and machine learning techniques to extract data citations even when the author does not include them as formal references.

Joseph Yracheta, Native BioData Consortium (NBDC), introduced his organization as the first 501(c)3 biobank and data repository housed within the jurisdiction of an American Indian nation. He emphasized that many principles and ideas discussed around OA and data accessibility have neglected consultation with Native American groups, and that open data can be considered “resource theft” and “more of the same taking” if it is not collected and analyzed in consultation with marginalized or indigenous populations. Yracheta acknowledged this seems at odds with the aspirational goals of an open data environment. He cautioned that when operationalized, open data can be weaponized against certain populations, particularly in the context of existing data bias against Native Americans and risks related to privacy and health information (Huria et al., 2019; Kukutai, 2023). These considerations for the cultural limits of data accessibility were not interrogated in the Nelson Memo. NBDC, in collaboration with Tribal groups and the National Institute on Minority Health and Health Disparities, is working to build infrastructure, expertise, and networks for indigenous data governance. The goal is to ultimately govern indigenous data sharing under Tribal law and Tribal cultural practices. Indigenous data sovereignty may not align with open data policies, which in some circumstances, such as data-sharing requirements for some journals, can be interpreted as a form of coercion parallel to the Belmont Report’s guidance for individuals.⁸ NBDC is advocating for a decentralized data governance system where the Tribal government is prime, participants are second, the federal funding agencies are last, and the Tribe and the participants control who collects the data, what happens to that data afterward, who gets to use them, who gets to see them, who gets to store them, and when they are destroyed.

Metrics and Infrastructure for Data Accessibility

Speakers discussed additional considerations on data accessibility metrics, infrastructure, and responsible use in a conversation moderated by Lori Schultz, University of Arizona. Clay Johnston, Harbor Health, explained that identifying metrics forces discipline in defining goals. Bodo Stern, Howard Hughs Medical Institute, Shillum, and Wittenberg agreed there is an over-reliance on citation metrics. Wittenberg referenced ongoing work to establish data reuse metrics at Make Data Count, OCLC, and the Research Data Alliance, though she noted that no single metric will be a perfect indicator of reuse or impact. Yracheta shared concerns that the United States is moving toward open data before creating metrics to assess the social implications. Schonfeld reflected on new definitions of success within the context of public access and the Nelson Memo, underlining output-driven metrics of how the data are used. One example is

__________________

measurements about improving knowledge accessibility for patients or reducing misinformation. Yracheta warned that marginalized communities might not be able to use their own data, even if the data are available, because usability requires infrastructure, training, and financing that are historically lacking in rural, indigenous, and other marginalized communities.

Several speakers agreed that more work is needed to develop digital and institutional infrastructure. In terms of metrics, Shillum noted that years of effort have gone into developing citation metrics infrastructure. Developing new metrics will entail a commensurate effort to define conventions, develop infrastructure, and promote adaptation. Wittenberg cautioned that action is needed to ensure data metrics are open, transparent, and sustainable. Not doing so leaves room for propriety metrics that are unstandardized and opaque. Yracheta agreed with her concerns, calling the proprietary metrics “black boxes.”

In terms of institutional infrastructure for metadata and PIDs, several speakers suggested that more communication is needed among stakeholders. Shillum observed that the research administration and library communities do not communicate often, despite overlap in data and processes used. One solution from ORCID is engaging vendors who provide tools to research administration to integrate PIDs. Wittenberg shared a collaboration between the Association of Research Libraries and the California Digital Library to increase collaboration among institutional offices, such as research offices and libraries, particularly as it relates to data management plans. She also noted that up-front work is needed to make data and research results available, but that those efforts should happen in parallel with advances toward making content more accessible.

Wittenberg proposed that a broader question for workshop participants is the responsibility of a government to its people to deliver information in perpetuity in a way that facilitates understanding. According to Johnston, the notion of usability, within the broader definition of accessibility, is the ultimate mandate of the Nelson Memo and public access; he noted that DHHS agencies may or may not agree with this statement. Schonfeld commented that there is a “profoundly problematic disjuncture” in policy making if the results anticipated from implementation analysis do not give insight into whether the purpose of the policies can be achieved. Shillum recalled the differences of opinion from previous panels about who is responsible for translating research for non-expert audiences. He encouraged agencies to provide new kinds of funding arrangements to investigate this question.

SUPPORTING AND EVALUATING NEW PUBLIC ACCESS POLICIES

In breakout discussions, participants considered approaches and action steps that could be taken by various stakeholders to support new public access policies. Breakout discussion participants then outlined potential criteria for success and benchmarks for progress. Breakout moderators shared their thoughts based on the discussions. Moderators were Igor Bado, Tisch Cancer Institute at Mount Sinai; Svasti Haricharan, Sanford Burnham Prebys Medical Discovery Institute; Iheoma Iruka, University of North Carolina; Julia Maués, Guiding Researchers and Advocates to Scientific Partnerships (GRASP); Jessica Polka, ASAPbio; Shillum; and Waibel.

The following opportunities for progress emerged based on the moderators’ reflections:

1. making research translatable,
2. creating accessibility frameworks for all audiences,
3. engaging patients and patient advocates,
4. leveraging persistent identifier infrastructure, and
5. generating interim research products.

Making Research Translatable

Haricharan, Iruka, and Waibel highlighted issues with communication and opportunities to translate research results into language for a general audience. Waibel stated that multiple parties have responsibility for translating the research, while Haricharan reported that from the publisher’s point of view, the best approach may be for publishers to facilitate the translation of research, although this would require agencies to provide

additional funding that could be put toward professional translation. A systems-level implementation approach is for publishers and funders to work together to embed translatability into peer review for publications and grant evaluation. She noted another potential next step is for publishers to work collectively with software providers on tools that prioritize research translatability. In a related area, Iruka noted the importance of creating plain-language summaries, including agreement on what constitutes plain language (e.g., for who, for what, under what context). Determining benchmarks for gauging the implementation of plain-language summaries often requires reflection on who the best partners could be and whether diverse perspectives across communities and partners are being included. Bado noted how accessibility can be layered, such as a visually impaired seeking a plain-language summary.

Creating Accessibility Frameworks for All Audiences

In terms of inclusivity and accessibility, both Iruka and Haricharan suggested establishing a gold-standard checklist for accessibility to all potential audiences. However, Iruka cautioned that there may be no shared agreement on what the correct standard is, and that people will have difficulty with compliance in the absence of that agreement. She suggested investing in a staffing workforce, such as a chief accessibility officer, who can begin to create infrastructure and enable a culture shift. Other opportunities might include engaging technology companies or the education system, even teaching children how to access information and what is expected. Iruka stated a goal that all public access is inclusive by 2030. Benchmarks toward this might include developing an accessible manuscript submission and review process, engaging diverse communities, securing dedicated funding sources, and formulating a scoring metric. Bado proposed developing fellowships to train people on implementing improved accessibility of publications.

Engaging Patients and Patient Advocates

Bado, Maués, and Polka discussed the importance of accessibility for patient engagement. Maués highlighted the critical importance of involving patients and patient advocates in research. For patient advocates, the intended outcome is to bring patient perspectives into the research early and often so they can direct the work to be more impactful, successful, and facilitate translation of the studies from researchers to communities. Maués suggested that patient advocates should be remunerated for their work to ensure that further disparities are not created where only patient voices who are privileged are heard. She suggested that stipulating patient involvement as a requirement of funding can ensure it will be included. For patients, she pointed out they often do not know the results of the clinical trials in which they participate, and it is important to show the impact of their participation and sometimes sacrifice. Polka suggested that funders support multiyear pilot programs to engage patient groups and create platforms for feedback or open review.

Leveraging Persistent Identifier Infrastructure

Polka, Shillum, and Waibel focused on digital solutions involving PIDs and metadata. Shillum and Waibel agreed that the following is needed: (1) reduce the complexity in the system that has been created; and (2) build infrastructure that enables rapid iteration and experimentation at a much lower level of effort. Shillum suggested that one way to do this is by enabling fuller participation in infrastructure of PIDs and structured metadata. For organizations like ORCID, this means doing more to build awareness of responsibilities for all participants in the research enterprise and providing appropriate training. In addition, Shillum stated that funders need to participate in that process, and publishers should enforce mandates around PIDs while also explaining incentives. He suggested two criteria for success regarding organizational participation in PID infrastructure. The first is reducing the level of effort required from individual researchers to participate, measuring the degree to which PIDs and metadata are used as a benchmark. A second criteria is better understanding the reuse of metadata that is available through PID structures, which could be benchmarked by assessing an institution’s output in this area. Polka added that another criteria for success could be holistic implementation of PIDs and requesting more complete metadata throughout the research life cycle, as current systems are tolerant of missing data. Her second criteria for success was the ability to create an audit trail to

determine provenance of information, which could be important for discerning trust and understanding links between stakeholders. A potential benchmark is a scorecard to hold organizations accountable for implementation.

Generating Interim Research Products

Polka shared approaches that centered on interim research products⁹ and promoting engagements with those products, as well as preparing data for maximum reuse. While the process of publishing can be separate from carrying out the research, there are opportunities to enable information sharing to better ensure research results reach the public arena. Furthermore, in the face of misinformation, she suggested that repositories, journals, and preprint servers could embed editorial process information in the metadata to help provide evidence about the types of assessments that have been done.

Reflections on Potential Approaches and Criteria for Success

Milagros Nores, Rutgers University, reviewed the highlights of the first breakout discussion, saying first that public access is tied to fundamental human rights and public good. There was also emphasis on transparency in research processes, with a need for more user-friendly systems and flexibility in approaches. She noted the importance of intentionality around what types of stakeholders are included, and thinking holistically about accessibility, discoverability, and populations lacking representation. Essential steps will be to reduce complexity, prioritize equity, and bring more patient groups and historically marginalized populations into the process to close engagement gaps and meet declared goals. Nores also noted that publishers are ready to be active partners in exploring new approaches to public access, including research result translatability, and may already be doing the work.

Darla Henderson, Federation of Societies for Experimental Biology, highlighted three themes from the second breakout discussion: the necessity for public access to research results in the United States goes hand in hand with scientific integrity and quality; comprehensive adoption of PIDs and open data should be embraced while respecting and including data owners and creators in the process; and lastly, there is a collective responsibility to improve the accessibility and usability of public access to research, emphasizing the need for clear frameworks for collaboration and minimizing burdens on participants. Additional suggestions made by individual speakers over the course of the workshop can be found in Box 1.

__________________

DISCLAIMER This Proceedings of a Workshop—in Brief has been prepared by Kyle Cavagnini, Carolyn Shore, and Megan Snair as a factual summary of what occurred at the meeting. The statements made are those of the rapporteurs or individual workshop participants and do not necessarily represent the views of all workshop participants; the planning committee; or the National Academies of Sciences, Engineering, and Medicine.

*The National Academies of Sciences, Engineering, and Medicine’s planning committees are solely responsible for organizing the workshop, identifying topics, and choosing speakers. The responsibility for the published Proceedings of a Workshop—in Brief rests with the institution.

REVIEWERS To ensure that it meets institutional standards for quality and objectivity, this Proceedings of a Workshop—in Brief was reviewed by Pamela Padilla, University of North Texas; Javier Robles, Rutgers, The State University of New Jersey; Cynthia Hudson Vitale, Association of Research Libraries. Leslie Sim, National Academies of Sciences, Engineering, and Medicine, served as the review coordinator.

SPONSOR This workshop was supported by the National Institutes of Health, contract number HHSN263201800029I, task order 75N98023F00010.

STAFF Tom Arrison, Director, Board on Research Data and Information; Clare Stroud, Director, Board on Health Sciences Policy; Carolyn Shore, Senior Program Officer; Emi Kameyama, Program Officer; Kyle Cavagnini, Associate Program Officer; Margaret McCarthy, Research Associate; Melvin Joppy, Senior Program Assistant.

For additional information regarding the workshop, visit https://www.nationalacademies.org/our-work/enhancing-public-access-to-the-results-of-research-supported-by-the-department-of-health-and-human-services-a-workshop

Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2024. Enhancing public access to the results of research supported by the Department of Health and Human Services: Proceedings of a workshop—in brief. Washington, DC: The National Academies Press. https://doi.org/10.17226/27480.