3

Priority Areas for the PCOR Data Infrastructure

In this chapter we discuss the conclusions reached by the committee based on the information-gathering activities carried out over the course of the study and on the committee’s subsequent deliberations and integrated judgment. Key information-gathering activities included (1) reviewing background documentation about the Assistant Secretary for Planning and Evaluation’s (ASPE’s) work, such as the reports discussed in Chapter 2; (2) public meetings with the U.S. Department of Health and Human Services (HHS) staff; and (3) three public workshops focused on gathering input from stakeholders. Appendixes B through D provide detailed summaries of the workshops, including the input received from the participants and the conclusions reached by the committee on the topics that were discussed.

This chapter focuses on aspects of the patient-centered outcomes research (PCOR) data infrastructure that ultimately emerged as areas that could benefit if prioritized as part of ASPE’s work over the next decade. The topics highlighted touch on many of the key elements of the framework currently used by ASPE to manage work related to the PCOR data infrastructure (see Figure 2-1). The chapter also offers additional input on strengthening the overall framework for building the data infrastructure over the coming years.

The committee’s charge was to identify issues critical to building data capacity for PCOR and for generating new evidence to inform health care decisions. In its deliberations, the committee noted the limitations of focusing solely on patients and concluded that broadening the focus to persons would increase the likelihood that ASPE could develop an infrastructure for research that addresses the priorities identified in the authorizing legislation,

by those who participated in workshops, and from the committee members’ own experiences in answering questions of interest to people. This distinction between people and patients is not intended to imply that these roles are necessarily separate but rather to underscore the broader context that might be missed when the focus is exclusively on the patient. Furthermore, a desire to broaden the focus does not imply that the challenges associated with the data infrastructure for PCOR have been addressed. As the remainder of this chapter will illustrate, many of the continuing challenges are best addressed in the broader context.

The committee uses the following definitions:

• People refers to individuals throughout their life course, whether or not they are interacting with the health services system or have had a clinical diagnosis. A focus on people would enable ASPE to identify the rich set of attributes by which people define themselves and would provide the larger context within which people seek medical intervention for health care problems. The broader context includes people’s communities, in other words the health impacts of where individuals live, work, and play. Focusing on people would also be useful for identifying challenges and opportunities in optimizing health.
• Patients refers to people who are interacting with the health services system to obtain a diagnosis or treatment for acute or chronic conditions, to manage symptoms that affect their quality of life, or for advice and interventions aimed at preventing disease or enhancing health. In the committee’s definitions, patients are people first, but their priorities and preferences may change in response to a new diagnosis or other change in health status.

The committee also offers a first conclusion, which resulted from a recurring theme that emerged from the workshops organized by the committee and from the committee’s subsequent deliberations integrating all the input. This conclusion, which serves as the basis of the new approach described above and represents a shift in perspective, could be transformative for both research and health care.

CONCLUSION 3-1: Broadening the focus from the patient to the person more generally and from populations to communities would enable a more comprehensive approach to the data infrastructure and a better understanding of the outcomes that matter to people.

INCLUDING HIGH-PRIORITY TYPES OF DATA IN THE DATA INFRASTRUCTURE

As Figure 2-1 shows, the data serve as the foundation of the PCOR data infrastructure. Data needs and priorities have evolved over the years on the basis of advances in science and policy considerations. The most recent reauthorization of the PCOR Trust Fund highlighted several new areas as priorities, including intellectual and developmental disabilities, maternal mortality, and health care cost as a consideration among the full range of outcomes data.

The committee’s first workshop focused on gathering input on emerging data needs and priorities from stakeholders, including researchers and representatives of patient organizations. Among the goals of the workshop were to identify key questions that stakeholders are most likely to want answered going forward and to identify gaps between what stakeholders need and what the data infrastructure allows.

A theme that emerged from the workshop was the magnitude of the gaps in the data on health disparities. Health disparities represent an evolving and expanding area of research, with corresponding data needs and data limitations that affect the ability to identify, understand, and address these disparities. Limitations exist for a variety of social determinants of health data and for a range of populations and communities. Data for specific populations are sometimes unavailable or are not representative. In other cases, the data might not be timely or might have other gaps that make it difficult to understand the impact of changes over long periods of time. The committee’s first interim report (see especially Appendix B, Chapter 2) discusses these limitations in further detail and highlights a variety of examples, ranging from gaps in the data available on disabilities to those concerning sexual and gender minorities and racial and ethnic groups. Improving the data available for understanding and addressing these disparities would require data strategies that prioritize these data needs.

The cost of care emerged as another topic area where there are substantial gaps in the information available. Both patient groups and researchers highlighted the clear need for more data on the total cost of care and a better understanding of cost considerations and their impact on the quality and length of life (see Appendix B, Chapters 3 and 4, as well as Appendix D, Chapter 2). Given the limitations on accessing health care delivery and the high cost of health care relative to the financial resources available to families in the United States, the ability to access care and to pay for that care can face barriers that impact the length and quality of people’s lives.

The committee’s third workshop, focused on identifying ways of enhancing collaborations, data linkages, and the interoperability of electronic databases, highlighted the particular need for and the underutilized

potential in performing linkages to mortality data (see Appendix D, Chapter 2). Death records from the Social Security Administration are often challenging to access, and the Centers for Disease Control and Prevention’s National Death Index database only makes data available with a substantial time lag and at a high cost. This means that researchers have limited ability to use mortality as an outcome measure in studies. The delayed access to, limited reliability of, and high costs of accessing mortality data impede PCOR and decrease the reliability of the findings related to this important outcome. Given that these data already exist, this might be an area where enhancements to the data infrastructure could yield an impact.

The committee’s review of the Office of the Secretary PCOR Trust Fund (OS-PCORTF) data infrastructure projects, summarized in Chapter 2 and Appendix E, shows that ongoing work related to the areas identified are particularly important to stakeholders. However, the examination of whether the PCOR data infrastructure currently contains the data elements needed to study and draw conclusions about optimizing outcomes for people and patients revealed that it does not. Existing data do not capture the richness of people’s characteristics and experiences, and some specific areas require more attention. Data types that have more recently been authorized through legislation and other priority-setting activities (e.g., social determinants of health, disability, and cost of care) are less likely to be available in the PCOR data infrastructure.

While such limitations are to be expected, opportunities exist for capturing data that are better able to characterize these complexities. A robust data infrastructure builds on the strengths of what is available today and has the flexibility to adapt, both as measures and terminologies become obsolete and as new technologies emerge. The input received as part of the workshops organized by the committee underscored topics that would be particularly worthwhile to prioritize in the near future. This input echoed findings from the stakeholder prioritization activity¹ and the most recent assessment of the portfolio of PCOR data infrastructure projects.²

CONCLUSION 3-2: A variety of data types were identified that are less likely to be available or easily accessible in the PCOR data infrastructure, including data on mortality, cost of care, social determinants of health, and disability status, as well as other characteristics of people associated with disparities in health outcomes. Increased attention to filling gaps in the availability of these data will enhance the utility of

___________________

¹https://aspe.hhs.gov/sites/default/files/migrated_legacy_files//197426/PCOR-Data-Infrastructure.pdf.

²https://aspe.hhs.gov/sites/default/files/migrated_legacy_files//194781/OS-PCORTFImpactReport508.pdf.

the infrastructure for answering questions that matter to people and will enable research on potential intervention targets.

A fundamental reason for the data limitations that make it difficult to answer questions important for PCOR is that most of the data available for research are not primarily collected for research purposes. While research questions require a relational or integrative perspective, the data collected tend to be transactional, that is, collected for payment or treatment purposes. This has several implications for the data available. First, the data collected are typically limited to or organized within subpopulations (e.g., those insured by Medicaid or Medicare). People who are uninsured, including those who have limited access or interactions with health care services, are likely to be underrepresented in many databases. Second, the information collected (and what is not collected) often makes the datasets poorly suited to answer a variety of research questions. Third, those who collect the data do not have an incentive to collect information in a way that is useful for secondary purposes, such as research. These realities present a challenge to answering critical questions, but the workshops also made it clear that opportunities exist to make the data infrastructure more suitable for answering questions of interest, so long as the potential uses of the data are carefully considered.

Much of the focus of the PCOR data infrastructure to date has been on data generated in the course of patient care or in federally funded research studies. The increasing use of person-generated data from various sources was a recurring theme throughout the workshops (see Appendixes C and D in particular). A focus on person-centered outcomes will enable an expansion of opportunities to incorporate person-generated data in the PCOR infrastructure. Integrating these types of data into PCOR will enhance learning about aspects of people’s lives that affect health but do not involve direct engagement with the health services system, including relevant measures of how well people are doing. The fact that this is an emerging area also means that in many domains, research on the validity and reliability of these types of measures is still in its early stages. Additional work and scientific testing are needed to ensure that the measures are useful for their intended purposes.

CONCLUSION 3-3: An area with opportunities for additional expansion is the collection of patient- and person-generated data and the routine integration of these data into data platforms that can be used both for research and for other purposes, including regulatory decision making and to inform shared decision making.

One source of additional data that might not otherwise be available to researchers is disease registries directed by patient groups. Disease registries

are compilations of data from a variety of other sources and are typically focused on a specific condition or diagnosis. Collaborations with patient organizations can help facilitate access to these types of databases, build patient engagement, and address patient concerns about participating in research studies, all important aspects of achieving a person-centered approach. For additional details on the input received by the committee on this topic, see in particular the discussion on patient data needs in Appendix B, Chapter 3, and on collaborations with patient groups in Appendix D, Chapter 6.

CONCLUSION 3-4: Patient-directed disease registries can be a source of in-depth, longitudinal, prospective clinical and patient-reported data that are not available from other data sources.

One topic that emerged from the committee’s second workshop, which included discussions on approaches and methods that could move PCOR forward, was the potential usefulness of adopting a longitudinal perspective on a person’s journey through the health care system and, more broadly, through life events that have a relevance to health (see Appendix C, Chapter 3). The fact that the data available for PCOR are typically not collected primarily for research poses challenges in this area as well, but these challenges could gradually be overcome by addressing the broader limitations of the data infrastructure discussed throughout this chapter, including: the lack of integration of data across the research data ecosystem; the lack of unique health identifiers; and the underutilization of certain sources of data, such as person-generated data and real-world clinical data.

CONCLUSION 3-5: Assembling a comprehensive longitudinal record of individuals’ health journeys, which also includes the social context of their lives to the extent possible, would facilitate more far-reaching outcomes research.

ADDRESSING FRAGMENTATION

The health care system in the United States is fragmented, resulting in fragmented data, and the PCOR data infrastructure aspires to overcome some of this fragmentation. The data that exist for PCOR are collected and curated in a variety of databases across the health system. These databases are typically constructed to serve their primary purposes and do not prioritize the ability to link with other databases. Other relevant data, such as data on social determinants of health, sometimes exist only outside of these databases or do not exist at all. The workshops identified these data silos as a major barrier both to research and to increasing the usefulness of the information available for decision making more broadly.

A prolific area of work aimed at facilitating data linkages has been focused on developing technical approaches to address data privacy concerns. Tokenization solutions, for example, assign unique keys to the records that are being linked, enabling the linkage to happen without the sharing of patient identifiers. Competition in this area encourages innovation, but the workshops also revealed the resulting challenges for the PCOR data infrastructure (see Appendix C, Chapters 2 and 4, and Appendix D, Chapter 2). Each approach has unique characteristics, and researchers might favor one over others. Furthermore, the approaches that exist are constantly evolving and cannot always be relied on to work in the future. Some solutions are proprietary, and data that use one method cannot be linked to data that use another method.

CONCLUSION 3-6: The data available for PCOR are fragmented across a variety of databases. Expanding data linkages could greatly increase the usefulness of these data for research.

The workshops highlighted the importance of collaborations among federal agencies and other partners (such as states, patient groups, and others) to continue to build a comprehensive data infrastructure based on fragmented data (see especially Appendix D). These collaborations can be particularly useful in addressing barriers that hinder data linkages, such as the limitations associated with person-level unique health identifiers and patient matching. Other areas identified where collaborations could further mitigate the challenges associated with data fragmentation include increasing consistency in the use of standards for data interoperability and element definitions and working with stakeholders and patients to promote data sharing.

State data collection systems are an especially rich source of detailed data that can be useful to state and local policy makers (see Appendix D, Chapter 3). State-generated data are also valuable at the national level, including for answering broader questions about issues that may be influenced by local policy, such as health care access and disparities. While many states have robust data collection systems, the data collected, their quality, and ease of access to the data all vary by state. Challenges associated with access make the use of state-generated data for research at the national level particularly difficult.

CONCLUSION 3-7: Collaboration among federal agencies and between federal agencies and other partners to address barriers that hinder data linkages, such as the limitations associated with the lack of unique health identifiers and patient or person matching, will improve the PCOR data infrastructure. The usefulness of data available for

PCOR could further be increased by sharing and adopting best practices among the states concerning the collection of data, data quality, and ease of access.

DATA NOT DESIGNED TO ADVANCE KNOWLEDGE

As discussed above, the PCOR data infrastructure relies primarily on data that were not collected primarily for research purposes, and in particular for patient- or person-centered research. Much of the work aimed to facilitate the use of these data for research is focused on the development of standards for clinical data and on enhancing the interoperability of data systems. ASPE has focused on “data standards” as one of the building blocks, and the “standardized collection of standardized clinical data” as one of the pillars in the PCOR data infrastructure framework to improve the data infrastructure (Figure 2-1). As a result, many of the projects funded by the PCOR Trust Fund have been centered on these topics. The previous stakeholder prioritization activity also concluded that additional work on standards should be a priority.

The workshops organized by the committee demonstrated that, within the context of PCOR, standardization is increasingly being applied to collecting, storing, analyzing, and exchanging data. Improvements in clinical data standards, in particular, and the adoption of interoperable systems are improving the availability and usefulness of these data for research.

One of the themes that emerged from the workshops is that standards are most useful when they address a specific problem or are driven by a specific use case (see Appendix C, Chapter 2). Because needs and norms evolve over time, standards need to evolve too. The work on standards that is happening across the globe holds lessons for PCOR in general, and lessons might also be learned from best practices that emerge for the development of standards.

The workshops also made it clear that in some areas there is a fair amount of agreement around what standards are needed and what useful standards look like. In other areas, such as for data on the social determinants of health, the work is just beginning, so wide agreement on standards may not yet be possible. In all cases, extensive testing of the potential standards is necessary.

CONCLUSION 3-8: Standards are most useful when their development is driven by their potential uses and a clear concept of the value they can contribute.

CONCLUSION 3-9: Taking an international perspective is important for the development of a PCOR data infrastructure; in particular, the

infrastructure focused on standards would benefit from building on work that happens internationally.

Workshop participants did not see a large role for ASPE in selecting areas for or developing standards (see Appendix C, Chapter 2). ASPE’s most valuable contributions in this area could be in developing an architecture and an implementation strategy that facilitate common language and interoperability across datasets as well as accessibility of the data. Other areas where ASPE could play an important role include convening stakeholder meetings to discuss and develop standards and taking the lead in cataloging existing standards.

CONCLUSION 3-10: ASPE, in collaboration with other partners and stakeholders, could add significant value in the area of standards for PCOR by:

• continuing to promote the development of a data infrastructure and an implementation strategy that facilitate the use of standards and access to the data;
• convening stakeholder meetings to enhance communication and work toward developing a common language for standards;
• facilitating access to the data and collaborations with existing organizations working in this area;
• leading efforts to catalogue and exemplify data standards and analytic standards for a holistic view of individuals’ health; and
• increasing consistency in the use of standards for data interoperability and element definitions.

CONCLUSION 3-11: Prioritizing projects that address fidelity or use of standards may convey greater value for the PCOR infrastructure than developing new standards.

GOVERNING DATA ACCESS

The way data are accessed, used, and shared for research is governed by privacy laws. The most relevant of these laws are the following four:

1. The Health Insurance Portability and Accountability Act of 1996 (HIPAA);
2. “Part 2”, which relates to regulations on substance-abuse data confidentiality;
3. The Family Educational Rights and Privacy Act (FERPA), which covers educational institutions; and
4. The Privacy Act, which covers federal government data resources.

The law that has the most impact on PCOR data is HIPAA, which does not cover all health data but does cover identifiable data from most doctors, hospitals, health plans, and contractors. While identifiable data can be used for research under certain conditions, HIPAA was primarily designed to enable data flows within a health care system rather than to address research considerations. New “information blocking” rules that were set forth by the 21st Century Cures Act³ and went into effect in April 2021 shift the default to the sharing of electronic health information for any lawful purpose, including research. Information blocking is defined as

While there are penalties for information blocking, the rules around enforcement were still under discussion during the writing of this report. A more detailed discussion of the privacy laws applicable to PCOR data is included in Chapter 4 of Appendix C (see in particular the overview presented by Deven McGraw).

As discussed above, some of the work in this area focuses on technical and methodological solutions to improve data access while preserving privacy. The Blue Button initiative is an additional effort that facilitates data sharing, primarily by providing a mechanism that enables people to download their own health records and then potentially share them with others for a variety of purposes, including research.⁵

Data access and privacy considerations were a recurring theme during the committee’s information-gathering activities (see especially Appendix C, Chapter 4, and Appendix D, Chapter 2). Challenges associated with various approaches to obtaining consent were also discussed as an area that needs more work, in light of secondary uses of data that may not be clear to patients. At the same time, it appears that relatively few OS-PCORTF projects have been funded in this area. The workshops made it clear that there are concerns about the laws and rules governing data access and data sharing.

HIPAA, in particular, was developed several decades ago, and its approach to setting thresholds for data disclosures makes it outdated. Moreover, interpretations of how HIPAA applies to particular situations are

___________________

³https://www.healthit.gov/curesrule/.

⁴https://www.healthit.gov/topic/information-blocking.

⁵https://www.healthit.gov/topic/health-it-initiatives/blue-button.

often inconsistent. There is a need for a new framework with guardrails that balance the risk of disclosure with the need for research that improves people’s health. This includes a need for a critical review of current privacy legislation, an understanding of public perspectives, and the development of recommendations for revisions or reform that would be applicable to the protection of health data in the Internet-enabled world, with a focus on preventing misuses of the data.

CONCLUSION 3-12: This is an opportune time to revisit and update the legislation and rules governing data privacy and the sharing of data for research.

CONCLUSION 3-13: Governance challenges that create barriers to developing the PCOR infrastructure can be found at all levels of the system. Data availability could be increased by exploring challenges at the local level, including variable interpretations of federal laws and regulations, and by identifying approaches to address those challenges.

DATA ACCESS OPTIONS

Although it is not unique to PCOR, there is an ongoing challenge in finding the right balance between protecting privacy and enabling increased use of data for research. The workshops touched on a variety of mechanisms for accessing PCOR-relevant data that endeavor to navigate this challenge (see Appendix C, Chapter 4, and Appendix D, Chapter 2). Some examples include Special Sworn Status for individuals; Centers for Medicare & Medicaid Services enclave; network participation; group access (for researchers, patients/people, clinicians, policy makers); and Master Data Use agreements. There was also discussion of the benefits possible when mature projects share with others how they have managed access. These processes could all benefit from additional streamlining to facilitate data use.

CONCLUSION 3-14: Investments in identifying mechanisms for facilitating the ability of researchers, patients, and other people to access data will contribute to increased use of the PCOR infrastructure.

While improving data access is essential, another theme raised by stakeholders was the importance of being transparent about how the data will be used (see especially Appendix B, Chapters 3 and 4). Workshop participants emphasized the need to involve the people whose data are being used, as well as their communities, in decisions at each stage of the process. Building and maintaining trust with those whose data are being sought is essential to

ensure that the data obtained are representative, complete, and reliable. This is especially important when the data could be perceived as sensitive, as is the case with some of the data on the social determinants of health. Collaborations with patient organizations can help in addressing patient concerns about participating in research studies and in building patient engagement, which are both important for achieving a patient-centered approach.

Researchers echoed the need to make PCOR data more widely available to empower patients and communities to use this information. Efforts to reduce disparities, in particular, cannot be accomplished by research alone. Clinicians are a key stakeholder group that generates a large portion of the data and also uses the data. Ensuring that relevant and easy-to-use PCOR data are available to clinicians is essential for obtaining their cooperation in producing high-quality data.

CONCLUSION 3-15: Building and maintaining trust among the people and communities whose data are being sought for research is essential for producing high-quality data, and patient groups can be helpful partners in these efforts. Including representatives of patients and other people in the research process to understand how to measure health impacts that matter to individuals is an important component in building trust. Providing value back to data donors, such as through the sharing of research results, could help underscore the importance and benefits of the information to stakeholders, including individuals, families, clinicians, and communities, in addition to enabling them to use the information in ways they find relevant. These uses could play a particularly important role in reducing health disparities, complementing research efforts in this area.

While the benefits of data sharing are clear, the workshops also highlighted some of the reasons behind reluctance to share and underscored the risks involved for the organizations providing the data (see the discussion in Appendix D, Chapter 5). Successful data sharing agreements can be established when these factors are taken into consideration.

CONCLUSION 3-16: Successful data sharing partnerships across health care systems and government agencies require participant trust, clear evidence of mutual benefit, and the ability to control risk.

ADVANCING RESEARCH PRACTICES AND ANALYTIC METHODS

Some of the committee’s input-gathering activities focused on advances in the methods used for PCOR and on identifying areas that could benefit from additional attention in the coming years to facilitate research (see

Appendix C, Chapter 3). Workshop participants emphasized the need for transparency and for consideration of related scientific principles, such as reproducibility of the data and methods used for PCOR. These considerations are important for all types of data and analysis, but the increasing use of tools such as machine learning and natural language processing raises the question of whether best practices can ensure that these tools do not introduce biases that hardwire current disparities in care delivery (e.g., algorithmic bias).

The workshops also illustrated the importance of highlighting best practices, not only for how results are shared but also for the sharing of other resources and components associated with the research process, such as the software developed for analyses. Sharing all these resources ensures that the data can be widely used and that the research can be replicated. Ultimately the goal of PCOR is to benefit people, so the questions of what happens to the research after it is completed and how the information is shared with those whose data are being used also deserve further attention.

CONCLUSION 3-17: PCOR products would be enhanced by investing in methods that are essential for the conduct of PCOR, such as including persons throughout the research continuum, addressing problems of missing data, improving study designs, ensuring appropriate inference from methods utilizing observational data, and addressing structural bias in data systems and studies.

CONCLUSION 3-18: Applying best practices to the analytic methods used in PCOR is important to facilitate the reliability and reproducibility of study results.

CONCLUSION 3-19: The results of PCOR are only replicable and most useful when the underlying data and comprehensive research documentation (such as analytic code) are made available for use by others.

PROJECT SELECTION TO SUPPORT THE DATA INFRASTRUCTURE FRAMEWORK

While previous sections discussed specific aspects of the PCOR data infrastructure that could benefit from additional work in the years ahead, this section offers a few general, big-picture observations on project selection. ASPE sets priorities for the OS-PCORTF with assistance primarily from a Leadership Council that consists of HHS agency heads and their designees. The Leadership Council also provides input on the selection of projects to fund. While ASPE regularly obtains additional input from others,

such as researchers and patient advocates, the question arises whether their lack of representation on the OS-PCORTF Leadership Council limits critical input on data infrastructure needs.

Although it is clear that the projects funded through the OS-PCORTF are well targeted toward further developing and enhancing key aspects of the data infrastructure, the PCOR data infrastructure has not reached its full potential to provide data that can answer questions that matter to people and enable them to make informed decisions. The committee’s work identified several areas that could benefit from additional attention. The emergence of new technologies and approaches (e.g., wearable devices, artificial intelligence, and privacy-preserving record-linkage solutions) are also continuing to present new opportunities for enhancing the data infrastructure.

Leaders from patient organizations that participated in the workshops noted that PCOR data are often not focused on the types of issues that are truly important to people and that would enable them to find answers to the questions they tend to have about their treatment options and potential outcomes (see Appendix B, Chapter 3). The lack of information on the costs of care was highlighted as an example, which is also in line with the goals of the recently broadened scope of PCOR, namely to take into consideration “the potential burdens and economic impacts of the utilization of medical treatments, items, and services.”

The workshops also made it clear that the data available do not capture complexities that need to be measured to understand how people’s characteristics and experiences influence health outcomes (see Appendix B, Chapter 2). There was an emphasis on the need to build flexibility into the data collection systems to allow them to adapt to evolving terminologies and technologies for capturing and processing data. This is particularly important for social determinants of health, an area that may be rapidly changing.

The workshops also revealed that the policy and regulatory aspects of data sharing and privacy protection need updating (see Appendix C, Chapter 4). To date, these issues have received less direct attention in the context of the PCOR data infrastructure work. Arguably, these questions need to be addressed at a broader level, but ASPE may have a role in facilitating some of the discussions by proactively initiating work focused on these topics.

Previous sections in this chapter touched on some aspects of the issues mentioned above, particularly in the context of emerging priorities for health data. These topics are also highlighted here because the ability to address them has implications for the overall approach to the data infrastructure framework. Beyond the emerging data needs identified, the broader context for the data infrastructure is continuously evolving, due to larger societal shifts, emerging public health concerns (e.g., infectious diseases), new medical treatments (e.g., gene immunotherapy), and other

developments. Nimbleness, mechanisms built in for stakeholder engagement, and transparent decision making are the key characteristics necessary for data infrastructure development.

CONCLUSION 3-20: The development of the data infrastructure might be enhanced and critical gaps could be filled by proactively identifying necessary projects in areas that examine the overall framework for the PCOR data infrastructure, particularly in the context of broader issues such as the balance between privacy and increased data use.

The discussions with stakeholders and the review of prior evaluations led the committee to identify two general categories of projects that may be particularly worthwhile to pursue because of a high expected return on the investment. One such category consists of projects focused on topics or aspects of the infrastructure that are unlikely to be funded by other entities, such as other agencies or the private market. HHS is uniquely well positioned to address needs related to the data infrastructure that would otherwise be unmet. The other category covers projects that could result in findings that have application beyond a single condition or disease. These types of projects are most likely to lead to reuse of the data and broader use of the data infrastructure in general.

CONCLUSION 3-21: Investments in areas unlikely to be funded or developed by other entities may have a particularly high value.

CONCLUSION 3-22: Investments in projects that have potential use and application beyond the condition or disease for which they are proposed will accelerate the use of the infrastructure.

DISSEMINATION OF RESULTS AND USE OF THE DATA INFRASTRUCTURE

Over the years, efforts to disseminate information about the PCOR data infrastructure and PCOR studies have increased. The discussions with stakeholders, however, revealed that awareness of the work could be increased by additional attention to this area. Dissemination efforts focused on external stakeholders, in particular, could increase the usefulness of the investments made in the projects by encouraging additional engagement and reuse of the data and methods (see Appendix B, Chapter 3, and Appendix D, Chapter 2).

CONCLUSION 3-23: There is a need to increase awareness among all stakeholders about new data infrastructure developments funded

by the Office of the Secretary PCOR Trust Fund. Increased awareness will enhance the efficiency and effectiveness of research, which in turn will increase the impact of the investments made in infrastructure development.

CONCLUSION 3-24: Investments in implementing and disseminating infrastructure tools and products will accelerate the achievement of overall PCOR infrastructure goals.

Input from patient advocacy groups highlighted the need for additional dissemination focused on diverse stakeholders outside the research community (see Appendix B, Chapter 3). While data on patient-centered outcomes are available in many areas that are important to patients, the information is rarely available in ways that would make it truly accessible to them. More widespread dissemination of information that is easy to use could also increase engagement.

CONCLUSION 3-25: Dissemination and translation of the research findings could be greatly enhanced by using forms of communication that are relevant to those outside the research community.

UPDATING THE DATA INFRASTRUCTURE FRAMEWORK

Building on what was learned from the workshop, and on the basis of the committee’s further deliberations, this section offers a few additional observations focused on strengthening the overall framework for building the data infrastructure in the coming years. As discussed in Chapter 2, Figure 2-1 is a visual representation of the framework ASPE has been using for the PCOR data infrastructure to operationalize the mandate for the OS-PCORTF and guide decision making. This representation of the key elements of the data infrastructure allows ASPE to keep track of how funds are invested and which aspects of the infrastructure are enhanced.

ASPE’s framework provides a description of the components, but it may not be fully actionable. A fragmented health care system and the resulting fragmented nature of the data available for research continue to be a barrier to developing insights that could ultimately improve health, which is the fundamental motivation for PCOR. The recent experience of the COVID-19 pandemic illustrates how this fragmentation has hindered the development of policies and solutions to address challenges.

Figure 3-1 provides an illustration of the committee’s perspective on the data infrastructure framework. This is not intended to replace the existing framework but rather to present an additional view that is explicitly focused on improved health as the goal. Improving health is the North Star

that guides the selection of projects and affects every decision. Reducing “voltage drops” at each step in the framework can contribute to realizing the potential of the PCOR data infrastructure. The most challenging need to address is likely between the development of insights and improving health, as shown on the left-hand side of the figure, which can be interpreted broadly to include informing policies, regulations, laws, and other mechanisms that drive improvements.

As discussed earlier, stakeholder input is important at each step. It is also important to note that the steps on the left-hand side of the figure go well beyond what can be accomplished through a solid data infrastructure alone, and therefore might not be directly achievable by ASPE or even by HHS. However, it is important to keep this broader context in mind to the extent that decisions about the data infrastructure are able to have an impact on progress toward the overall goal.

CONCLUSION 3-26: Explicitly focusing on improved health as the goal of the PCOR infrastructure may be a useful way to prioritize projects and target infrastructure investments.

Finally, the committee’s deliberations echo the findings of the most recent portfolio assessment concerning the need for additional feedback and metrics that would enable ASPE to assess the impact of the investments in the data infrastructure. ASPE tracks information about what has been produced by each project, showing publications, presentations, and other public resources, as well as narrative summaries of accomplishments, as part of the description of completed projects on its website. Measuring impact (what happened as a result of the work) is always a challenging task, but obtaining additional feedback from stakeholders and building a strong feedback loop that attempts to ascertain what types of evidence have been generated as a result of the work could provide additional insights and strengthen the overall framework for the data capacity infrastructure.

CONCLUSION 3-27: A tighter feedback loop with the external end users and developers of evidence would enhance the value of data infrastructure investments. Examining what evidence was generated due to ASPE interventions and identifying what impact it had on policy and knowledge would also help close the gaps observed in realizing the potential of the PCOR data infrastructure.