2

Federal Partnerships

This chapter summarizes workshop presentations and discussion focused on federal partnerships and collaborations with other stakeholder groups. The brief overview of the input received from the presenters is followed by the committee’s conclusions.

Micky Tripathi, National Coordinator for Health Information Technology, discussed the work that the Office of the National Coordinator for Health Information Technology (ONC) is doing to support activities related to patient-centered outcomes research (PCOR). He noted that more than a decade after the passage of the Health Information Technology for Economic and Clinical Health Act of 2009, the United States has several well-functioning electronic health records (EHR) systems that are widely used by health care providers and represent a common foundation for capturing and sharing clinical data, even if not all providers are satisfied with their EHRs. Tripathi argued that this is a good time to consider the opportunities offered by EHRs, beyond their core functions of supporting medical records and payment processing, stating that EHRs are still a barely tapped source of information for research.

One area of focus for ONC in recent years has been the advancement and harmonization of data standards, including supporting the use of common data models, looking at research-data models as well as clinical data-models, and thinking about data integration. ONC has supported work on patient matching, aggregating, and linking through the development of tools and advancement of data standards. Tripathi also mentioned structured data capture as another area of past work aimed at facilitating the reuse of these data for clinical research.

Another area of work at ONC focuses on real-world data. Projects included testing a standards-based approach to establishing a coordinated registry network for data regarding women’s health technologies, collecting patient-reported outcomes (PROs) through health IT and leveraging Fast Healthcare Interoperability Resources (FHIR), and using privacy preserving machine learning techniques to enable health information exchanges to support COVID-19-focused PCOR.

Tripathi said that ONC has done work related to creating a privacy and security framework for PCOR by developing resources that support the protection of privacy and the security of electronic health data. The agency also worked on identifying limitations in developing machine learning training datasets when pursuing complex health related research questions. ONC also collaborated with MITRE Corporation to increase the variety of reliable and robust synthetic data, particularly for opioid, pediatric, and complex care use cases. The project focuses on enhancing an open-source synthetic data engine that uses publicly available data to generate synthetic health records. Such tools can safeguard patient privacy and support appropriate stewardship practices in which real patient data are only accessed and used when necessary. These and other mentioned projects have been funded through the Patient-Centered Outcomes Research Trust Fund (PCORTF).¹

Looking ahead, Tripathi said that ONC has several initiatives that will directly support PCOR and patient engagement in PCOR. One initiative is focused on standards for EHR data. ONC will be releasing a new version of the U.S. Core Data for Interoperability, which is a standardized set of health data classes and constituent data elements for EHRs and for nationwide, interoperable health information exchanges. The data are supposed to be available not only for provider-to-provider and provider-to-payer exchanges but also for patients.

Allison Oelschlaeger, Office of Enterprise Data & Analytics, Centers for Medicare & Medicaid Services (CMS), discussed CMS data and data linkages. For context, Oelschlaeger noted that more than 130 million Americans receive health coverage through programs administered by CMS—including Medicare, Medicaid, and the Children’s Health Insurance Program—and the health insurance marketplace. As a result, CMS collects large amounts of data, which are an invaluable resource for comparative clinical effectiveness research. The data include information on enrollment and patient characteristics; utilization and cost of health care services, such as treatments and therapies; and diagnoses.

___________________

¹ For more information on some of the ONC PCOR data infrastructure projects, see https://www.healthit.gov/topic/scientific-initiatives/building-data-infrastructure-support-patient-centered-outcomes-research.

Despite the wealth of data collected by CMS, Oelschlaeger said this information is often insufficient on its own for research on clinical effectiveness. Much information is missing, including

• cause of death data;
• clinical data (for example, lab results);
• certain patient demographic data (for example, income and high-quality race/ethnicity data);
• data related to social determinants of health;
• health behaviors data;
• patient-generated health information; and
• human services data.

Oelschlaeger said that options for enhancing the CMS data with the missing information could include undertaking new data collection, imputation, or linkages to other data sources.

The PCOR Trust Fund funded several CMS projects throughout the years. One of these projects was Blue Button 2.0,² which is an application programming interface (API) that allows beneficiaries to connect their data to applications and services they trust and enables them to contribute their data to research projects. Blue Button leverages FHIR standards, also discussed by Tripathi. CMS data linkage projects that have been funded by the PCOR Trust Fund include³

• Augmenting the National Hospital Care Survey Data through Linkages with Administrative Records;
• National COVID-19 Longitudinal Research Database, which is linked to CMS data; and
• National Death Index—Medicare Enrollment Data Linkage.

Oelschlaeger said that CMS also has other projects funded through other agencies to link CMS data and data from other sources, including

• Surveillance, Epidemiology, and End Results;
• U.S. Renal Data System;
• Health and Retirement Survey; and
• National Health and Aging Trends Study.

___________________

²https://bluebutton.cms.gov/.

³ For more information on specific projects, see https://aspe.hhs.gov/collaborations-committees-advisory-groups/os-pcortf/explore-portfolio.

Oelschlaeger discussed several barriers to data linkages that involve CMS data. She said that for privacy reasons, personal information is collected less often, and datasets are increasingly lacking patient identifiers that are sufficient for linkages. Private and proprietary datasets, such as registries, have additional limitations for sharing identifiers. She also noted that the volume of patient records necessary to access in order to perform the linkages is often larger than what is needed once the linkage is complete.

According to Oelschlaeger, researchers increasingly want to use tokenization solutions. These are methods that assign unique keys to the datasets being linked, enabling the linkage to happen without the sharing of patient identifiers. The challenge CMS encountered is that there are a variety of tokenization solutions, and researchers have a variety of preferences about what to use. Oelschlaeger noted that the lack of consistency (the differences in degree of difficulty across different populations) is also a challenge in probabilistic matching, specifically the question of how good the match needs to be in order for the linkage to happen.

Oelschlaeger also discussed the CMS Virtual Research Data Center (VRDC), which is meant to provide a secure and efficient mechanism for researchers to virtually access and analyze CMS data and potentially address some of the challenges. The VRDC allows researchers to access CMS data and perform their own analyses and manipulation of those data virtually, from their independent workstations, and it allows them to download aggregate results from the analyses they perform. The VRDC enables faster access at a lower cost to the CMS data that are already linked by a unique beneficiary ID. Researchers also have the ability to upload other data to the VRDC to perform additional linkages.

Meagan Khau, CMS Office of Minority Health, described the mission of her office as serving as the principal advisor to the agency on the needs of minority populations, including racial and ethnic minorities; people with disabilities; the LGBTQ+ community; individuals with limited English proficiency; rural populations; and persons otherwise adversely affected by persistent poverty or inequality. She highlighted two sections of Executive Order 13985, which are focused on the role of data in meeting the needs of these populations. Specifically, Section 4(a) focuses on methods for assessing equity, which underscores the need to collect demographic data in order to fully assess the extent of existing health disparities and the impact of health equity responses. Section 9(a) establishes a workgroup to gather necessary data to measure and advance equity.

Khau noted that federal datasets often are not disaggregated by key demographic variables, such as race, ethnicity, gender, disability, income, and veteran status. CMS is working on gaining a better understanding of what datasets contain data elements of interest, and what standards are being used. As an example, for race, the Office of Management and Budget

1997 standards use five categories, whereas the 2011 Department of Health and Human Services (HHS) standards⁴ use 14 categories. Various standards and approaches are used in various EHRs.

Detailed, disaggregated data are needed to target interventions, in Khau’s view. As an example (Figure 2-1), she noted that in 2018 the prevalence of diabetes in the population as a whole was around 11 percent. Using five race categories, the prevalence was highest among Native Hawaiian and other Pacific Islanders, at 15 percent. However, when the data are further disaggregated, it becomes clear that within this group the prevalence of diabetes is particularly high among Samoans, at 22 percent.

In terms of the ability to assess equity with respect to demographic data elements identified in Executive Order 13985, such as race, ethnicity, religion, income, geography, gender identity, sexual orientation, and disability, Khau pointed out that not all these data elements are collected across the different HHS programs. This means that in some cases new data collections may be necessary, and it will be important to apply the right standards to each of the data elements, in light of how the data are anticipated to be used.

Khau highlighted several projects that the CMS Office of Minority Health is working on, using CMS data. The Mapping Medicare Disparities Tool⁵ is an interactive map that allows users to identify areas of disparities between subgroups of Medicare beneficiaries (for example, by race or ethnicity), chronic disease prevalence, health outcomes, spending, and utilization. The tool uses Medicare fee-for-service data. The office has also released two reports that use stratified data to look at health disparities.⁶

Khau highlighted several collaborations between the CMS Office of Minority Health and other federal partners. These projects are summarized in Box 2-1. She also called attention to the office’s Minority Research Grant Program, which provides funding for principal investigators at minority-serving institutions to conduct research focused on opportunities to embed health equity into CMS programs.

Mitra Rocca, Center for Drug Evaluation and Research of the Food and Drug Administration (FDA), said that the FDA has 13 projects that are funded by the PCOR Trust Fund.⁷ The projects are as follows:

___________________

⁴https://aspe.hhs.gov/basic-report/hhs-implementation-guidance-data-collection-standards-race-ethnicity-sex-primary-language-and-disability-status.

⁵https://data.cms.gov/mapping-medicare-disparities.

⁶ See https://www.cms.gov/files/document/racial-ethnic-gender-disparities-health-care-medicare-advantage.pdf and https://www.cms.gov/files/document/omh-rural-urban-report-2020.pdf.

⁷https://aspe.hhs.gov/collaborations-committees-advisory-groups/os-pcortf/explore-portfolio.

• Collection of Patient-Provided Information through a Mobile Device Application for Use in Comparative Effectiveness and Drug Safety Research;
• Common Data Model Harmonization (CDMH) and Open Standards for Evidence Generation;
• Cross-Network Directory Service;
• CURE ID: Aggregating and Analyzing COVID-19 Treatments from EHRs & Registries Globally;
• Developing a Strategically Coordinated Registry Network (CRN) for Women’s Health Technologies;

• Development of a Natural Language Processing (NLP) Web Service for Public Health Use;
• Enhancing Data Resources for Studying Patterns and Correlates of Mortality in Patient-Centered Outcomes Research: Linking National Death Index (NDI) and Commercial Claims;
• Making Medicaid Data More Accessible Through Common Data Models and FHIR APIs;
• SHIELD - Standardization of Lab Data to Enhance Patient-Centered Outcomes and Value-Based Care;
• Source Data Capture from Electronic Health Records (EHRs): Using Standardized Clinical Research Data;
• Standardization and Querying of Data Quality Metrics and Characteristics for Electronic Health Data;
• Utilizing Data from Various Data Partners in a Distributed Manner; and
• WHT-CRN Project: Bridging the PCOR Infrastructure and Innovation through Coordinated Registry Network (CRN) Community of Practice.

One of the projects that Rocca leads is the Source Data Capture from EHRs. The goal of the project, which is a collaboration with the University of California, San Francisco (discussed in additional detail by Laura Esserman, Chapter 4), is to develop methods and tools to automate the flow of structured EHR data into external systems. The second project, co-led by Rocca in collaboration with the National Institutes of Health (NIH), is Common Data Model Harmonization and Open Standards for Evidence Generation. This project is a collaboration among five agencies, focusing on harmonizing across multiple common data models and generating real-world evidence from real-world data. An FDA project that is focused on data linkages in particular is the Enhancing Data Resources for Studying Patterns and Correlates of Mortality in PCOR, a project that links commercial claims data to data from the National Death Index, which is a centralized database of death records from state vital statistics offices. A collaboration among the Centers for Disease Control and Prevention (CDC), CMS, and FDA, the goal of this project is to increase the availability of information on the cause of death.

With input from the leads of the various PCOR Trust Fund projects at FDA, Rocca identified several ways collaborations could evolve to meet PCOR and data challenges going forward:

• Develop an infrastructure to support research.
• Adopt Findable, Accessible, Interoperable, and Reusable (FAIR) principles as a goal and use metrics to measure the progress.

• Use distributed data models to conduct analyses across multiple institutions, with data remaining behind data-partner firewalls.
• Build trust and validation so they are engineered into the system, which would enable collaborators to run analytic software for one another.
• Develop open-source tools to support sharing, discovering, and reusing research data.
• Convene workshops with internal and external stakeholders around particular problems.
• Improve the quality and completeness of EHR data.

In terms of the barriers and potential solutions to the access and use of linked public data, Rocca said that there is a need for a strategy and a set of standards at the HHS level that address the challenges associated with lacking a master identifier to help link several sources of data. Related to that, there is a need for standards that address the issue of re-identification. Rocca said there is also a need for a systematic review of HHS data sources with an eye toward transparency and a need to develop informed consent guidelines that enable the sharing of both public- and private-sector data.

Rocca also mentioned the need to establish a formal ontology at the HHS level to make it easier to find data, and for establishing a metadata registry and repository for data elements, controlled terminologies, and mapping for controlled terminologies. She also highlighted the need for interoperability, and challenges associated with the lack of standardization for the data that are collected. She argued that the integration of health care and clinical research will require a change in culture that begins at the point of care, where data are generated.

Regarding suggestions for building an interoperable data capacity for PCOR, Rocca highlighted the following:

• Linking existing databases within HHS and other federal government agencies and the private sector;
• Developing a universal data use agreement;
• Applying tools, standards, and services developed as part of PCOR Trust Fund projects to other types of HHS data;
• Encouraging the development of common architectures and integration frameworks to enable interoperability, rather than developing single solutions; and
• Focusing the PCOR Trust Fund investments on cutting-edge solutions that may result in technical leaps.

Adi Gundlapalli, Public Health Informatics Office, CDC, said that patient-level data with sufficient granularity are essential for improving

health outcomes and that these data need to be made available in a way that preserves privacy. He argued that current laws and policies around the use of patient-level data are nuanced, and sometimes conflicting, which creates confusion for researchers, providers, and patients. This issue needs to be addressed in a way that balances individual privacy considerations, the risk of re-identification, and the utility of the datasets.

Making disease-specific datasets available for public use is an area where balancing the considerations discussed by Gundlapalli would be particularly important. He noted that recently CDC made available three COVID-19 datasets for public use, and this was accomplished by working closely with HHS to apply privacy preserving measures, including, in some cases, suppression algorithms, due to the small cell sizes. Gundlapalli said that the work done at CDC over the past year to acquire, store, provide secure access to, and analyze large datasets with high dimensionality will be an enduring resource for data capacity.

Regarding barriers and potential solutions for increased access to linked public data, Gundlapalli reiterated the risks of privacy breaches and re-identification, which, he said, have to be addressed. He noted that CDC receives only de-identified data. However, there are use cases where linking individuals within a dataset or across datasets, such as vaccination and case records for COVID-19, has tangible benefits for public health action. Because of this, CDC, in collaboration with the HHS Office of the Chief Information Officer, has been evaluating the feasibility of implementing privacy preserving record linkage (PPRL) techniques for public health data at the state, tribal, local, and territorial levels before the data are sent to CDC, with the COVID-19 vaccination data as a use case. These techniques can ensure that personally identifiable information remains within the jurisdictions’ firewalls.

PPRL solutions are now available through commercial vendors, and PPRL algorithms have been applied to large, commercially available health datasets, such as laboratory data, pharmacy data, claims information, and EHRs. There are also many published examples of real-world applications of PPRL and their associated benefits, and Gundlapalli argued that these benefits have to be balanced with the risk to privacy and the efforts required to implement PPRL solutions. He added that potential linkages of publicly available data with private, proprietary data are especially interesting, and these opportunities merit detailed consideration.

As far as building interoperable data capacity, Gundlapalli said that CDC also has ongoing projects that address this issue, and it may be useful to consider what can be learned from that work. CDC has been actively working on public health data modernization, an effort that was given a boost with recent COVID-19-related funding. Through a set of targeted investments across three priority thematic areas, CDC aims to promote the

reporting of clinical and laboratory data to ensure that core public health surveillance systems are interoperable, and the agency supports crosscutting upgrades such as the migration of the data to the cloud and access to new data sources. CDC is hoping that these efforts will also support the interoperable data capacity for PCOR. Gundlapalli summarized the three priority areas as (1) data sharing across the public health ecosystem, (2) using CDC systems for ongoing data modernization, and (3) adopting new standards and approaches for public health reporting.

Alison Cernich, Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), said that NICHD, as well as NIH more broadly, are heavily invested in ensuring that research incorporates PCOR to the greatest extent possible, and that data gathered through their research are available to other researchers for further analyses. She pointed out that NIH issued a new policy on data sharing for NIH-funded research.⁸ The policy will go into effect on January 25, 2023, and it is based on an understanding that sharing scientific data accelerates biomedical research discovery by enabling validation of research results, providing accessibility to high-value datasets, and promoting data reuse for future studies. The policy is intended to ensure that there is a plan in place for the sharing of data from NIH-funded research. It also specifies allowable costs for data management and sharing, and discusses ways of selecting a repository for the data.

Cernich noted that NIH supports several domain-specific data repositories that are open for both submitting and accessing data. An example is the NICHD Data and Specimen Hub, which allows clinical research data, including PROs, to be posted and shared. NICHD also funds the Data Sharing for Demographic Research infrastructure, which provides curation and archiving services for data relevant to health policy and health systems research as well as broader demographic research.

Concerning barriers, Cernich said that data sharing might be limited by either the parameters of the initial consent provided by the participant or based on NIH policies aimed at protecting the individual. Data sharing has to be balanced with these considerations as a first step. Beyond that, she said, it is important to ensure that data that are shared comply with guiding principles, such as the FAIR data principles and the Transparency, Responsibility, User focus, Sustainability, and Technology (TRUST) data repository principles.

Cernich noted that the COVID-19 pandemic accelerated the need for data that are based on common standards and can quickly be aggregated to produce new knowledge. Tools and resources available for use in public health emergencies and disasters through the Disaster Research Response

___________________

⁸https://grants.nih.gov/grants/guide/notice-files/NOT-OD-21-013.html.

Resource were useful to address this need initially. The PhenX Toolkit was another resource of data standards, clinical report forms, measurement protocols, and survey instruments, some of which included patient-centered outcome variables, such as life changes, household events, and overall impact.

NICHD has also been working on pointing the community toward standard data models, such as the Observational Medical Outcomes Partnership (OMOP) or HL7 FHIR. Cernich said that adopting standard data models can be challenging, because not all of them can accommodate every type of data, but NICHD has been encouraging the use of these standards. The agency has also been developing common data elements where those are lacking. For example, early in the pandemic, NICHD convened a group of stakeholders to specify common data elements for pregnancy as part of a large-scale study to look at the effects of COVID-19 in the context of pregnancy. These data elements were intended to be generalizable to other studies of pregnancy so they could be harmonized and aggregated for more robust analyses.

Cernich also discussed the Gabriella Miller Kids First Data Resource, which is supported by NICHD in collaboration with the National Heart, Lung, and Blood Institute; the National Cancer Institute; and the Common Fund. This resource will use FHIR standards to integrate EHR data, and it is also expanding to include genomic data on conditions in children.

Another NICHD collaboration with ONC, CDC, and the National Center for Health Statistics (NCHS) is focused on developing an HL7 FHIR implementation guide for maternal health, which will specify models for using EHR data standards to identify individual women and their individual pregnancies over time and across health systems. NICHD would also like to be able to link the records of parents and their children. These linkages would make it possible to monitor the impact of emerging public health concerns, such as infectious diseases, to examine the adverse effects of commonly used medications during pregnancy and postpartum on the pregnant person and the infant, and to determine predictive models that can help address various inequities such as inequalities in maternal morbidity and mortality. Cernich noted that NICHD is leading a consortium of projects focused on maternal health to enable coordination and collaboration within this portfolio through the PCOR Trust Fund.

NIH was a pioneer in developing PCOR measures, Cernich said, through projects such as the PROMIS Initiative and the Quality-of-Life Initiative, but she argued that there is a need to continue to refine and implement standard measures of patient perception and evolve with technology. She highlighted the All of Us project, which includes participants in developing some of the project’s modules and is also integrating wearables and other data sources to describe the group of people who are participating.

Cernich pointed to a need for new linkages for data within HHS. For example, NIH and NCHS reached an agreement on the use of data from the National Death Index for research, but other data on vital statistics remain difficult to integrate because of costs and challenges associated with establishing these types of agreements. She also said that access to HHS data needs to be expanded for NIH intramural investigators and that opportunities for doing this exist.

Jacob Kean, Salt Lake City Veterans Affairs Health Care System in the Department of Veterans Affairs (VA), and University of Utah, began by discussing the Department of Defense (DoD) and Department of Veterans Affairs Infrastructure for Clinical Intelligence (DaVINCI) as an example of notable progress on infrastructure for PCOR over the past 5 to 10 years. DaVINCI is a data warehouse and analytic platform that combines DoD and VA health care data, and it serves as an interface between the systems supporting the EHRs at the two agencies. The project began in 2014 and has been building on early successes to achieve scale. To date, DaVINCI has supported more than 60 operations and research projects, and transferring data between the two agencies is a key aspect of this.

Kean discussed lessons learned in several areas, including governance and compliance, data standardization, data quality, education, and partnerships. In the area of governance and regulatory compliance, DoD and VA explored several potential collaborative project governance options and resources for the navigation of regulatory policy, before they settled on an arrangement that was suitable for the DaVINCI project. Kean underscored the time and effort needed to complete this process and noted that establishing partnerships between federal agencies and other entities, such as states or private organizations, can be even more challenging.

In terms of data standardization, DaVINCI uses the OMOP Common Data Model, which Kean described as an enabler and an accelerator. For example, without the OMOP, a data user would need to look at separate data sources for inpatient, outpatient, and surgical encounters, and possibly look at seven or eight different tables for information on medications. OMOP provides researchers with an advantage because they can approach the dataset with an understanding of its structure. While a complete alignment is probably impossible, HHS efforts to harmonize standards are helping remove the barriers posed by the lack of standardization. Kean also underscored the value of HHS support for the development of open-source tools around agreed-upon standards and data models so that the consistency of the work around clinical data improves and also reduces the barrier to entry.

DaVINCI has maintained a focus on data quality throughout project development and execution. This facilitates the use of the data, for example by providing information on fitness for use. Kean said that among the most

effective DaVINCI efforts so far are those related to education. Educational outreach targets current and potential data users and focuses on the structure and provenance of the data. He encouraged HHS to continue to develop and enhance existing educational efforts on the appropriate management and use of data resources.

Kean also discussed challenges and opportunities associated with linked data, including both public and private data. The past few years have witnessed great advances in PPRL technologies and in the scaling of these technologies in a variety of contexts. These advances are happening in parallel with a new set of challenges. One challenge is that there are now thousands of datasets available, and it is difficult to gain a comprehensive understanding of this ecosystem. Kean said that HHS could assist with these efforts by promoting standards, standards education, and tool development. These standards and tools can help with a comparison of similar datasets and the appreciation of strengths and limitations.

Another challenge is related to the costs of linking datasets on a large scale. Kean noted that it is difficult to assign value to the different datasets, but HHS could consider cloud-hosting solutions that enable research and make it possible to shift some of the work from a project level to an agency level. For instance, a PCOR sponsor could sponsor the cost of many data sources and allow others to access the data.

In the future, technology solutions such as blockchain technologies could greatly advance the PCOR data infrastructure. Inherent in every blockchain technology is a distributed ledger, which documents all manipulation and use of data and helps to prevent any unauthorized use. Moreover, these technologies promote federated learning models, which eliminate data transfer. Blockchain technologies could promote the assignment of value to data through combinations of data attributes and data provenance. Furthermore, these technologies could enable data self-sovereignty, which may be a pinnacle of patient centeredness, because it would allow individuals to control access to and use of their data. Kean encouraged HHS to promote regulatory and policy solutions around these transformative technologies.

HHS can play a role in helping to develop and promote use cases, according to Kean. One such use case would be a situation where decentralized models are superior to centralized models for regulatory security and other reasons. Another one could be a situation where trust between entities is a paramount concern. Kean said that the DaVINCI project shows the feasibility of building interoperable data capacity. To date, HHS has played an essential role and could in the future prioritize efforts to advance regulatory guidance, align standards, and foster knowledge of the appropriate use of the growing PCOR ecosystem.

DISCUSSION

During the workshop, the formal presentations were followed by additional discussion among the workshop participants, including the speakers, committee, and audience members. Among the topics that were explored in further detail were the need to harmonize data collections, and data elements in particular, across the federal government. The importance of increasing the adoption of standards and leveraging new opportunities for collaboration made possible by standardized health information technology was also highlighted as part of the discussion. Speakers expressed a desire for coordinating efforts to increase the interoperability of data systems, which would serve as a foundation for the scalability of common approaches in areas such as informed consent.

Speakers acknowledged that increasing the consistency and the use of standards will be a slow and gradual process. In some cases, it might be necessary to accept that heterogeneity exists and consider ways of working within those parameters. It is also important to note that the existing datasets tend to focus on specific populations that are only a subset of the population as a whole and might differ from the overall population on a variety of dimensions.

The discussion also echoed conversations about the challenges resulting from the differing data collection goals in clinical care and research contexts. A related issue is the burden placed on providers and patients who are asked to provide the data. To mitigate these challenges, more clarity is needed about the potential uses and value of the data.

Participants also discussed the usefulness of data that originate from sources other than the context of patient care (such as population surveys) and the potential conceptual limitations that result from focusing on the patient, rather than the individual, regardless of whether the person has a disease, diagnosis, or interaction with a health care provider. This discussion echoed the conversations and committee conclusions that emerged from the first workshop in this series.⁹

Another theme that emerged centered on the challenge of prioritizing projects, given the complexity and broad scope of the PCOR data infrastructure. Priorities are set by legislation or by the agencies, and there are mechanisms for input from committees and workgroups. However, the discussion also made it clear that awareness about the data infrastructure projects is limited among external stakeholders and end users.

___________________

⁹https://www.nap.edu/catalog/26297/building-data-capacity-for-patient-centered-outcomes-research-interim-report.

CONCLUSIONS

The session with federal agency representatives highlighted several key areas where additional work, and in particular collaborative work, is especially needed to continue to build and strengthen the PCOR data infrastructure.

CONCLUSION 2-1: Collaboration among federal agencies and between federal agencies and other partners (such as states, patient groups, and others) is essential for continuing to build the patient-centered outcomes research data infrastructure. The areas where additional collaboration would be particularly useful include the following:

• Increasing consistency in the use of standards for data interoperability and element definitions;
• Addressing barriers that hinder data linkages, such as the limitations associated with health identifiers and mitigating potential selection biases resulting from linkage error;
• Balancing the burden of the data collections and disclosure risks with the value of the datasets;
• Communicating the usefulness of the data collections to those who are asked to provide data about themselves and those who collect the data;
• Promoting discussion and education about fitness for use of the data; and
• Working with stakeholders and patients to promote sharing of data.

While there is frequent collaboration among HHS partners on PCOR data infrastructure work, and the Office of the Assistant Secretary for Planning and Evaluation’s public website contains a comprehensive list of past and current projects funded from the PCOR Trust Fund, additional dissemination efforts focused on external stakeholders could further increase the usefulness of these investments.

CONCLUSION 2-2: There is a need to increase awareness among all stakeholders about new data infrastructure developments funded by the Patient-Centered Outcomes Research Trust Fund. Increased awareness will enhance the efficiency and effectiveness of research, which will increase the impact of the investments made in infrastructure development.