4

Researcher Perspectives on Data Needs

This chapter summarizes data needs conveyed by researchers. Speakers in this session were asked to focus on the questions below. The chapter also discusses a case study on the use of the patient-centered outcomes research (PCOR) infrastructure to study whether vitamin D can reduce the burden of COVID-19. The chapter concludes with the committee’s conclusions.

• Looking ahead, what are the main data needs?
• What are the implications of the (recently broadened) statutory scope for PCOR?
• What questions cannot be answered and who is not served by the current PCOR data infrastructure?
• What new data sources could be incorporated into the PCOR data infrastructure?
• What data capacity challenges is the U.S. Department of Health and Human Services (HHS) best positioned to address in the context of its public mission, authorities, programs, and data resources?

David Meltzer, University of Chicago, discussed the use of existing data, including PCOR datasets, to conduct research on whether vitamin D could reduce the burden of COVID-19. Box 4-1 summarizes some of the key studies to date, including the studies initiated by Meltzer at the beginning of the pandemic, using University of Chicago data. This case study was intended to illustrate a specific application of how the data infrastructure can be used to study an emerging health question, and how research projects can build on each other. Box 4-1 illustrates limitations and sum-

marizes Meltzer’s observations related to the usefulness of the PCOR data based on this work.

Andrew Bazemore, American Board of Family Medicine, noted that although we live in a time of unprecedented health-data availability, there are also some structural blind spots in the U.S. health care system. He cited Kerr White’s research on the ecology of medical care and the notion that out of 1,000 people in a community in a given month, 750 might experience illness or injury, 250 will seek primary care, and 10 will be hospitalized, of which just one will be hospitalized in an academic health center,

in other words in the type of place where most of the research is being conducted.¹ While Kerr White’s original study was published in the early 1960s, this “ecology” has not changed. Bazemore said that it is important to understand where people land within the broader health care setting, and where the borders are between wellness and illness, between illness and care-seeking, and between primary care and the hospital, in order to

___________________

¹ K.L. White, T.F. Williams, and B.G. Greenberg. (1996). The ecology of medical care. Bulletin of the New York Academy of Medicine 73(1), 187–212.

better understand how to provide access to high-quality care that is truly patient-centered.

Bazemore noted that primary care, which is the largest platform for health care delivery, is not well represented in the data available. For example, about 0.3 percent of National Institutes of Health (NIH) research funding ends up in family medicine research settings. A study that looked at the first six cycles and then the subsequent six cycles of funding from PCORI found that between 18 and 30 percent involved primary care sites, even though about half of U.S. health care delivery happens in those settings. Even in the case of studies that involved primary care sites, typically their role was focused on patient recruitment. Places where patients with the highest risk of poor patient-centered outcomes are receiving care, including safety net clinics, federally qualified health centers, small health clinics, and rural health clinics, would be even less likely to be included.

Bazemore said that it would be useful to better understand patients’ experiences in primary care settings, including how they differ from the experiences of those who receive care from a specialist, in an emergency room, or in other places. It would also be important to understand how the characteristics of primary care influence patient outcomes. Patients often report that what is important to them is that their primary care doctor involve them as much as they want to be involved in their health care. Information about community health risk factors facing patients would provide further context.

Among the data sources that are missing from the current PCOR data infrastructure, according to Bazemore, are not only primary care practices themselves but also two other data sources: primary care registries and health information exchanges focused on primary care and the safety net. On the analytical side, Bazemore noted that a major concern is that new technologies such as artificial intelligence and machine learning do not typically involve primary care patients in their algorithm development. This has implications for the data that are being generated for PCOR.

Bazemore also underscored the need to incorporate social determinants of health data into the data environments being built. This includes geo-coded secondary ecological data, ranging from individual variables to indices, such as the social deprivation index, the area deprivation index, and the social vulnerability index. Patient-reported data on social determinants of health are also needed, in combination with the ecological proxies. This information together can enable researchers to really understand the patient experience and the dynamic neighborhood features that inform and complement the patient perspective.

Kurt Stange, Case Western Reserve University, began by saying that as a family and public health physician, he has spent his career doing “stealth” research, trying to understand how to advance health and health care for

whole people in a system that is designed to fragment them into their parts. The problem for PCOR is not only fragmented and siloed data but also a system that is designed for fragmentation in data use.

Stange raised the question of whether it would be possible to shift the model from trying to use data to drive quality from the top down to using data to support the local agency of those on the front lines who are trying to integrate, personalize, and prioritize care for all people. In the effort to solve the puzzle of health and patient-centered health care, the current approach tends to divide things up into parts: diseases, risk factors, risk groups. The parts are valued, but efforts to integrate the whole are not sufficiently supported.

Regarding specific needs, Stange highlighted data that would support the functions of integrating, personalizing, and prioritizing care for whole people. There is also a need to support care for people whose health needs cannot be addressed by relying on a single disease label, or a risk label, or a group label. Linking different sectors affecting health is also necessary.

As a starting point for addressing the prioritizing function, Stange cited a 2015 National Academies of Sciences, Engineering, and Medicine Vital Signs report that provided a useful framework for identifying core metrics for assessing health and progress in health care.² The report proposed the following criteria for core measures:

• importance for health,
• strength of linkage to progress,
• understandability of the measure,
• technical integrity,
• potential for broader system impact, and
• utility at multiple levels.

Stange argued that this is an efficient and effective way of thinking about data. He noted that the report also provides guidance on measuring performance for useful PCOR domains. Stange said that the broadened statutory scope of PCOR provides an opportunity to focus on the whole as well as the parts. It also provides opportunities to support those on the front lines trying to contextualize care, as well as to support relationship-centered care.

Concerning the question of who is poorly served by the current PCOR data infrastructure, Stange underscored previous points about the limitations of the data on people living with multiple chronic conditions and various disadvantaged groups. He added that the data also have limitations

___________________

² Institute of Medicine. (2015). Vital Signs: Core Metrics for Health and Health Care Progress. Washington, DC: The National Academies Press.

for those who are not “helpfully defined by their disease” or are defined by other data collected for another purpose. The health care system tends to offer people a disease label, but that is not necessarily the label or the data most helpful to them in terms of what is important in their lives.

For additional data that could be incorporated into the PCOR data infrastructure, Stange suggested the Person-Centered Primary Care Measures, which he and his coauthors developed based on what patients, clinicians, and (to a lesser extent) payors said was important to them in health care.³Box 4-2 shows these measures, which Stange said are widely used and are also pending endorsement by the National Quality Forum and the Centers for Medicare & Medicaid Services (CMS) for use in the CMS Quality Payment Program.

Stange also discussed specific data challenges that he believes HHS could be particularly well positioned to address. These include

• reframing data use to support primary care as a force for integration and equity for individuals and families;
• reframing data use to support public health as a force for integration and equity for communities and populations;
• supporting the integration of primary care and public health;
• supporting primary care research about the care of whole people;

___________________

³ R.S. Etz, S.J. Zyzanski, M.M. Gonzalez, S.R. Reves, J.P. O’Neal, and K.C. Stange. (2019). A new comprehensive measure of high-value aspects of primary care, The Annals of Family Medicine, 17(3), 221–230. http://www.annfammed.org/content/17/3/221.

• bringing together the Office of the Assistant Secretary for Planning and Evaluation, the Centers for Disease Control and Prevention, and the National Committee on Vital and Health Statistics in their congressionally mandated role to update the Health Insurance Portability and Accountability Act of 1996 to make data sharing safe and less onerous; and
• raising the budget cap for NIH research project grants.

Robert Califf, Verily Life Sciences and Google Health, started by saying that in his experience the key question people want answered in most health care scenarios is this: Out of the options at my disposal, which diagnostic strategy and treatment is best for me? While this is at the core of the terminology comparative effectiveness, the data available and associated context are not enabling researchers to design and conduct crisp, reliable comparative effectiveness studies.

Califf pointed out the important role the pragmatic randomized trial played in providing answers related to the COVID-19 pandemic. He said that a priority going forward should be to identify a core set of data that would provide reliable information that, when coupled with appropriate study design, could enable multiple pragmatic clinical trials to be conducted to answer the many questions that people have about the options they have, and about the option that would ultimately be best for them. These trials could include not only drugs and devices but also behavioral and health service interventions and systems of care as well. He noted that more data is not necessarily better, and that time invested in identifying the essential data that are needed would be time well spent.

Califf also pointed out that computing has changed since the early days of PCOR and that some questions that were not feasible to be examined before are within scope now. This includes questions related to the roles of deep molecular, behavioral, and social determinants of health, given that it has long been recognized that social, cultural, behavioral, and biological determinants not only interact to combine in complex ways to impact individual outcomes, but also have common dimensions across groups of people. Going forward, it will be important to look at opportunities afforded by different ways to integrate information using a different approach to computing. He cited the example of COVID-19 trackers, which constantly scrape information off the Internet, integrate the data, and present them at a variety of levels, ranging from countries to states, counties, and even individual hospitals. This information has been useful for optimizing clinical trial recruitment and for deploying interventions. In theory, this approach could now be used for all diseases, starting with a substrate of real-time information about the status of health from a geospatial and temporal reference point.

Califf also argued that “de-identified data” is somewhat of a myth, because re-identification is usually possible if someone is determined to accomplish that. Furthermore, in many cases identifiable data are going to be the most useful for research and the most valuable for translating research into practice. He argued that the current rules that exist are not fit for purpose, because they make research very difficult. What is needed, Califf said, is a system that enables researchers to use identifiable data more easily, while at the same time imposing extreme penalties on people who take advantage of this access and misuse the data. He added that health systems already use fully identified data for operations purposes, and if that is done, then the data should perhaps also be available to produce generalizable knowledge that could be spread across health systems and could benefit people beyond those involved in a specific health care operation.

Califf echoed the arguments made by others about the importance of integrating research and care. He noted that PCOR has already made a big difference in this area, but it would make a bigger difference yet to continue this work.

Califf also brought up the challenges associated with patient-reported outcome data collected through cell phones. The use of cell phones is widespread and access to broadband internet is also growing, but these types of data collections do not reach everyone equally. In the case of digital technologies, older people are one group that is likely to be underrepresented, not only because of challenges related to access to cell phones, but also due to more limited skills at using the technology effectively. Califf also echoed challenges associated with the lack of data for a variety of populations more broadly, as discussed in Chapter 2. He highlighted two additional groups that receive relatively little attention: people living in rural areas and those struggling with addictions. There are rising health concerns specific to both of these populations, and therefore there is a need to develop approaches that would address the data limitations associated with these groups.

David Cella, Northwestern University, summarized key data needs as a common data model for patient-reported outcomes; common data elements for patient-reported outcomes; comparative effectiveness metrics across conditions; and medical and nonmedical cost data. He said that despite advances in artificial intelligence and natural language processing, structured data are still useful, in part because they enable comparative effectiveness research not only within conditions, but also across conditions, and they enable a look at overall value for cost. Cella listed insurance deductibles, copayments, caregiver expenses, and work productivity (for example, absenteeism) as the key components of medical and nonmedical cost data needed.

Cella argued that the broadened statutory scope for PCOR opens a major opportunity to examine costs in the context of effectiveness, which

has not previously been possible. It also opens the potential for interagency collaboration around cost-effectiveness research and application. These collaborations could include Agency for Healthcare Research and Quality, NIH, CMS, and possibly the U.S. Food and Drug Administration. The broadened statutory scope of PCOR also provides ingredients for a learning health systems approach across provider organizations.

Concerning the questions that cannot be answered with the current PCOR infrastructure, Cella said it continues to be difficult to answer crosscutting comparative effectiveness research questions as they relate to patient-reported outcomes. He echoed earlier points about the difficulty of accessing patient-reported outcomes data, particularly through the Chicago Area Patient-Centered Outcomes Research Network, which is 1 of 13 Clinical Data Research Networks.

Cella mentioned that in 2014 he was part of the PCORnet Patient-Reported Outcomes Common Measures Working Group, which recommended a set of common measures for PCORnet. The recommendations included nine core items for adults, focused on general health, quality of life, physical function (two questions), depression, fatigue, sleep, social roles and activities, and pain. For children, the core items recommended were focused on: general health, quality of life, pain, fatigue, stress, depression, peer relationships, and family relationships. Most of these items are from the Patient-Reported Outcomes Measurement Information System, which provides deep and wide item banks to cover these domains. From each of the domains, the working group selected a single question that would work best, if one could only ask a single question. Cella said that with large networks such as PCORnet, sometimes one question is sufficient to obtain a good estimate for a cohort. The working group’s recommendations were not implemented, however, because of technology limitations, such as electronic health records that could not adequately speak to one another and share a common data model overall.

Cella cited a 2019 paper that found that among member organizations of the New England Journal of Medicine Catalyst Insights Council, around 38 percent used the Patient-Reported Outcome Measures System (PROMS) and an additional 17 percent had plans to start using it within 3 years. Cella added that even those who use PROMS capture information on less than 50 percent of patients, often excluding underrepresented minorities and patients with lower educational levels. Most who collect these data do so primarily for operational reasons, to improve their metrics, or to improve their own patients’ experience or engagement. And there is still not much outward-facing incentive for doing patient-reported outcome assessments in clinical settings.

Cella listed the following as data-capacity activities that he considers HHS to be best positioned to address:

• promoting technologies to capture patient-provided data, including patient-reported outcome measures;
• promoting patient activation and engagement;
• endorsing health care quality initiatives and patient-reported outcome performance measures; and
• “funding the mandate” or finding other ways of encouraging clinical providers and provider organizations to collect the data.

Giselle Corbie-Smith, University of North Carolina, argued that having incomplete data on race, ethnicity, and other social identities leads to erasures of the experiences of some populations. This lack of data and these erasures diminish the potential of PCOR to advance health equity. The data infrastructure needs to be robust enough to allow data to be disaggregated in ways that can detect differences among small populations. For example, an inability to disaggregate data to compare Filipino health care workers to other Asians and Pacific Islanders could mean missing a disproportionate impact of the COVID-19 pandemic on Filipino nurses and nurses of Filipino descent.

Corbie-Smith said there has been growing momentum to understand the social determinants of health and that some information of this kind is being captured in electronic medical records. However, she added, below that surface there is a need to better understand the role of structural racism, community context, and social drivers of health. There is a need for data on patients’ experiences within the health care systems, as well as outside the walls of hospitals and clinics. Specifically, there are limited data on community health resources outside the health care system: the data are either completely missing or, when available, are often dissociated from the health care system. In the case of the COVID-19 pandemic, it became clear that this information is necessary to be able to provide equitable testing resources and equitable vaccine resources.

Corbie-Smith also discussed the critical role of engagement, particularly engagement of patients, community-based organizations, and faith-based organizations, in the case of disasters such as the COVID-19 pandemic. Borrowing a term used by Ralph Ellison in the Invisible Man, Corbie-Smith said that the network of community service providers is “unvisible” to health care and public health systems. There is a tendency to think that health is created within the health care system when, in fact, community service providers are often the ones addressing social determinants of health. She also pointed out that there is a crisis around trust in science, and that misinformation is filling the gap. If researchers made their work more accessible to patients, providers, and communities, trust in the work of scientists would likely also increase.

With regard to opportunities, Corbie-Smith referred to “data democratization,” which echoed what other speakers said about the need to make the data more widely visible and accessible beyond the research community, and particularly to patients and communities. Beyond obviously benefiting from the use of the data, patients can also help researchers interpret the meaning of data.

Communities can also benefit from the use of data to think about how to effect change around health equity, because it is unreasonable to expect that achieving health equity could happen within the PCOR context alone. Corbie-Smith noted that data visualizations are a helpful tool for democratizing data.

As also discussed by others, collecting more complete data on race and ethnicity is another area that represents an opportunity. This includes avoiding misclassification and collecting data that allow for disaggregation to understand small populations. The data and research also need to reflect the intersection of social identities.

Corbie-Smith emphasized the need for robust stakeholder engagement in defining strategic areas of research. This includes not only stakeholder input on research questions within a particular study, but also input on the overall strategy for PCOR. Stakeholder engagement needs to include communities not commonly reflected or recognized.

Corbie-Smith also highlighted the opportunities presented by including networks of community service providers in the research. These stakeholders include not only federally qualified community health centers but also community service providers that are providing a matrix of care. Keeping these providers visible will lead to a better understanding of the lived experiences of patients, what is important to them, and how they actually can be healthy.

Corbie-Smith also discussed the use of mobile technologies to collect person- and community-level data. While not all of her patients are technologically savvy enough to use telehealth, many of them have smartphones. Smartphones, wearables, and methodologies such as ecological momentary assessment make it possible to understand what is happening with patients outside of the health care context. Combining person-centered data with community-level data makes it possible to understand the interaction between the community, the physical built environment, and the social environments and how that impacts health. Corbie-Smith said that ecological momentary assessment can help provide answers to questions such as the impact of structural racism and interpersonal racism on the health of individuals, particularly in communities that are over-policed.

In terms of analytical opportunities, Corbie-Smith said that there is a need for analyses that reflect the complexities. She said that traditional approaches assume that factors such as food insecurity, housing insecurity,

and intimate partner violence are independent of each other. There is also a tendency to assume linearity, instead of recognizing the complexity of the systems in which patients live, work, grow, play, and age. Corbie-Smith also underscored the need for transparency in research and around data democratization in order to demonstrate the trustworthiness of science, which fundamentally is what is needed to move forward.

Scott Ramsey, Fred Hutchinson Institute for Cancer Outcomes Research (HICOR), began by discussing that institute’s mission and its Value in Cancer Care initiative, which engages oncology providers, patient partners, payers, health system representatives, and researchers to improve the value and efficiency of cancer care delivery in Washington State. This initiative was formed based on the realization that the rising costs of cancer care and problems with care coordination threaten society’s ability to eliminate cancer as a cause of suffering and death.

HICOR’s community engagement program shares data about clinic performance and costs across a common population-based data platform. The network includes all of the 28 oncology practices in Washington State, five of the state’s major health insurance providers, representatives from local and state government, and a number of patient advocacy groups and patient advocates, including those who represent typically underrepresented minorities.

The database, which is updated on a regular basis throughout the year, includes insurance claims from major payors in the state and is linked to the two cancer registries. The data are used to produce an annual community cancer care report, which documents several oncology quality metrics and the average cost of care for each clinic for specific services. The database also serves as a platform for the oncology community to share best practices, and for a low-cost way to capture outcomes from prospective studies that have started in response to quality issues that were identified from the database and that were prioritized by the stakeholders.

Based on this work, Ramsey shared the lessons learned about key characteristics of databases that can best serve patients. His list included

• comprehensive capture of the patient experience;
• data elements informed by patients and other stakeholders;
• data relevant to treatment decisions;
• ongoing mechanism for direct feedback on patient-identified priorities;
• information that is relevant and accessible; and
• information that is timely.

He noted that in his experience, inadequate accessibility and timeliness have been problems for cancer data for many years.

Ramsey asked whether databases are built from the perspective of a patient or that of a researcher who is trying to adapt the patient perspective. Thinking about the patient as a data consumer is one way to begin to address this issue. When patients seek medical care, they arrive with a particular medical history and a range of social determinants might influence their perspectives. Their neighborhood and their network influence the care available to them, their interest and ability to understand the health system that serves them, their care plan, and the rationale for that plan. Because a subset of patients wants to interrogate the data themselves to understand what they should do, it is important to think about whether a proposed data infrastructure allows understanding of health systems and interrogation in a way that makes sense to patients. Ramsey said he believes that the technology industry is ahead of government agencies in thinking about these issues.

Thinking specifically about cancer care, Ramsey said that based on the literature and based on his own experience working with cancer patients, measures of patient satisfaction do not correlate well with the process measures of care and key outcomes. To address these disparities, there is a need for more granular and relevant measures of patient experience. According to Ramsey, the factors that influence patients’ sense of well-being during cancer care, and that are not always addressed by providers, include feeling that they are supported, dealing with uncertainty, perceived loss of autonomy, and trust in the health system.

Ramsey said that in seeking to understand the relationship between treatment and outcomes, the research community sometimes suffers from the well-known “streetlight effect,” using only the data that are available. Those data typically focus on what happens in the health care system, whereas, he believes, in many cases the individual’s experiences and environment play a bigger role in their care and outcomes.

The social determinants of health are generally not captured in traditional claims and electronic health records databases. However, it is becoming increasingly clear to the cancer care community that social determinants of health may play a bigger role than any other factor in the observed differences in outcomes among cancer patients. Social determinants of health influence patients over a lifetime, but to date little has been done to understand this at a level of specificity that can address policy. The limited data available certainly limit the ability to do this research.

Ramsey also discussed his research on financial toxicity in cancer care, and the emerging picture of vast and lasting impacts on patients’ financial well-being, which translates into impact on their well-being in other domains. He argued that there is a need for a data infrastructure that allows researchers to study this problem in other chronic conditions. This would necessitate linking to existing financial databases, such as credit reporting

databases, to understand the scale and scope of this problem. He noted that HICOR has been able to link its cancer registry with credit data from Transunion, although accomplishing this took 2 years.

Ramsey argued that HHS does not necessarily need to create new data. Instead, he said, the agency is best positioned to facilitate access to existing data that currently live in the private sector; create regulations that foster interoperability; establish privacy safeguards; and improve timeliness of databases, particularly in areas such as cancer care, which is quickly evolving.

CONCLUSIONS

The final session of the workshop included researchers working in a variety of areas related to patient-centered outcomes. Their input echoed many of the points made by others throughout the workshop. In particular, it is clear that limiting the focus to the person as the patient, as opposed to the person as a whole, limits not only thinking about the data but also the outcomes and impacts that matter to people, both inside their medical relationships when they are patients and, more generally, outside of medical relationships.

The fragmented nature of the data infrastructure and the data silos represents a particular hurdle for researchers. This hurdle could be overcome by a focus on facilitating data linkages, which in turn could increase the usefulness of the information available for research as well as for decision making more broadly.

Researchers described a variety of barriers that limit their ability to access the data available in the many existing databases, ranging from databases that can be considered a part of the PCOR data infrastructure to databases owned by private companies. Focusing on facilitating and simplifying access represents an area that could further enhance the usefulness of PCOR data.

Researchers echoed the need to make PCOR data more widely available to empower patients and communities to use this information. Efforts to reduce disparities, in particular, cannot be accomplished by research alone.

The need for information on the cost of health care and the ways cost factors into care decisions represented another area where researchers echoed the need expressed by other stakeholders for more data.

This page intentionally left blank.