Skip to main content

Currently Skimming:

Appendix B: Building Data Capacity for Patient-Centered Outcomes Research: Interim Report 1Looking Ahead at Data Needs
Pages 59-148

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 59...
... Appendix B Building Data Capacity for Patient-Centered Outcomes Research: Interim Report 1– Looking Ahead at Data Needs (Full text of the committee's first interim report released on September 10, 2021.) 1 1 https://www.nap.edu/catalog/26297/building-data-capacity-for-patient-centered-outcomes research-interim-report.
From page 61...
... APPENDIX B 61 Committee on Building Data Capacity for Patient-Centered Outcomes Research: An Agenda for 2021 to 2030 Committee on National Statistics Division of Behavioral and Social Sciences and Education Board on Health Care Services Health and Medicine Division Computer Science and Telecommunications Board Division on Engineering and Physical Sciences A Consensus Study Report of
From page 62...
... . Building Data Capacity for Patient-Centered Outcomes Research: Interim Report 1 -- Looking Ahead at Data Needs.
From page 63...
... APPENDIX B 63 The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institu tion to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research.
From page 64...
... 64 BUILDING DATA CAPACITY FOR PCOR Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study's statement of task by an authoring committee of experts. Reports typi cally include findings, conclusions, and recommendations based on information gathered by the committee and the committee's deliberations.
From page 65...
... BRIDGES, The Ohio State University JULIE BYNUM, University of Michigan ANGELA DOBES, IBD Plexus, Crohn's & Colitis Foundation OLUWADAMILOLA FAYANJU, The University of Pennsylvania DEBORAH ESTRIN, Cornell Tech CONSTANTINE GATSONIS, Brown University ROBERT GOERGE, Chapin Hall, University of Chicago GEORGE HRIPCSAK, Columbia University LISA IEZZONI, Massachusetts General Hospital S CLAIBORNE JOHNSTON, The University of Texas at Austin MIGUEL MARINO, Oregon Health & Science University ELIZABETH McGLYNN, Kaiser Permanente DAVID MELTZER, University of Chicago PAUL TANG, Stanford University and Palo Alto Medical Foundation KRISZTINA MARTON, Study Director MEGAN KEARNEY, Associate Program Officer (until June 2021)
From page 66...
... COUPER, University of Michigan JANET M CURRIE, Princeton University DIANA FARRELL, JPMorgan Chase Institute ROBERT GOERGE, Chapin Hall at the University of Chicago ERICA L
From page 67...
... DEVOE, Oregon Health & Science University R ADAMS DUDLEY, University of Minnesota RICHARD G
From page 68...
... 68 BUILDING DATA CAPACITY FOR PCOR COMPUTER SCIENCE AND TELECOMMUNICATIONS BOARD LAURA HAAS (Chair) , University of Massachusetts, Amherst DAVID CULLER, University of California, Berkeley ERIC HORVITZ, Microsoft Corporation CHARLES ISBELL, Georgia Institute of Technology BETH MYNATT, Georgia Institute of Technology CRAIG PARTRIDGE, Colorado State University DANIELA RUS, Massachusetts Institute of Technology FRED B
From page 69...
... Wagner Gradu ate School of Public Service, New York University; Elizabeth Patzer, Depart ment of Surgery and Health Services Research Center, Emory University; Robert L Phillips, Jr., Center for Professionalism and Value in Health Care, American Board of Family Medicine Foundation; Russell Rothman, Insti tute for Medicine and Public Health, Vanderbilt University Medical Center; and Mariana F
From page 70...
... 70 BUILDING DATA CAPACITY FOR PCOR x ACKNOWLEDGMENTS Department of Statistics, Iowa State University. They were responsible for making certain that an independent examination of this report was car ried out in accordance with the standards of the National Academies and that all review comments were carefully considered.
From page 71...
... APPENDIX B 71 Contents Summary 1 1 Introduction 7 2 Health Disparities Data Needs 19 3 Patient Perspectives on Data Needs 29 4 Researcher Perspectives on Data Needs 41 Appendixes A Biographical Sketches of Committee Members 57 B Workshop Agenda 63 C Biographical Sketches of Workshop Speakers 67 xi
From page 73...
... APPENDIX B 73 Boxes, Figures, and Table BOXES 1-1 Key Data Infrastructure Functionalities in the Existing Strategic Framework for Patient-Centered Outcomes Research, 12 1-2 Building Blocks of the Patient-Centered Outcomes Research Data Infrastructure, 13 1-3 Statement of Task for the Overall Study, 15 2-1 An Example of How Data Silos Hinder the Ability to Answer Policy Questions: The Impact of Medicaid Expansion on Equity in Hospitalizations for Complex Cancer Surgery, 21 2-2 An Example of How Data Silos Hinder the Ability to Answer Policy Questions: The Impact of Hospital Value-Based Purchasing on Equity in Posthospitalization Functional Status, 22 2-3 Disability Questions for Potential Inclusion in Electronic Health Records, 23 4-1 Case Study: The Use of the Patient-Centered Outcomes Research Infrastructure to Study Whether Vitamin D Can Reduce the Burden of COVID-19, 42 4-2 Person-Centered Primary Care Measures, 46 xiii
From page 74...
... 74 BUILDING DATA CAPACITY FOR PCOR xiv BOXES, FIGURES, AND TABLE FIGURES 1-1 Patient-Centered Outcomes Research Trust Fund: Three streams of work and funding, 9 1-2 ASPE's strategic framework for the patient-centered outcomes research data infrastructure, 11 2-1 Example of data available on social determinants of health, 20 3-1 Evidation Health's vision for person-generated health data frame work, 36 3-2 Episodic real-world evidence data points versus continuous data using digital technologies, 38 TABLE 3-1 Examples of Patient-Experience Information Related to Chronic Fatigue Syndrome and Myalgic Encephalomyelitis in the Food and Drug Administration's The Voice of the Patient Report, 35
From page 75...
... ASPE asked the National Academies of Sciences, Engineering, and Medicine to appoint a consensus study committee to identify issues critical to the continued development of the data infrastructure for PCOR. The committee's work will contribute to ASPE's development of a strategic plan that will guide its work related to PCOR data capacity over the next decade.
From page 76...
... This presents a fundamental chal lenge to answering critical questions, but the workshop also made it clear that opportunities exist for making the data infrastructure more suitable for answering questions of interest, so long as the potential uses of the data are carefully considered. CONCLUSION 2-2: The data available for patient-centered outcomes research are often collected for reasons other than research, which limits their usefulness.
From page 77...
... Health disparities represent an evolving and expanding area of research, with corresponding data needs. Limitations exist for a variety of social determinants of health data and for a range of populations.
From page 78...
... Information about the cost implications of medical care is an area in which data are particularly limited, because it has only recently been included in the statutory scope of PCOR. CONCLUSION 3-1: The patient-centered outcomes research data infrastructure has not reached its full potential to provide data that can answer questions that matter to patients and enable them to make informed decisions.
From page 79...
... FOCUS ON THE PERSON AS A WHOLE The input received from workshop participants made it clear that limit ing the focus to the patient limits not only thinking about the data but also the outcomes and impacts that matter to people in general. CONCLUSION 4-1: Broadening the focus from the patient to the person more generally would enable a more comprehensive approach to the data infrastructure and a better understanding of the outcomes and impacts that matter to people.
From page 81...
... . The PCOR data infrastructure provides decision makers with objective, scientific evidence on the effectiveness of treatments, ser vices, and other interventions used in health care.
From page 82...
... of the Public Health Service Act instructed the Secretary of HHS to: … provide for the coordination of relevant Federal health programs to build data capacity for comparative clinical effectiveness research, including the development and use of clinical registries and health outcomes research networks, in order to develop and maintain a comprehensive, interoperable data network to collect, link, and analyze data on outcomes and effective ness from multiple sources including electronic health records.2 1 https://www.ssa.gov/OP_Home/ssact/title11/1181.htm. 2 https://aspe.hhs.gov/collaborations-committees-advisory-groups/os-pcortf/about-os-pcortf.
From page 83...
... As the coordinating agency for the data infrastructure investment port folio across HHS agencies, ASPE guides the PCOR data infrastructure's strategic framework and vision, sets funding priorities, and coordinates interagency workgroups. ASPE's work is assisted by a Leadership Council for the PCOR Trust Fund, which includes representatives from other HHS agencies, including the Administration for Children and Families, the Administration for Community Living, AHRQ, the Assistant Secretary for Preparedness and Response, the Centers for Disease Control and Pre vention (CDC)
From page 84...
... Data collected as part of clinical care include data collected for health care delivery and for billing purposes. Examples of primary data collected as part of research stud ies include data from clinical trials and national health surveys.
From page 85...
... APPENDIX B FIGURE 1-2 ASPE's strategic framework for the patient-centered outcomes research data infrastructure. 11 SOURCE: Workshop presentation by ASPE, May 3, 2021.
From page 86...
... . Linking of Clinical and Other Data for Research Researchers will be able to follow patients across the care continuum over time, including those enrolled in clinical trials.
From page 87...
... Governance structures refer to entities that are needed to develop and apply the rules and policies needed for building an interoperable and sustainable research network. Governance structures support the efficient use of the data infrastructure for research across individual and organizations' boundaries of control and owner ship.
From page 88...
... This first interim report summarizes the discussion and conclusions from the first workshop, which focused on looking ahead at data user needs over the next decade. The second workshop in the series centered on data standards, methods, and policies that could make the PCOR data infrastructure more useful.
From page 89...
... • Assess anticipated changes to health care priorities and priorities for health data and their impact on building data capacity into the fore seeable future, as identified by the Assistant Secretary for Planning and Evaluation. • Evaluate the feasibility and utility of developing a phased-in approach to building the interoperable data capacity for patient-centered outcomes research with existing databases in the Department of Health and Human Services, other federal departments, and the private sector in a phased approach, such as projects identified in the Cures Act Title III Section 4003 (Interoperability)
From page 90...
... • Discuss what questions cannot be answered and who is not served by the current PCOR data infrastructure. • Discuss what HHS is best positioned to address and how the agency could maximize resources available for the PCOR data infrastructure (representing 4% of the PCOR Trust Fund)
From page 91...
... OVERVIEW OF THE REPORT This report is organized around the three main sessions of the work shop. Chapter 2 discusses presentations on the data needs related to health disparities, Chapter 3 focuses on patient organization needs, and Chapter 4 describes the input received from researchers.
From page 93...
... data infrastructure in terms of o disparities in the data, including knowledge about patient out comes, taking into consideration differences in patient prefer ences and values? o challenges associated with using the data to understand dispari ties and health equity?
From page 94...
... As an example of how data silos hinder the ability to answer policy questions, Joynt Maddox discussed efforts to answer the question of how FIGURE 2-1 Example of data available on social determinants of health. NOTES: EMR = electronic medical records; OT = occupational therapy; PT = physi cal therapy; SDOH = social determinants of health.
From page 95...
... create mapping and linkages across data sources; (3) incentivize the collection of Z-codes rel evant to social determinants of health and data on functional status through payment policy; and (4)
From page 96...
... Morris said that data on disabilities are rarely collected or documented in common data sources, such as electronic health records, patient experi ence surveys, or in big data sources, such as claims data. ICD-9-CM or ICD-10-CM codes are often insufficient for identifying who has a disability.
From page 97...
... As an example, the sexual orientation item BOX 2-3 Disability Questions for Potential Inclusion in Electronic Health Records 1. Are you deaf or do you have serious difficulty hearing?
From page 98...
... Lunn also emphasized that sexual and gender minority terms tend to change at a rapid pace, and therefore frequent re-evaluation with community engagement is critical to selecting the optimal terms to use. Kaleab Abebe, University of Pittsburgh, echoed comments made by other speakers related to some of the key challenges associated with the data infrastructure for PCOR.
From page 99...
... He also emphasized the need to develop metrics for the data infrastructure portfolio to enable researchers to better understand successes and failures and potentially utilize this information when considering future work. Abebe noted that the social determinants of health can change over time, so measures need to be developed with the understanding that they need to be flexible.
From page 100...
... CONCLUSION 2-1: Health disparities can occur across a broad range of characteristics and populations. Data limitations affect the ability to identify and understand these disparities in many areas.
From page 101...
... CONCLUSION 2-2: The data available for patient-centered outcomes research are often collected for reasons other than research, which limits their usefulness. Opportunities exist for increasing the utility of the data infrastructure by carefully considering the multiple uses to which the data might be applied.
From page 102...
... While such limitations are to be expected, opportunities exist for capturing data that are better able to characterize these complexities. A robust data infrastructure builds on the strengths of what is available today and has the flexibility to adapt, both as measures and terminologies become obsolete and as new technologies emerge.
From page 103...
... ? • What questions cannot be answered and who is not served by the current PCOR data infrastructure?
From page 104...
... Angove discussed challenges associated with obtaining patient agree ments for participating in data collection as well as broader engagement in research. PAF's experience and the research it has done on this topic indicate that most patients have a very limited understanding of research, and especially of terminology such as "comparative effectiveness" research and "patient-centered outcomes" research.
From page 105...
... To do this, it is essential for electronic health records to capture patient-centered outcomes in a systematic way, with special attention paid to confirming that data are collected from underrepresented groups and groups disproportionately harmed by dementia. Finally, out comes of ePCTs must be relevant and usable by decision makers, including health care systems.
From page 106...
... Epstein-Lubow argued that the broadened statutory scope for PCOR increases opportunities for learning about patient-centered outcomes but also involves some risks. For example, if inadequate attention is paid to underrepresented groups, including members of groups at higher risks of negative health effects of dementia, this could lead to underrepresented patient-centered outcomes data in these areas.
From page 107...
... , a nonprofit association of more than 140 health related organizations, including leading patient advocacy groups. She iden tified a list of key data needs based on what the NHC's patient groups are hearing from the patients these organizations serve, and based on NHC's work in the areas of regulations, real-world evidence, and value assessment: • patient-centered outcomes and impacts that really matter to patients, collected consistently; • burden, including costs incurred by patients and their families; • social determinants of health (for example, transportation, housing)
From page 108...
... Oehrlein said that the broadened statutory scope for PCOR presents opportunities to more effectively assess treatment alternatives and value from the patients' perspective. In particular, she finds that patient advocates and patients are often surprised to discover that data on out-of-pocket costs and other costs important to patients have not been systematically col lected.
From page 109...
... The data include individually generated, individually permissioned data, such as data from wearables and environment data, as well as system-generated individually permissioned data, such as data from electronic health records. Figure 3-1 illustrates the range of data sources considered by Evidation.
From page 111...
... Examples discussed included predicting flare events in an autoimmune condition using wearable and survey data; enhancing recovery modeling for limb surgeries with personalized predictions of out come tailored to individual characteristics; and early detection, monitoring, and management of COVID-19 in everyday life. For a potential roadmap of how to achieve the full potential of person generated health data and digital clinical measures, Patrick-Lake referenced
From page 113...
... CONCLUSIONS Representatives of patient organizations argued that PCOR data are often not focused on the types of issues that are truly important to people and that would enable them to find answers to the questions they tend to have about their treatment options and potential outcomes. Information on costs was highlighted as particularly important, which is in line with the goals of the recently broadened scope of PCOR to take into consideration "the potential burdens and economic impacts of the utilization of medical treatments, items, and services." CONCLUSION 3-1: The patient-centered outcomes research data infrastructure has not reached its full potential to provide data that can answer questions that matter to patients and enable them to make informed decisions.
From page 115...
... • What questions cannot be answered and who is not served by the current PCOR data infrastructure? • What new data sources could be incorporated into the PCOR data infrastructure?
From page 116...
... COVID cases o Secure computing environment, collaborative ecosystem o Sample size/coverage to allow analysis of levels, seasonality, location, racial subgroups marizes Meltzer's observations related to the usefulness of the PCOR data based on this work. Andrew Bazemore, American Board of Family Medicine, noted that although we live in a time of unprecedented health-data availability, there are also some structural blind spots in the U.S.
From page 117...
... . NOTE: COVID = coronavirus disease 2019; EHR = electronic health records; NCATS = National Center for Advancing Translational Sciences; PCOR = patient-centered outcomes research; PRO = patient-reported outcome; RCT = randomized controlled trial; Rx = prescrip tion; VA = U.S.
From page 118...
... Among the data sources that are missing from the current PCOR data infrastructure, according to Bazemore, are not only primary care practices themselves but also two other data sources: primary care registries and health information exchanges focused on primary care and the safety net. On the analytical side, Bazemore noted that a major concern is that new technologies such as artificial intelligence and machine learning do not typi cally involve primary care patients in their algorithm development.
From page 119...
... It also provides opportunities to support those on the front lines trying to contextualize care, as well as to support relationship centered care. Concerning the question of who is poorly served by the current PCOR data infrastructure, Stange underscored previous points about the limita tions of the data on people living with multiple chronic conditions and various disadvantaged groups.
From page 120...
... For additional data that could be incorporated into the PCOR data infrastructure, Stange suggested the Person-Centered Primary Care Mea sures, which he and his coauthors developed based on what patients, clini cians, and (to a lesser extent) payors said was important to them in health care.3 Box 4-2 shows these measures, which Stange said are widely used and are also pending endorsement by the National Quality Forum and the Centers for Medicare & Medicaid Services (CMS)
From page 121...
... This includes questions related to the roles of deep molecular, behavioral, and social determinants of health, given that it has long been recognized that social, cultural, behavioral, and biologi cal determinants not only interact to combine in complex ways to impact individual outcomes, but also have common dimensions across groups of people. Going forward, it will be important to look at opportuni ties afforded by different ways to integrate information using a different approach to computing.
From page 122...
... There are rising health concerns specific to both of these populations, and therefore there is a need to develop approaches that would address the data limitations associated with these groups. David Cella, Northwestern University, summarized key data needs as a common data model for patient-reported outcomes; common data ele ments for patient-reported outcomes; comparative effectiveness metrics across conditions; and medical and nonmedical cost data.
From page 123...
... He echoed earlier points about the difficulty of accessing patient-reported outcomes data, particularly through the Chicago Area Patient-Centered Outcomes Research Network, which is 1 of 13 Clini cal Data Research Networks. Cella mentioned that in 2014 he was part of the PCORnet Patient Reported Outcomes Common Measures Working Group, which recom mended a set of common measures for PCORnet.
From page 124...
... This lack of data and these erasures diminish the potential of PCOR to advance health equity. The data infrastructure needs to be robust enough to allow data to be disaggregated in ways that can detect differences among small populations.
From page 125...
... Corbie-Smith also highlighted the opportunities presented by includ ing networks of community service providers in the research. These stake holders include not only federally qualified community health centers but also community service providers that are providing a matrix of care.
From page 126...
... Corbie-Smith also underscored the need for transparency in research and around data democratization in order to demonstrate the trustworthiness of science, which fundamentally is what is needed to move forward. Scott Ramsey, Fred Hutchinson Institute for Cancer Outcomes Research (HICOR)
From page 127...
... Those data typically focus on what happens in the health care system, whereas, he believes, in many cases the individual's experiences and envi ronment play a bigger role in their care and outcomes. The social determinants of health are generally not captured in tradi tional claims and electronic health records databases.
From page 128...
... Researchers described a variety of barriers that limit their ability to access the data available in the many existing databases, ranging from databases that can be considered a part of the PCOR data infrastructure to databases owned by private companies. Focusing on facilitating and sim plifying access represents an area that could further enhance the usefulness of PCOR data.
From page 129...
... Researchers echoed the need to make PCOR data more widely available to empower patients and communities to use this information. Efforts to reduce disparities, in particular, cannot be accomplished by research alone.
From page 131...
... is a senior fellow at the HealthPartners Institute and a senior advisor for the Alliance of Community Health Plans. Previously, he served as a senior advisor to the board of directors and the senior management team of HealthPartners, and prior to that, he was HealthPartners' medical director and chief health officer, responsible for quality of care and health and health care improvement.
From page 132...
... Previously, she has worked for clinical technol ogy and pharmaceutical organizations, where she has led implementation of various technology solutions focused on business optimization and acceler ating the delivery of new therapies to patients safely. She is currently serv ing as principal investigator on a study to enhance engagement, research participation, and collaboration through the IBD Partners Patient Powered Research Network.
From page 133...
... He is a leading authority on the evalu ation of diagnostic and screening tests and has made major contributions to the development of methods for medical technology assessment and health services and outcomes research. He is a world leader in methods for applying and synthesizing evidence on diagnostic tests in medicine and is currently developing methods for comparative effectiveness research in diagnosis and prediction and radiomics.
From page 134...
... His current research is on the clinical information stored in electronic health records. Using data mining techniques, he is developing the methods necessary to support clini cal research and patient safety initiatives.
From page 135...
... She is the lead of Kaiser Permanente & Stra tegic Partners Patient Outcomes Research To Advance Learning (PORTAL) Network.
From page 136...
... He has directed innovation and technology teams in provider organizations, academic insti tutions, corporate research organizations, and product development organi zations. Most recently, he led the creation, development, deployment, and evaluation of the application of artificial intelligence to physician point of-care solutions integrated within an electronic health record system.
From page 137...
... • Discuss what HHS is best positioned to address and how the agency could maximize resources available for the PCOR data infrastructure (representing 4% of the PCOR trust fund) , in the context of the HHS public mission, authorities, programs, and data resources.
From page 138...
... Smith, Director, Division of Healthcare Quality and Outcomes, ASPE 11:30 am-1:00 pm EDT PCOR Data Infrastructure Limitations and Opportunities: Disparities and Health Equity Research Discussion questions: • What are the limitations of the PCOR data infrastructure in terms of: o disparities in the data, including knowledge about patient outcomes, taking into consid eration differences in patient preferences and values o challenges associated with using the data to understand disparities and health equity o lack of data on some populations • What are opportunities and priorities for enhancing data capacity in this area? • What data capacity challenges is HHS best positioned to address in the context of their public mission, authorities, programs, and data resources?
From page 139...
... statutory scope for PCOR? • What questions cannot be answered and who is not served by the current PCOR data infrastructure?
From page 140...
... statutory scope for PCOR? • What questions cannot be answered and who is not served by the current PCOR data infrastructure?
From page 141...
... REBEKAH ANGOVE is vice president for patient experience and pro gram evaluation at Patient Advocate Foundation (PAF) , where her work is focused on strategically expanding PAF's patient-centered research and pro gram evaluation initiatives.
From page 142...
... He is a nationally and internationally recognized leader in cardiovascular medicine, health outcomes research, health care quality, and clinical research, and one of the most frequently cited authors in biomedical science. Califf received his M.D.
From page 143...
... Corbie-Smith cur rently serves as a multiple principal investigator on the National Institutes of Health/National Institute on Minority Health and Health Disparities– funded RADx-Underserved Populations Coordination and Data Collection Center. Corbie-Smith received her M.Sc.
From page 144...
... Epstein Lubow is a geriatric psychiatrist with research, teaching, policy, clinical, and administrative expertise related to geriatric psychiatry patients and family caregiver health, and serves as team leader of the stakeholder engagement team for the National Institute on Aging's Imbedded Pragmatic Alzheimer's Disease and AD-Related Dementias Clinical Trials Collaboratory. He is also an associate director of the Centers for Disease Control and Preven tion BOLD Public Health Center of Excellence for Dementia Caregiving.
From page 145...
... He is the lead of the University of Chicago network site as part of the Chicago Area Patient Centered Outcomes Research Network.
From page 146...
... in pharmaceutical health services research with a concentration in comparative effectiveness research/patient-centered outcomes research from the University of Maryland School of Pharmacy. BRAY PATRICK-LAKE is senior director of Strategic Partnerships at Evi dation Health, where she develops collaborations to support the design and implementation of participant-centered studies and the regulatory and clinical acceptance of digital measures.
From page 147...
... His division conducts research on how health policies influence health care quality and outcomes in state and federal programs. In addition, the division is responsible for managing the patient-centered outcomes research data infrastructure portfolio across HHS.
From page 148...
... He is a family and public health physician, practicing at Neighbor hood Family Practice, a federally qualified community health center in Cleveland, Ohio. Stange is active in practice-based, multimethod, par ticipatory research and development that aims to understand and improve primary health care and community health.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.