12

Workshop 4, Session 2: Informing Evidence-Based Policy Making

CONCEPTUALIZING AND CONDUCTING POLICY-RELEVANT RESEARCH

Renuka Tipirneni is an assistant professor in the Department of Internal Medicine and faculty adviser to the policy engagement team at the University of Michigan’s Institute for Healthcare Policy and Innovation. She began her presentation by discussing how to pose research questions that will be relevant to health policy. Tipirneni offered suggestions for how to best conceptualize policy-relevant research:

• Read the actual bill, regulation, or court case, for it is essential to understand the details through all the technical language.
• Think of questions that are both interesting and addressable in a time frame that would be useful for policy makers.
• Engage with policy stakeholders early in the process of formulating research questions to understand what those stakeholders want to know, what time frame is ideal, and what aspects of the research questions have the greatest potential to impact policy.

She used her own work evaluating Michigan’s Medicaid expansion as an example. When the bill authorizing the state’s Medicaid expansion passed, she read each provision of the law and asked herself about the importance of that provision and how she might be able to assess its impact. She paid particular attention to the provision that required the Department of Health to ensure that all Medicaid enrollees have access to primary care and that the department should require all new enrollees to be assigned to and have an appointment scheduled with a primary care physician within 60 days of enrollment, a provision that was later extended to 90 days. Tipirneni noted that as a primary care provider herself, she wondered how the department would be able to ensure and monitor access to primary care providers for patients with Medicaid within those time frames.

Tipirneni considered several methods when she was developing a rigorous study to answer a policy-relevant question that examined both appointment availability and wait times for patients who had Medicaid coverage:

• Conduct a secondary data analysis using federal or state surveys that monitor access to appointments or access to primary care practitioners.
• Analyze visits documented in health care claims data to see when and how often people visited their primary care provider.
• Survey primary care practitioners or their health systems to identify how they are handling capacity issues and if they could meet the demand for patients who needed an appointment within the designated time frame.

• Survey or interview enrollees to ask them whether they were able to get an appointment in a timely manner and if they were able to get the care they needed.
• Conduct a simulated patient, or “secret shopper,” study of calling clinics as if one were a patient to see what kind of appointment access was available.

Tipirneni explained that as a member of the formal evaluation team for Michigan’s Medicaid expansion, she has used each of these methods for different studies. She said she used the simulated patient method for this particular study because she was interested in both appointment availability and wait times.

Tipirneni offered several suggestions for conducting effective policy-relevant research. She said researchers should first consider a variety of methods, such as secondary data analysis or primary data collection, given that there is no one best approach for considering a research question. Considerations such as time frame for the study, cost, or availability of data should inform method selection. She reiterated that researchers should be cognizant of the allotted time frame when selecting their research method. For example, she ultimately chose to use the simulated patient approach because she needed answers quickly and that was feasible for her and her team. Researchers should incorporate input from collaborating stakeholders when developing and refining the study protocol. She also noted the importance of building a research team that has the necessary expertise and can be nimble to address questions in real time.

Tipirneni next discussed translating research findings to achieve policy impact. She said this process begins with developing products that are useful for different audiences. As an example, she said her team presented the results of a single study in three different forms. She and her collaborators published a traditional peer-reviewed academic paper in a well-respected journal to present results to researchers. The team created an issue brief that it could pair with the journal article for a national audience of both media and policy makers. For stakeholders in the state’s Medicaid program—the audience she most wanted to inform—the team prepared an issue brief that she updated regularly as the team collected additional data. The issue brief focused on the information stakeholders prioritized: appointment availability and the regions in the state that had better access and worse access. Tipirneni noted that each document presented the results in a different order and with a different emphasis designed to provide information in a form most accessible and useful for each document’s intended audience. The issue briefs, for example, had policy considerations and implications at the start of the document.

Tipirneni offered several overall suggestions for effectively translating research for policy impact, including the following:

• Identifying the target audience and when to disseminate information to the intended audience, including at the local, state, or national level.
• Engaging policy stakeholders early and often.
• Knowing what information the intended audiences are most interested in learning, and present that information in a format that they are accustomed to seeing.
• Writing issue briefs carefully and in a nonpartisan manner, paying close attention to wording so as to not alienate policy makers of one political party.
• Continuing to engage and develop relationships among stakeholders in the research and policy communities to facilitate long term impact.

In closing, Tipirneni summarized her suggestions for researchers seeking to use their study findings to inform policy. Those suggestions included

• reading the policy or piece of legislation prior to generating research questions to address that legislation or policy;
• engaging with policy stakeholders early and often during the research process;
• considering a variety of methods, including secondary data analysis and primary data collection;
• framing research as an iterative process that involves developing questions and methods in collaboration with both researchers and policy stakeholders;
• writing issue briefs, not just academic papers, to translate research findings for policy stakeholders; and
• developing relationships with policy makers and maintain them over time to truly have an impact on policy.

CHANGING ACCESS TO CARE FOR UNDOCUMENTED IMMIGRANTS WITH DATA

Lilia Cervantes is director of immigrant health and an associate professor in the Department of Medicine at the University of Colorado Anschutz Medical Campus. She began by describing the Latina/o community in the United States. They are a heterogeneous population with respect to immigration generation and status, country, language of origin, and race. She said the Latino community faces a disproportionate burden of social challenges. She said the poverty rate among Latinos is 17 percent, about double the poverty

rate among non-Latino White individuals. In addition, nearly a third of individuals in the Latino community report limited English proficiency, and Latino patients often do not have access to interpreters when needed in health care systems. She said the 8 million members of the Latino community that are undocumented are the least likely of any racial or ethnic group to have health insurance.

Cervantes explained that she was inspired to pursue research as a means of changing health policy when caring for a patient with end-stage kidney disease who was undocumented and a mother of two boys. Instead of receiving dialysis three times a week, she came to the hospital every 7 days for emergency dialysis when she was critically ill. Her two sons faced significant physical and psychosocial distress on a weekly basis because they did not know if their mother would survive to the following week. After her third cardiac arrest, the patient decided to stop emergency dialysis, found a couple to adopt her sons, and passed away on Mother’s Day 2014.

Cervantes explained that U.S. citizens can receive Medicare-covered kidney replacement therapy, including dialysis and kidney transplantation, because of the 1972 Medicare end-stage renal disease entitlement program.¹ However, the availability of kidney replacement therapy varies by state for undocumented immigrants, who are excluded from Medicare, most Medicaid programs, and provisions of the Affordable Care Act. In 1986, Congress passed the Emergency Medical Treatment and Active Labor Act,² which prohibits refusal of care in an emergency situation regardless of immigration or insurance status. Cervantes said this legislation motivated every state to create its own emergency Medicaid program. Many states adopted the Emergency Medical Treatment and Active Labor Act’s exact language that only requires reimbursement for care when an undocumented immigrant is critically ill. She explained that the Centers for Medicare and Medicaid Services (CMS) does not provide regulatory guidance defining an emergency medical condition, instead deferring to the states to determine the qualifying conditions (Figure 12-1).

While considering how to change policy in her state of Colorado, Cervantes found a perspective article that challenged the nephrology community, and physicians in general, to obtain more data to best define options to reform state policies for undocumented immigrants with kidney failure (Straube, 2009). She began by engaging various stakeholders in qualitative research and connecting with decision makers to identify the research questions that they felt were important and whose answers could persuade them to change policy. Patients were her primary partners in this work, so she also

___________________

¹Social Security Amendment of 1972, Public Law 603, 92nd Cong., 1st sess. (October 30, 1972) §2991.

²Emergency Medical Treatment and Active Labor Act, 41 U.S.C. § 1395dd.

conducted qualitative interviews with patients and their families (Cervantes et al., 2020a). She and her colleagues also conducted qualitative interviews with 50 interdisciplinary clinicians and learned that many of them faced burnout from witnessing needless suffering and high mortality, experienced physical exhaustion from overextending themselves to provide bridge care, and were detaching so as to numb themselves from feeling too much empathy for these patients (Cervantes et al., 2018a).

Cervantes also assessed mortality change or mortality differences between patients who rely on emergency dialysis and those that receive standard dialysis. The results showed that patients who receive emergency dialysis had a 14-fold greater mortality within 5 years of initiating dialysis and a nearly 10-fold greater utilization of services resulting from emergency department visits and hospitalization (Cervantes et al., 2018b). She said that at the time of her study, investigators in Dallas, Texas, were conducting a study in which they enrolled a subset of undocumented immigrants with kidney failure into private health insurance off the state insurance exchange thanks to a private philanthropist (Nguyen et al., 2019). The research team followed the patients for a year and assessed mortality and cost. They found patients who received emergency dialysis had a mortality that was five-fold greater at 1 year than patients enrolled in private insurance who were able to receive standard care (Nguyen et al., 2019). In addition, care for patients who went on standard dialysis cost approximately $6,000 per person per month less than the cost of care for those who relied on emergency dialysis.

Cervantes and her collaborators developed relationships with community-based organizations while conducting their research. Those community-based organizations were able to connect them with policy stakeholders including the state Medicaid agency and the governor’s office. Cervantes and her colleagues provided policy stakeholders with research evidence to support policy change. Colorado’s Medicaid agency decided in February 2019 to expand access to standard dialysis by including it as a service under the Emergency Medicaid Program.

Cervantes collaborated with a former state Medicaid director and several other colleagues to write a piece for the Clinical Journal of the American Society of Nephrologists that described the steps they took to achieve this policy change (Cervantes et al., 2020b). She also partnered with the National Kidney Foundation to send an open letter to state Medicaid directors, co-signed by several national physician organizations, outlining this issue and describing the steps to create policy change (American College of Physicians Colorado Chapter et al., 2021).

Cervantes and her colleagues continue to engage in the evidence dissemination strategies they learned from the dialysis project to address other health care policy matters. During the COVID-19 pandemic, Cervantes and her

colleagues learned that patients who were undocumented did not have access to outpatient COVID-19 care. Cervantes and her collaborators conducted a qualitative study that identified several themes: patients described misinformation and disbelief; they relied heavily on social media for information about COVID-19; they thought COVID-19 was a compounder of disadvantage because they had no safeguards such as health insurance or paid sick leave; and they described a reluctance to seek medical care because they worried about being turned away or deported or about the cost of care (Cervantes et al., 2021a). A retrospective analysis of all individuals who had been hospitalized since the onset of the pandemic found that of the 1,000 people she and her collaborators identified, Latinas/os accounted for two-thirds of the hospitalizations and half of the deaths (Podewils et al., 2020). Cervantes and her colleagues combined those qualitative and quantitative research findings to present evidence to support policy change to the state Medicaid agency. This contributed to an expansion of emergency Medicaid care to include outpatient COVID-19 as a qualifying condition. Cervantes and her collaborators used their data to reduce misinformation throughout the state. They also received funding to launch a community health worker program that provided culture- and language-concordant COVID-19 information and COVID-19 vaccinations.

Cervantes closed by noting her team received three times as many media requests to highlight an issue after publishing their qualitative findings compared to when they published their quantitative findings. She said qualitative and mixed methods research can provide needed context when communicating research evidence to policy makers.

DISSEMINATION RESEARCH TO PROMOTE EVIDENCE-INFORMED POLICY MAKING

Jonathan Purtle is an associate professor of public health policy in management and director of policy research at New York University’s Global Center for Implementation Science. He began by explaining policy making is more of an art than a science, but he would be addressing the science part of policy making (specifically, dissemination research). He would also discuss how he and his collaborators have applied research methods to determine how to better infuse research evidence into policy making processes. Dissemination research, he explained, is a branch within the broader field of implementation science that is concerned with packaging and communicating research evidence to different audiences. According to the National Institutes of Health definition, “dissemination research is the scientific study of targeted distribution of information and intervention materials to a specific public health or clinical practice audience. The intent of dissemination research is

to understand how best to spread and sustain knowledge and the associated evidence-based interventions.”³

Purtle said he was drawn to the field of dissemination research because he wanted to understand how to translate high-quality, policy-relevant research into public policy in general and state legislation in particular. He said most of the 7,383 state legislators in the United States do not have expertise in health issues. “We want to understand how to shape their knowledge and their attitudes in ways that are aligned with evidence,” said Purtle, “and we want the best policies on the books from a population health and health equity perspective.” He explained dissemination research resembles communication research in that the goal is to understand how to better communicate about research. However, dissemination research is unique in that dissemination involves communicating about evidence to a clinical practice audience. He noted that the Agency for Healthcare Research and Quality (AHRQ) published a systematic review of communication and dissemination strategies focused mainly on health care providers (McCormack et al., 2013). Purtle and his colleagues published a review in 2020 that laid out a three-phase approach to conducting dissemination research (Purtle et al., 2020).

Purtle said his work, which is focused on policy makers, is informed by a personal observation that while systematic reviews present intensively reviewed evidence for developing practice guidelines, a policy brief is typically based on anecdotal evidence. “We contend that by doing research about the dissemination of research evidence, we can improve how evidence spreads, and we can change knowledge and attitudes in policy makers in ways that we want to see,” said Purtle. He added the caveat that while dissemination research can improve how evidence spreads, it does not guarantee that evidence will consistently inform policy because of the nature of the policy making process,

The first phase for dissemination is to conduct formative audience research. He said formative audience research is the broad, descriptive research to understand a target audience of policy makers. This phase informs how to design dissemination materials and the channels and sources through which to distribute them. Purtle explained that the first aim of formative audience research is to gain information about the intended audience’s awareness of the problem a given policy is seeking to address. “Often, as researchers, we have deep content expertise and solutions to problems, but state legislators and the general public do not see those problems as problems yet. They do not have the background information so often we have to educate them about that first before we can share evidence about policy solutions,” said Purtle. For example, Purtle and his collaborators surveyed 475 U.S. state legislators to see if they

___________________

³ Available at https://grants.nih.gov/grants/guide/pa-files/PAR-18-007.html (accessed September 14, 2022).

were aware of the large body of epidemiologic evidence showing that adverse childhood experiences are risk factors for mental health and substance use issues in adolescence and adulthood (Purtle et al., 2019a). Knowing whether their knowledge aligned with the evidence on this particular issue would help inform what a policy brief might emphasize or not need to emphasize. Formative audience research also aims to determine policy makers’ attitudes about a specific evidence-supported policy. When Purtle’s team surveyed legislators’ attitudes related to state behavioral health parity laws, they found that nearly three-quarters of the legislators believed the evidence that parity laws increase access to behavioral health (Purtle et al., 2019b). However, only around half of the legislators believed the evidence that mental health and substance use disorder treatments can be effective, and only 16 percent believed the evidence that parity laws do not increase insurance premiums. These results helped the team identify topics around which to design research. Purtle and his collaborators also found that 40 percent of legislators surveyed did support behavioral health parity laws (Purtle et al., 2019b). The research team also looked at the adjusted odds ratio comparing whether legislators supported such laws and their beliefs about the effectiveness of those laws, the associated costs, treatment effectiveness, and political ideology. The team used that analysis to gain insight into the types of knowledge or attitudes they might want to shift to enhance support for policy aligned with evidence.

The second phase of dissemination research is audience segmentation research. Audience segmentation research seeks to generate data to inform how to design and distribute dissemination materials specifically to different groups of policy makers and legislators within a given target audience. Purtle said conducting audience segmentation research is standard practice in marketing, advertising, and health communications. He explained the general premise of audience segmentation research is that there will be a great deal of heterogeneity in a population in terms of their knowledge, attitudes, and behaviors related to a specific issue. There are two ways to conduct audience segmentation research: demographic separation and empirical clustering. Demographic separation involves stratifying an audience by a demographic variable of interest. For example, liberal, moderate, and conservative legislators have different opinions about the role of adverse childhood experiences as risk factors for adult behavioral health conditions (Table 12-1) (Purtle et al., 2019a). Empirical clustering uses a method such as latent class analysis or K-means clustering to identify different audience segments (Hagenaars and McCutcheon, 2002; Shukla and Naganna, 2014). From the same survey of 475 state legislators, Purtle identified three different audience segments related to the effectiveness of behavioral health and substance use disorder treatments: budget-oriented skeptics with a negative bias toward people with mental illness, passive supporters, and action-oriented supporters (Purtle et al., 2018). The research

TABLE 12-1 Audience Segmentation According to Beliefs About Mental Health and Substance Use Disorder

SOURCE: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-018-0816-8. Used under Creative Commons license CC BY 4.0, available at https://creativecommons.org/licenses/by/4.0/. No changes made. Presented by Jonathan Purtle on July 6, 2022, at Accelerating the Use of Findings from Patient-Centered Outcomes Research in Clinical Practice to Improve Health and Health Care: A Workshop Series (data from Purtle et al., 2019a).

team then input different variables into the latent class analysis for these three groups to identify factors that would influence their support for a behavioral health bill that would be introduced in their state legislature. Purtle explained that based on the results of that analysis, “we can think about how we might craft a policy brief and messages for legislators in the budget-oriented skeptics with stigma [negative bias] group that might be much different from what we emphasize in a policy brief and messages for legislators in the action-oriented supporter or passive supporter groups.”

The third phase of dissemination research focused on policy makers is dissemination effectiveness research. Dissemination effectiveness research involves conducting randomized controlled trials of different dissemination strategies to see which strategies are most effective. As an example, Purtle discussed a cluster-randomized controlled trial of all U.S. state legislatures to determine how much it matters to legislators if dissemination materials include economic evidence and how much it matters to have state-tailored evidence as opposed to national data (Purtle et al., 2022). His team randomized states and disseminated different policy briefs by email. The research team then tracked engagement with the evidence by email views and the rate at which legislators clicked links in the email to view the policy briefs. All the emails concluded with an offer for expert consultation for more information, which they tracked as an outcome. They also tracked mentions of adverse childhood experiences and related terms on legislators’ social media accounts two months after sending materials. The results, said Purtle, indicated that including economic evidence increased engagement with dissemination materials among Democrats but not Republicans, which they did not expect (Purtle et al., 2022). He said that finding was useful when crafting different dissemination plans to reach Democrats and Republicans.

Purtle concluded his remarks with a quote from the late scholar Carol H. Weiss, who said, “The policy making process is a political process, with the basic aim of reconciling interests in order to negotiate a consensus, not of implementing logic and truth. The value issues in policy making cannot be settled by referring to research findings.” Purtle said policy making is a political act, something researchers such as himself find frustrating, but it is an important reality to embrace because it is not going to change.

DISCUSSION

Strengthening Relationships between Researchers and Policy Makers

Session moderator Lauren Hughes opened the discussion by asking the panelists for insights on how to strengthen the relationships between policy makers and researchers. Cervantes replied that she had success with using the connections she had with community-based organizations who had relationships with policy makers. Purtle suggested identifying the legislators who have a demonstrated interest in an issue to be addressed and reaching out to them specifically to start cultivating a relationship. He noted researchers should consider reaching out as constituents to their own elective officials.

Considerations for Informing Policy

Hughes asked the panelists to discuss how to effectively leverage narrative evidence to advance policy. Cervantes suggested researchers engage patients and stakeholders who are interested in the issue to be addressed. Researchers can use stories from those patients and stakeholders as narrative evidence to present to a legislator or decision maker as an illustration of what patients and stakeholders want to be part of a solution to the issue. She includes quotes from patients and stakeholders in policy briefs and brings patients and other stakeholders to meetings. Purtle added that narratives work to influence policy makers because, as research has shown, they are more engaging and easier to process than experimental results and data. However, narratives can have unintended consequences. He cited one study showing that narratives work to garner support from Democrats for policies that would improve food security, but that they reduced support among Republicans.

Hughes asked speakers to discuss approaches for conceptualizing policy-relevant questions, Purtle replied that he seeks as many perspectives as possible. Cervantes agreed with Purtle, and noted the importance of including opponents to the policy to be addressed to get their perspectives. She also finds understanding the policy pathway in other states and the data that produced change in those states to be informative.

Considering Different Dissemination Tools

Hughes asked the panelists for examples of dissemination products beyond peer-reviewed publications that they have found useful in their work. Cervantes said she found social medal useful, adding that a Twitter tutorial comprising a series of tweets, each with a specific nugget of information, proved important in both her kidney dialysis and COVID-19 projects. She also noted the importance of including patients in dissemination efforts because they are often unaware of policies that can benefit them. Purtle agreed that Twitter is a useful dissemination tool, especially if the tweet contains links to additional information so that users can click and learn more about an issue. He also includes his phone number in emails and policy briefs that he sends to legislators, which often results in phone calls from the legislator or staff member.

Supporting Translating Research to Policy

An audience member then asked speakers to discuss their ideas for how AHRQ or other funders can support infrastructure development at research institutions and provide incentives to support researchers who want to engage in the policy space. Purtle noted that many academics are hesitant to work on

policy issues because they believe they will be penalized in terms of tenure and promotion, so perhaps AHRQ could emphasize that this type of work should be valued, rewarded, and supported. Cervantes added that academic institutions are realizing that service and engagement in advocacy work is important for promotion. In that regard, AHRQ funding mechanisms could shift some of its funding to reflect policy change as an important endpoint for research, particularly as it affects upstream drivers of health inequities. She suggested AHRQ might also develop a curriculum that would educate clinicians and researchers about engaging in research that changes policy. She said such a curriculum would include how to put a policy brief together and how to think about research questions that can lead to policy change.

Speakers’ Advice for Researchers

Hughes asked the panelists what advice they would give to a colleague considering engaging in policy-related work. Cervantes said that she advises her medical students and residents who want to engage in policy that it is important to engage with stakeholders, particularly community-based organizations, who have policy experience and can lend their expertise. Purtle noted that to a newcomer, policy making can seem so complex that it is overwhelming. He advises people interested in engaging in policy work to first identify the exact policy that needs changing, who has the authority to change that policy, and narrow the issue to a reasonable scale.

CLOSING SUMMARY OF WORKSHOP 4

Hughes concluded the workshop by summarizing her takeaways from the presentations and discussions. She described several key points highlighted by speakers about various aspects of effective communication of research findings:⁴

• Being aware of the effect of language on how the message is conveyed and its reach
• Engaging trusted messengers who look and sound like the patients and families that research is intending to serve
• Remembering the public, separate from patients, providers, and policy makers, is a critical constituency to which dissemination of information needs to occur

___________________

⁴ These points were made by the individual workshop speakers/participants identified above. They are not intended to reflect a consensus among workshop participants.

• Communicating research findings to the public, patients, providers, and policy makers frequently requires the message to get through a great deal of noise, stress, and inequities in the current system
• Considering the effects of health literacy and readiness to change when planning communication efforts
• Understanding how subject matter experts can use media platforms to disseminate accurate information
• Considering integrating health communications professionals into a research team to support effective dissemination

Hughes noted that speakers provided several suggestions to guide researchers to develop policy-relevant research questions including: reading the bill, law, or court case to be addressed by the research and determining whether the research question is realistic and addressable within a specified time frame. Hughes noted there was a substantial discussion about translating evidence into policy and approaches for disseminating research findings to policy makers that goes beyond authoring peer-reviewed papers for the academic literature. Speakers had several suggestions for best approaches to disseminating research evidence to policy makers including⁵

• Knowing the audience, including those who hold different opinions
• Selecting communication materials that are most effective and useful for each audience
• Avoiding language that could alienate policy makers of one political party
• Incorporating narrative evidence to provide context to quantitative data
• Understanding the baseline awareness of the evidence and attitudes toward interventions among policy makers
• Conducting audience segmentation to better understand knowledge, attitudes, and behaviors that will inform and allow for tailoring communications and dissemination plans

___________________

⁵ These points were made by the individual workshop speakers/participants identified above. They are not intended to reflect a consensus among workshop participants.

This page intentionally left blank.