3

Quality Measurement and Quality Improvement

Quality measurement has been characterized as “fundamental to systematic improvement of the healthcare system” (Burstin et al., 2016). In the early 2000s, two notable reports from the Institute of Medicine (IOM), To Err is Human (2000) and Crossing the Quality Chasm (2001a), fostered a national conversation about the quality of health care. At that time, the IOM defined the six aims of quality improvement in health care—i.e., to make health care more safe, timely, equitable, efficient, effective, and patient centered (IOM, 2001a). This chapter provides an overview of quality measurement in the nursing home setting and describes how such measures can, in turn, be used to improve the quality of care for residents of nursing homes.

THE PURPOSE OF QUALITY MEASUREMENT

The primary purpose of quality measurement is to support work aimed at improving the quality of care and outcomes (Conway et al., 2013; Rantz et al., 2002). The Agency for Healthcare Research and Quality (AHRQ) defines quality improvement as

In addition to this overarching goal of quality improvement, quality measurement serves several other distinct purposes, including

• assisting consumers in making choices about providers and facilities;
• ensuring care providers’ accountability for outcomes, including through public reporting, value-based purchasing programs, and accreditation or certification;
• providing evidence to inform treatment decisions, optimize clinical interventions, and elucidate the effectiveness of interventions on patient and family outcomes;
• guiding quality improvement activities; and
• producing new knowledge through clinical and health services research to guide policy (AHRQ, 2012; Basch et al., 2013; Conway et al., 2013; Landrum et al., 2019).

Accurate and interpretable information on nursing home quality needs to be readily available to all those seeking nursing home care as well as to the friends or family that may help with nursing home decisions. Faced with a determination concerning the need for nursing home care, people may feel overwhelmed with anxiety and questions, such as: How do I choose? How can I be sure the care will be what I need? Will the care be of good quality, and how will I know if the quality is good? How can I get the best possible care that I want and now need? In addition, such information needs to be available to public and private payers for nursing home care, namely the Medicare and Medicaid state and federal payers, private insurance companies, and individuals who have the resources to pay privately. Payers want to know that they are paying for high-quality care.

State and federal regulators of nursing homes are charged with the responsibility of ensuring a level of quality of care that “protects” the public (IOM, 2001b). This protection is a basic assurance that the quality of care meets a fundamental level so that, when needed, safe nursing home care is available in a community. However, people often say that they want “high-quality care” or the “best care” instead of care that meets a minimal, fundamentally safe level. This desire to know when nursing homes actually provide high-quality care underlies the need for good measures of quality in nursing homes.

Nursing homes in the United States and around the world face unprecedented times, as the COVID-19 pandemic has had a disproportionate impact on nursing homes where the most vulnerable older adults live and receive care (Cockburn, 2020). Friends and family who were barred from entry to places they had been visiting frequently struggle with the loss of family members and loved ones they could not visit and comfort

during the end of life. With thousands dying in a short period of time, the public sentiment has been that the “protections with basic assurance of quality” have been lost to widespread poor care in nursing homes where fundamental basic needs go unmet for days or weeks on end (Cockburn, 2020; Karlawish et al., 2020; Simpson, 2020). The tragedy of the pandemic provides an opportunity for re-envisioning nursing home care using quality measurements to guide those changes.

PRINCIPLES AND DEFINITIONS

The Centers for Medicare & Medicaid Services (CMS) defines quality measures as “tools that help us measure or quantify healthcare processes, outcomes, patient perceptions, and organizational structure and/or systems that are associated with the ability to provide high-quality health care and/or that relate to one or more quality goals for health care. These goals include effective, safe, efficient, patient-centered, equitable, and timely care” (CMS, 2020a). Measuring the quality of care relies on comparing that care against recognized standards of care (NQF, 2021). Therefore, individual quality measures facilitate consistent comparisons against those standards. Donabedian’s framework is commonly used to categorize quality measures as measures of structure, process, or outcome (Donabedian, 1988). Structural measures include the size of the nursing home, the types and numbers of staff, and the profit or not-for-profit status of the home. Process measures include the specific steps in the systematic delivery of care, such as assessments of clinical conditions, whether personal decisions about care (e.g., end of life) are discussed and recorded for all staff to honor, or if personal care is actually completed. Outcome measures include such things as mortality rates, infection types and rates, and satisfaction with care. Many issues need to be considered when developing quality measures (see Table 3-1).

TABLE 3-1 Criteria to Develop and Evaluate/Measure Quality

SOURCE: Landrum et al., 2019.

Quality measurement in nursing homes can range from simple care measures to complex care management measures. For example, “The resident’s face is clean and free from food particles” or “The resident’s breath is fresh and free from odor” would be examples of simple care measures (Rantz et al., 2002). Complex care management measures include such things as documentation that the advanced directive form was discussed and completed with (a) items checked related to antibiotics, hydration, tube feedings, and hospitalizations; (b) other feelings and beliefs documented; and (c) the signature of the resident or responsible person and the primary care provider’s signature, with dates, obtained. Extracting simple measures of everyday care from records and observations of workflow can be an easy method to rapidly collect data, compare to past performance, develop plans to improve performance, and implement those plans (Rantz et al., 2002). Discoveries of new and better ways to treat health conditions occur regularly, and continuous measurement is essential to ensuring that such improvements are incorporated into care delivery in a timely manner. More importantly, quality data will show if those changes have produced measurable improvements for the residents of nursing homes and their families or significant others in areas that are important to them.

Any system of quality measurement needs to support quality improvement through comparative reports, often designed as feedback reports. These reports need to be easily visualized and understood so that care staff can see how making changes in clinical care will lead to improvements in quality. To be most effective, these feedback reports need to be available in as close to real time as possible and emphasized quickly at the point of care. In addition to measurement and feedback, the implementation of quality improvement efforts can benefit from other approaches, including learning collaboratives and coaching (Ivers et al., 2012, 2014; Powell et al., 2015).

Ensuring that a nursing home makes lasting quality improvement changes requires systematic follow-up and reinforcement by organizational leaders sustained over many months, if not years (Norton et al., 2018; Rantz et al., 2012a,b, 2013; Vogelsmeier et al., 2021). Quality assurance and quality improvement are therefore related in that quality assurance “involves assessing or evaluating quality; identifying problems or issues with care delivery and designing quality improvement activities to overcome them; and follow-up monitoring to make sure the activities did what they were supposed to” (Jevaji, 2016). (See Chapter 8 for more on oversight and regulatory approaches to quality assurance.)

EVOLUTION OF QUALITY MEASUREMENT IN NURSING HOMES

In the 1800s, following the Crimean War, Florence Nightingale developed multiple standards for nursing care and called for change in the

profession to achieve higher standards (Nightingale, 1859). She initiated a process to set standards for care, and then measured the care that was actually delivered in order to improve the quality of care delivered (Burstin et al., 2016; Rantz et al., 2002). This process was prompted by the appalling death rates and poor infection control practices in the Crimean War, strikingly similar to what has happened during the pandemic of COVID-19 in nursing homes in the United States and throughout the world.

The 1986 IOM report Improving the Quality of Care in Nursing Homes laid the foundation for the evolution of quality measurement based on resident assessment information, noting that a “system to obtain standardized data on residents is essential” (IOM, 1986, p. 24). That report envisioned that this assessment system would have “multiple uses both for nursing home management and for government regulatory agencies” (IOM, 1986, p. 24) and that a registered nurse would perform the assessments “upon admission, periodically, and whenever there is a change in resident status” (IOM, 1986, p. 26).

These assessments, part of the Resident Assessment Instrument process, included the Minimum Data Set (MDS)¹ for ongoing assessment of resident needs and conditions (Morris et al., 1990). In 1989, the Health Care Finance Administration (the predecessor to CMS) sponsored the multistate Nursing Home Casemix and Quality demonstration project to develop a quality measurement system using MDS data (Abt Associates Inc., 2002). These quality indicators, developed and tested during the late 1980s and 1990s and publicly reported in 2002, were the forerunners of today’s quality measures that staff use for quality improvement and that regulators use in the survey process.

National Quality Forum

In 1999, the Strategic Framework Board, a precursor to the National Quality Forum (NQF),² was created to design a strategy for national quality measurement and reporting, to articulate guiding principles and priorities for the system, and identify barriers and solutions for implementing quality measurement systems. The guiding principles were

1. (1) There should be a single level of quality available to all Americans;
2. (2) Quality should not be determined by where someone lives or receives care, or by type of insurance;

___________________

¹ The Minimum Data Set “is part of the federally mandated process for clinical assessment of all residents in Medicare and Medicaid certified nursing homes. This process provides a comprehensive assessment of each resident’s functional capabilities and helps nursing home staff identify health problems” (CMS, 2012).

² For more information, see www.qualityforum.org (accessed November 1, 2021).

1. (3) Quality measurement systems should include focused actions with single sets of priorities that are translated through the local health care delivery system; and
2. (4) National standards are needed to ensure consistency in measurement requirements imposed by different stakeholders across the health care system (McGlynn, 2003).

Finally, the board developed a framework for selecting national goals that were linked with the IOM’s aims for improving safety, effectiveness, patient-centered systems, timeliness, efficiency, and equitable service delivery in the health care system (IOM, 2001a).

Today, the NQF uses a consensus-based process to endorse quality measures. At the time that NQF was created, the science of quality measurement was still developing and, for the most part, measures were not widely available for many health care settings and clinical environments. As a result, most measures were developed and adopted through individual health care organizations, which resulted in nonstandard measures across organizations and competing measurement systems across settings. Furthermore, because of a lack of oversight, regulation, and enforcement, there was insufficient stakeholder engagement (e.g., among health care workers and patients) in quality measurement processes and reporting.

The NQF’s Measures Application Panel for Post-Acute Care/Long-Term Care Workgroup provides multistakeholder, pre-rulemaking input to CMS. In 2020, the panel identified measures of care coordination, interoperability, and patient-reported outcomes as being among the most important measures in health care settings (NQF, 2020).

CARE COMPARE³AND THE CMS FIVE-STAR RATING SYSTEM

In the late 1990s, federally mandated public reporting began for nursing homes on Nursing Home Compare, a web-based report card for certified nursing homes. The hope was that, as with other types of public reporting, consumers could use the information to help inform their choices and providers could use the information for quality improvement. Originally the website was not widely promoted, and it limited its information to regulatory deficiencies; a decade later, it expanded to include nurse staffing data (Konetzka et al., 2020). In 2002, through the Nursing Home Quality Initiative, the website added 10 clinical quality measures based on aggregated assessment data from the MDS (CMS, 2003). At this time, the

___________________

³ Formerly known as Nursing Home Compare, CMS changed the name to Care Compare in fall 2020 (CMS, 2021a).

website was more broadly promoted and allowed consumers to compare quality measures across nursing homes nationwide (CMS, 2003; Harris and Clauser, 2002). In 2008, Nursing Home Compare began publishing a five-star composite rating for each nursing home, ranging from one to five stars. The five-star rating is based on quality in three domains: inspections (based on deficiencies and effort needed to correct those deficiencies), staffing, and quality measures (Konetzka et al., 2020). Each of these domains is translated into a composite star rating, which the website also reports.

Calculation of the Star Rating

Calculating the overall star rating begins with assigning ratings in the inspections domain, based on the in-state distribution of inspection scores, divided into percentiles. Nursing homes can then gain one or two extra stars by performing well in the staffing or quality measures domains, or they can lose up to two stars by performing poorly in them. Thus, the inspections domain is weighted most heavily (CMS, 2021a,b). CMS regularly updates the rating system and website, not only by updating the reported scores on a quarterly basis (CMS, 2021a), but also by creating new measures, amending the list of included measures, and improving the interface. Although calculating the inspections score has changed little over the years, substantial changes have been made to the staffing and quality measures domains (CMS, 2020b; RTI International, 2012).

Staffing Domain

The staffing component of the rating system underwent a significant change when the underlying data source was changed. From Nursing Home Compare’s inception, staffing data were based on the Online Survey Certification and Reporting system (later replaced by the Certification and Survey Provider Enhanced Reporting system) data reflecting facility-reported staffing hours for the 2 weeks prior to the annual Medicaid recertification inspection. Thus, if nursing homes anticipated their survey dates and increased their staffing numbers prior to the survey, the data would not reflect actual staffing ratios throughout the year (Sharma et al., 2019). Furthermore, nuances of staffing, such as weekday versus weekend staffing, could not be examined. There were also concerns that facilities might inflate their numbers, given that the data were not audited. In 2016, CMS began requiring nursing facilities to submit ongoing data on staffing which are audited in the new Payroll-Based Journal system (CMS, 2018; Geng et al., 2019; OIG, 2021), and in April 2018, CMS began using these new staffing data in Care Compare. The reported measures

did not change, but the underlying data were presumed to be more valid. Concerns remain that average staffing ratios do not fully reflect all of the aspects of staffing that contribute to quality, such as expertise among staff (Snyder et al., 2019).

Quality Measures Domain

The quality measures domain has undergone changes in both data source and content. CMS has removed some measures, particularly those whose validity became more doubtful over time (e.g., “improvement in walking”) and added other newly established measures (e.g., the percent of residents who received influenza vaccinations, hospital readmission). Prior to the use of the five-star system, all quality measures were based on the MDS. Starting in 2018, CMS added several measures based on Medicare claims. These claims-based measures have the advantage that they are not based on facility-reported data and therefore may be more valid, as concerns have been raised about gaming of the MDS-based measures (Davila et al., 2021; Perraillon et al., 2019a,b). However, these newer measures have the disadvantage of being based only on data for Medicare fee-for-service beneficiaries, leaving out those in Medicare Advantage plans and the under-65 population that may not be on Medicare,⁴ which could raise concerns regarding the adequate measurement of quality for facilities that have disproportionate numbers of these types of residents. Table 3-2 shows the specific quality measures (both MDS-based measures and claims-based measures) for short-stay and long-stay residents used in the calculation of the five-star rating.

The weight or possible points assigned to each measure used in the five-star rating has also changed over time. Measures on which most facilities have improved (so that there is very little variation left to distinguish performance) receive fewer points than those with more potential for improvement. There have also been multiple changes in how the total quality measure scores are assigned to star ratings, as keeping constant thresholds would result in an increasing number of nursing homes receiving four or five stars in this domain. Finally, average star ratings vary by state; for example, average ratings range from 2.48 in Louisiana to 4.02 in Hawaii (CMS, 2021c).

___________________

⁴ The most current list of measures and the technical specifications for calculating them can be found on the CMS website: https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/CertificationandComplianc/Downloads/APPENDIX-New-Claims-based-Measures-Technical-Specifications-January-2020.pdf (accessed November 1, 2021).

TABLE 3-2 Quality Measures Used in the Five-Star Rating

SOURCE: CMS, 2021a.

NOTE: Additional quality measures are reported and available, but are not included in the calculation of the five-star rating.

Overall Star Rating

Star ratings increased over the first five years (2009–2013) of use of the five-star rating system (see Table 3-3), particularly in the quality measures and staffing domains (Abt Associates Inc., 2014). In 2019, CMS announced updates to the five-star rating system, including new thresholds for staffing and quality measurement domains, and the use of payroll-based data for staffing data (CMS, 2019). As shown in Table 3-3, many nursing homes’ subsequent ratings fell (Bose and Wilson, 2019; Reape, 2019). (See later in this chapter for more on self-reported data and the five-star system.)

TABLE 3-3 Nursing Homes’ Five-Star Ratings, 2009–2019

SOURCES: Abt Associates Inc., 2014; Reape, 2019.

Risk Adjustment and Validity of Care Compare Measures

One systematic review of the effectiveness of Care Compare included an assessment of evidence on the risk adjustment, correlation, and validity of the included quality metrics (Konetzka et al., 2020). The findings of this review are summarized below.

Risk Adjustment

One concern regarding the measures included in Care Compare is inadequate risk adjustment—that is, a failure to account for the fact that some nursing homes serve a sicker population than others, which makes it difficult to perform as well on measures of quality. Many providers believe their facilities’ ratings misrepresent the quality of the care they provide due inadequate risk adjustment (Kim et al., 2014; Perraillon et al., 2019a). Many measures in Care Compare do incorporate some risk adjustment. For example, among staffing measures expected staffing ratios adjust for resident severity, and among clinical quality measures the hospitalization measures are adjusted for resident age, sex, and comorbidities, among other factors. Despite substantial research and testing of measures of clinical quality, risk adjustment remains imperfect because of “inherent tradeoffs between the desire for validity and the desire to avoid complexity” (Konetzka et al., 2020, p. 304; see also Zimmerman et al., 1995). For example, Konetzka and colleagues (2020) noted specific findings from the following studies:

• Resident case-mix⁵ accounts for half of the variation in measures that have not been adjusted for risk (Li et al., 2010);
• False negative rates are high among unadjusted measures (Li et al., 2009); and
• The introduction of appropriate risk-adjustment approaches would significantly change overall facility rankings (Arling et al., 2007; Mukamel et al., 2008).

___________________

⁵ Case mix refers the diversity and complexity of care needs for a given population.

Konetzka and colleagues (2020) concluded that adding fairly simple risk variables, such as a regression adjustment for resident comorbidities, could greatly improve the validity of nursing home rankings.

Correlation of Measures

The validity of measures can be assessed by examining either the correlation among reported measures or the correlation of reported measures with broader, unreported measures. Konetzka and colleagues (2020) noted that a lack of correlation among reported measures could be due to the fact that an individual provider truly excels in one domain (e.g., staffing) but not in another (e.g., a specific clinical measure). However, they added that very low correlations may signal that the measures themselves have poor validity. Some studies have demonstrated low correlations among reported measures (Brauner et al., 2018; Saliba et al., 2018), while others show little correlation between reported measures and quality of life (QOL) (Kim et al., 2014) and resident or family satisfaction (Çalıkogğlu et al., 2012; Williams et al., 2016). Two broad measures of quality, the rate of admissions or readmissions to the hospital and mortality, have mixed evidence for their correlation with the individual measures reported in Care Compare (Fuller et al., 2019; Neuman et al., 2014; Saliba et al., 2018; Snyder et al., 2019; Unroe et al., 2012; Xu et al., 2019).

Validity of the Composite Five-Star Rating

Despite limited evidence on the validity of the overall five-star composite rating, Konetzka and colleagues (2020) concluded that the star ratings capture insightful information about the extremes regarding nursing home quality. First, the overall star rating appears to accurately reflect the nursing home characteristics commonly associated with low or high quality (e.g., nonprofit status, percentage of Medicaid residents, ownership status, socioeconomic characteristics of the residents) (Konetzka and Gray, 2017; Perraillon et al., 2019b; Unroe et al., 2012). Second, nursing homes with higher star ratings have lower rates of hospital admissions or readmissions and mortality (Cornell et al., 2019; Unroe et al., 2012).

Another important consideration is whether the ratings accurately reflect differences in levels of quality between facilities and how likely families are to use them to select a nursing home. Qualitative studies suggest that only ratings of the extremes (facilities with five stars versus facilities with one star) are predictive of rehospitalization for heart failure (Unroe et al., 2012), patient safety outcomes (Brauner et al., 2018), and resident and family satisfaction (Williams et al., 2016). Such studies have not found significant differences among nursing homes rated two, three, or four stars. However, Cornell and colleagues (2019) conducted a rigorous study and

found stronger negative relationships between star rating and outcomes such as mortality and long-term nursing home admission, but not for hospital readmissions. While Konetzka and colleagues (2020) concluded that the five-star composite rating appears valid at the extremes, they questioned “whether the star ratings are helpful to consumers choosing among nursing homes that are closer to average” (Konetzka et al., 2020). Other findings conclude that star ratings could be improved by adding consumers’ assessments of their quality of care experience (Mukamel et al., 2021). Although the five-star composite measure seems to have face validity and predictive validity at the extremes, the evidence is mixed as to whether the star ratings are helpful to consumers choosing among nursing homes that are closer to average. (See below for more on the usefulness of the five-star rating system for consumers.)

Summary

The validity of Care Compare appears to have improved overall with the additions of the five-star composite measure, claims-based quality measures, and the use of payroll-based staffing data. However, more work is needed in regards to the individual measures including better approaches to risk adjustment, improved correlation among measures that should be correlated, and refinement of the composite measure to better distinguish modest increments in quality.

Relationship between the Five-Star Rating and COVID-19

Limited, but mixed, evidence exists on the relationship between COVID-19 cases among residents in nursing homes and the home’s quality ratings. For example, several studies suggest that nursing homes with higher quality ratings are associated with lower rates of COVID-19 cases and deaths (Khairat et al., 2021; Ouslander and Grabowski, 2020; Williams et al., 2021). However, “this relationship, particularly with regard to case rates, can be partially attributed to external factors: lower-rated nursing homes are often located in areas with greater COVID-19 community spread and serve more socioeconomically vulnerable residents than higher-rated nursing homes” (Khairat et al., 2021, p. 2025).

In contrast, a literature review by Ochieng and colleagues (2021) of studies between April 2020 and January 2021 found that 8 of 12 studies did not find an association between quality ratings and COVID-19 cases or deaths (Abrams et al., 2020; Chatterjee et al., 2020; Chen et al., 2020; Figueroa et al., 2020; New York State Department of Health, 2021; Rowan et al., 2020; Sugg et al., 2021; White et al., 2020a). All four studies that

demonstrated associations between quality ratings and COVID-19 cases or deaths examined data from single states (Bui et al., 2020; He et al., 2020; Li et al., 2020a; Rau and Almendrala, 2020).

Evidence on Gaming of the Data

Konetzka and colleagues (2020) questioned whether the improved performance shown by providers on Care Compare measures truly reflects improvements in residents’ outcomes. Part of the challenge of the current rating system is that both staffing and quality measures are based on self-reported data from nursing homes, which could allow for “gaming” where nursing homes could falsify their data or use questionable strategies to improve scores.

Efforts to assess gaming in Care Compare “have used a common, if indirect, strategy . . . based on the assumption that ‘real’ improvement should be at least somewhat correlated either with the mechanisms to improve quality or with related (untargeted) measures of quality where spillovers from true quality improvement would be expected” (Konetzka et al., 2020, p. 303). Several studies have shown a lack of correlation between self-reported measures of improvement (e.g., outcomes, staffing ratios) and associated process or structural measures as well as a decrease in correlations after public reporting was initiated (Han et al., 2016; Werner et al., 2013; Zinn et al., 2008). Furthermore, several studies have shown decreases in these correlations along with changes in documentation and coding patterns after Care Compare began publicly reporting quality measures (Konetzka et al., 2015; Ryskina et al., 2018; Werner et al., 2011). Furthermore, a 2020 analysis by Integra Med Analytics found that the self-reported nursing home data are often underreported and uncorrelated to hospital-based measures (Integra Med Analytics, 2020)

Additionally, qualitative evidence also suggests that nursing home staff may be changing data or using other strategies to improve their scores without necessarily improving care (Davila et al., 2021; Perraillon et al., 2019a). Nursing home staff “reported substantial coding-related efforts to improve scores on the clinical quality measures, often led by a centralized director of clinical operations in the case of chain facilities” (Konetzka et al., 2020, p. 303). Examples included a staff member asking a resident about pain level⁶ only after the resident had received pain medications or counting inappropriate staff types among reports of staffing ratios.

___________________

⁶ The committee recognizes that pain is no longer a quality measure as of October 2019 (CMS, 2019).

Investigative journalism has also drawn attention to the issue of gaming of self-reported data (Silver-Greenberg and Gebeloff, 2021; Thomas, 2014, 2015). For example, a 2014 article in the New York Times found that two-thirds of nursing homes being monitored for quality held four or five stars in the staffing and quality measures domains (which are largely based on self-reported data), but more than 95 percent had only one or two stars in the inspection domain (based on the findings of state surveyors). (Thomas, 2014). Similarly, a study of the early years of the five-star rating system showed that 71 percent of nursing homes had four or five stars in the quality measures domain while only 34.1 percent of nursing homes had four or five stars in the inspections domain (Abt Associates Inc., 2014). A 2021 article in the New York Times noted that “in one sign of the problems with self-reported data, nursing homes that earn five stars for their quality of care are nearly as likely to flunk in-person inspections as to ace them” (Silver-Greenberg and Gebeloff, 2021). The article further noted that auditing of self-reported data is rare, even when nursing homes are cited for misreporting such data (Silver-Greenberg and Gebeloff, 2021). In 2015, the U.S. Government Accountability Office (GAO) recommended auditing of self-reported data to ensure quality and reliability (GAO, 2015).

Usefulness of Five-Star Rating and Care Compare for Consumers

As noted earlier, concerns have been raised about the ability of the five-star rating system to provide useful information to consumers (Edelman, 2015; Konetzka et al., 2020; LeadingAge, 2015). In 2007, Phillips and colleagues questioned the ability of performance measurement systems (in general) “to truly help consumers differentiate among homes providing different levels of quality” adding that “for consumers, performance measurement models are better at identifying problem facilities than potentially good homes” (Phillips et al., 2007). In 2016, the GAO issued a report highlighting challenges to the usefulness of the five-star rating system and Care Compare for consumers (GAO, 2016). First, they noted difficulty in understanding and comparing overall star rating, given the complexity of the calculation and potentially “masking the importance of the component ratings” (GAO, 2016, p. 16). GAO noted, as discussed previously, concerns about the ability to distinguish among nursing homes with two, three, or four stars. They stated, “for example, in one state, 28 percent of homes with a 3-star overall rating had a better health inspection score than the average health inspection score for homes with an overall 4-star rating” (GAO, 2016, p. 17). GAO also noted the inability of consumers to compare the quality of nursing homes across states (as ratings are relative only to other nursing homes in the same state). Finally, GAO noted the lack of information about consumer satisfaction, stating “many stakeholders told

us that they would like to see resident satisfaction included in the five-star system” (GAO, 2016, p. 23). They further concluded that “until consumer satisfaction information is included in the rating system, consumers will continue to make nursing home decisions without the benefit of this key performance measure and may not be choosing the home that would best meet their needs” (GAO, 2016, p. 24). (See later in this chapter for more on resident and family satisfaction and experience surveys.)

What is Missing from Care Compare?

Care Compare does not include several aspects of high-quality care that are important for consumers to consider when seeking nursing home care that best meets their needs. These areas include resident and family satisfaction and experience of care (discussed later in this chapter), palliative and end-of-life care, implementation of the resident’s care plan, psychosocial and behavioral health services, safety indicators (e.g., emergency preparedness and response, infection prevention and control), staff satisfaction, staff employment arrangements, health information technology adoption and interoperability, and the nursing home’s financial performance.

Palliative and End-of-Life Care Measures

Nursing homes often serve as the final home for many residents prior to death; currently, about 25 percent of Medicare beneficiaries die in a nursing home (Teno et al., 2018). Furthermore, over one-third of older Americans have a nursing home stay in the last 90 days of life (Teno et al., 2018). Thus, measuring the quality of palliative and end-of-life care for people receiving care in nursing homes is important. Measures of palliative and end-of-life care parallel, and in some cases overlap, those in other settings (e.g., hospital, home) and other patient populations (e.g., care of older adults). Areas of particular focus for palliative and end-of-life care include the assessment and management of common end-of-life symptoms, open and empathetic communication with patients and families, elicitation and documentation of patients’ preferences for life-sustaining treatments, mitigation of psychosocial and spiritual/existential distress, and grief and bereavement support (NQF, 2012a). Specific measures also include bereaved family evaluations of care at the end of life (NQF, 2012a) and concordance between patient preferences for care at the end of life and care received (Sanders et al., 2018).

Few palliative care measures have been developed specifically for use in the nursing home setting, and these tools have limited psychometric support and no single measure is widely used (Steel et al., 2003; Thompson et al., 2011). Some have proposed using existing administrative data such as MDS

pain measures and Medicare claims to measure rates of potentially burdensome treatments and transitions (e.g., end-of-life hospitalizations, intensive care unit admissions) and hospice use for long-term care residents (Gozalo et al., 2011; Mukamel et al., 2012, 2016; Temkin-Greener et al., 2016).

To date, the quality of end-of-life care in nursing homes is consistently measured and reported by the Veterans Health Administration’s community living centers. The key measure that the Veterans Health Administration uses is the Bereaved Family Survey, an NQF-endorsed measure administered to next of kin for all veterans dying in an inpatient Veterans Health Administration facility, including community living centers (NQF, 2012b; Thorpe et al., 2016). Family evaluations of overall end-of-life care for “veterans who died in Community Living Centers were better than those of veterans dying in acute or intensive care units but worse than those dying in hospice or palliative care units” (Ersek et al., 2015).

Other Measures

In the committee’s judgment, several other measures or areas could also be better represented on Care Compare. For example, the committee identified the implementation of residents’ plans of care and psychosocial and behavioral health services provided by each nursing home as important topics to consumers who are looking for specific services that they need and that are important to them (provided that the data can be validated). For example, consumers may be interested in information about costs and specialized services (Konetzka and Perraillon, 2016), and measures of patient safety can be difficult to elucidate (Brauner et al., 2018). Other safety indicators are likely important for consumers, especially following the pandemic, such as emergency preparedness, emergency response management, and infection prevention and control. Finally, consumers often value aspects of quality other than those reported in Care Compare (Mukamel et al., 2020), including staff satisfaction and staff employment arrangements (e.g., full-time, part-time, contract, and agency staff), both of which have been demonstrated to affect the quality of care (White et al., 2020b,c). Consumers also are likely interested in the availability of single-occupancy rooms (Grabowski, 2020; Silow-Carroll et al., 2021), health information technology adoption and interoperability (Vest et al., 2019), and financial performance (Weech-Maldonado et al., 2019), which can reflect the long-term stability and availability of the services they are seeking. Some of these measures can be readily reported from data already collected by CMS, but others will need to be further developed and tested before being publicly reported.

RESIDENT- AND FAMILY-REPORTED OUTCOMES: QUALITY OF LIFE, EXPERIENCE OF CARE, AND SATISFACTION

Measures of resident and family satisfaction represent an aspect of quality that is not currently reflected in the five-star rating (Williams et al., 2016), and consumers want to be able to see resident and family satisfaction on Care Compare (Konetzka and Perraillon, 2016; Schapira et al., 2016). As noted by Castle and colleagues, “measuring and reporting satisfaction with care may be important in helping seniors and their families choose a nursing home and also may be important in helping facilities improve some aspects of quality” (Castle et al., 2018).

The committee’s vision of quality, as emanating from the nursing home resident’s valuation of the benefits and harms of care, makes the resident’s voice central to achieving and measuring quality. This vision, rooted in principles of autonomy and self-determination, centers on an assessment of the quality of care for the nursing home resident by

• Establishing advance care plans (Fleuren et al., 2020);
• Ascertaining goals (Glazier et al., 2004);
• Identifying needs and symptoms (Saliba and Buchanan, 2012);
• Identifying preferences for residential care and activities (Housen et al., 2009; Roberts et al., 2018; Saliba and Schnelle, 2002);
• Measuring outcomes (Edelen and Saliba, 2010; NQF, 2013; Saliba et al., 2012); and
• Assigning relative values to quality measures (Weimer et al., 2019).

This section focuses on the assessment of quality by nursing home residents and their families, which is typically not included in national quality reports (Sangl et al., 2005). For a nursing home worker’s perspective on measuring quality of life, see Box 3-1.

Consumer assessment of quality can be of three types: (1) observation of the care, staff, and environment; (2) evaluation of the experience of care; and (3) satisfaction with the technical delivery and resulting outcomes of care. Observations can be guided by validated measures which can be scored to interpret quality and, in turn, help with choosing a facility (Rantz et al., 2006). The experience of care includes how the consumer perceives the way care is delivered, communication with providers, and characteristics of the care environment (e.g., cleanliness, odors, amenities). Technical measures of care processes and outcomes are only moderately correlated with resident and family reports of the experience of care (Kane et al., 2003; Li et al., 2016). Obtaining residents’ assessment of their care experience becomes even more important in the nursing home setting, where residents have high levels of support needs and rely on the nursing home staff and environment to meet their needs on a continuing basis for weeks, months, or even years (IOM, 2001a; Katz and Akpom, 1976; Saliba and Schnelle, 2002).

Design Considerations

Several important design principles need to be considered in efforts to systematically assess and report on the experience of residents in nursing homes. The science for developing reliable and valid consumer surveys related to health care is well established (Bolarinwa, 2015; Cleary and McNeil, 1988; CMS, 2020c). The process begins with extensive literature reviews and intensive interviews with patients, families, and other stakeholders to identify those aspects of care that are most important to measure. Survey methodology also delineates methods to quantify and minimize the bias that can be introduced by aspects such as the mode of interview, interviewer type, concerns about recrimination, and the ordering and wording of questions (Krosnick, 1999). Several of these considerations are particularly important for nursing home settings. Anonymity of survey responses is critical to obtaining unfiltered answers to questions, particularly if a person is in an ongoing, dependent relationship with the nursing home being evaluated. The wording of items and responses needs to be clear and written to match the cognitive steps the respondent employs to arrive at an answer. For example, a response option of yes/no may not be appropriate when the actual answer varies depending on conditions or frequency. Care needs to be taken to create items that are worded in a neutral manner and not in a way that encourages a desired response. The recall of multiple episodes can be challenging, particularly if a respondent is being asked to identify a single defining characteristic despite variation that occurs either day to day or over time.

Interacting with these design principles are the characteristics of the nursing home setting and the resident. Although residents with moderate

cognitive impairment can answer questions about their symptoms and preferences (Saliba and Buchanan, 2008), these questions need to be carefully worded and tested for clarity and message (Housen et al., 2008; Sangl et al., 2007). For those residents unable to self-report or to reliably report on their care experiences, proxy respondents might be used. While family evaluations of care are not interchangeable with residents’ evaluations, proxy reports are widely used in evaluating care for patients who cannot speak for themselves (e.g., individuals with disabilities, very young children and infants, individuals at the end of life) and where family members are involved in the care to the degree that they are considered as the recipients of care (i.e., the patient-family as a unit of care, such as that which occurs in hospice or for pediatric patients). Thus including families’ evaluations and experiences is warranted (Frentzel et al., 2012; Kane et al., 2005; Li et al., 2016).

There will be discord when the assessment is about events experienced by the resident but not the proxy. Additionally, because of the ongoing nature of care and number of encounters, the care experience may have day-to-day variation. Most importantly, as noted above, residents in the nursing home are reliant on the staff, who essentially control most elements of their environment and daily activities. This may interfere with resident evaluation of the care experience in two ways. First, residents and families may accommodate themselves to the shortcomings of the environment or care delivery, and, second, given the interaction of dependence and staff control, they may hesitate to criticize out of fear of recrimination.

Quality of Life

QOL is a multidimensional construct comprised of behavioral competence, objective environment, perceived QOL, and psychological well-being (Lawton, 1991). Although some models include objective and subjective components, generally the emphasis is on a person’s own assessment of his or her QOL, based on the person’s values and goals. While QOL is influenced by the quality of care (Kane et al., 2003), it encompasses broader concepts of psychological, social, spiritual, and existential well-being.

Under a contract from CMS, researchers developed and tested a survey to measure nursing home resident-reported QOL in order to determine the psychosocial domains that were absent or not sufficiently emphasized in the MDS (Kane et al., 2003, 2004). The resulting instrument contained 54 items covering resident satisfaction in 10 domains: autonomy, comfort, security, meaningful activity, relationships, functional competence, enjoyment, privacy, dignity, and spiritual well-being. A shortened 34-item version was also developed. The testing of the related items showed only modest correlation with a general satisfaction score, suggesting that the measures

of experience addressed different constructs from satisfaction. The survey developers acknowledged that the survey was not a full assessment of QOL, as it did not include measures of affect, functional status, or health. Evaluating QOL (and the impact of care on QOL) is particularly important for long-term care residents because the nursing home is their permanent home, the place they will spend the final weeks, months, or years of their lives. As such, QOL was folded into the resident and family experience measures described in the following sections.

CAHPS Measures

AHRQ sponsored the development of the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) program.⁷ This suite of surveys captures reliable and valid data to assess some elements of patients’ experience with various aspects of the health system and providers, and CAHPS survey data are used to provide publicly reported quality scores for Medicare and Medicaid. CMS mandates the collection of CAHPS surveys in several settings or populations (e.g., hospitals, Medicare Advantage, home health care, hospice care) by independent, credentialed survey vendors, and AHRQ supports the ongoing evaluation of item performance and association of ratings with patient characteristics (Klein et al., 2011; Martino et al., 2016).

Nursing Home CAHPS

CAHPS, with support from CMS, developed three “experience of care” surveys, available in English and Spanish, for nursing homes: the Long-Stay Resident Survey, the Discharged Resident Survey (for short-stay patients), and the Family Member Survey. The three surveys serve different purposes. “The [Long-Stay Resident Survey] provides the perspective from residents able to respond about their care; the [Discharged Resident Survey] presents a view from persons receiving rehabilitation. [The Family Member Survey] represents the experiences of those residents receiving the most days of care and who would otherwise be unable to voice their experiences” (Frentzel et al., 2012, p. S26). Furthermore, “measures based on consumer feedbacks offer unique insights into care quality and residents’ quality of life because, by construction, they may largely reflect the varied aspects of interpersonal care experiences of residents and their family members, which are likely not well captured by clinical measures” (Li et al., 2016, p. 10).

The Long-Stay Resident Survey is the only CAHPS survey administered in person, with the sample consisting of residents in the facility of more

___________________

⁷ For more information about CAHPS, see https://www.ahrq.gov/cahps/about-cahps/index.html (accessed November 5, 2021).

than 100 days (AHRQ, 2020; Sangl et al., 2007). Items ask about environment, care, communication and respect, autonomy, and activities. The Discharged Resident Survey is mailed by an independent vendor to nursing home residents who have been discharged after short stay (defined as less than 100 days) and differs from the long-stay version by including questions about therapy services. The Family Member Survey is also administered by mail and asks about the facility staff’s meeting of resident’s basic needs (e.g., eating, drinking, toileting), staff kindness and respect toward resident, nursing home provision of information and encouragement of family involvement, nursing home staffing, care of belongings, cleanliness, and an overall rating of quality (AHRQ, 2020; Frentzel et al., 2012). Nursing home CAHPS is the first CAHPS survey to include items for both quality of care and quality of life (Sangl et al., 2007).

While CMS mandates the collection of CAHPS surveys in other health care settings, the collection of the Nursing Home CAHPS survey is not required. This situation has been attributed to the resource requirements for in-person interviews in the long-stay version, concerns about the exclusion of persons with cognitive impairment, industry resistance to public reporting of detailed measures of resident experience, and concerns about whether family or proxy reports fairly capture the experience of residents (Frentzel et al., 2012; MedPAC, 2021). However, not implementing CAHPS in nursing homes when the surveys are carried out in other health care settings disadvantages nursing home residents and families in preventing their ability to provide feedback about their care experiences, and to use such information to make informed decisions when choosing a nursing home. Nursing homes are also disadvantaged by not having consumer reports of their experiences to improve services and care delivery.

Reliability and Validity of Nursing Home CAHPS

While many nursing home administrators report using resident satisfaction surveys, and satisfaction information is reported as being useful, the surveys being used vary widely and may not be adequately validated (Castle, 2007; Castle et al., 2004, 2018). Comparatively, Nursing Home CAHPS measures had extensive item development (described in the previous section on “design considerations”) and testing for reliability and validity (AHRQ, 2018, 2019; Castle et al., 2018; Frentzel et al., 2012; Sangl et al., 2007). Development of Nursing Home CAHPS resident surveys was based on a literature review, the use of focus groups with nursing home residents and their families about what topics and measures were important to them, cognitive testing to ensure residents could understand and answer the questions, field testing, and testing to develop composites and transition items (AHRQ, 2018). Similarly, the Family Member Survey was developed

based on a literature review, focus groups with residents and their families about what topics and measures were important to them, a public call for measures, cognitive testing with people who have family members in nursing homes, input from a technical expert panel regarding the use and value of a family member survey, field testing, and psychometric testing (AHRQ, 2019).

A 2018 analysis of the Discharged Resident Survey concluded that “the standardization and reliability that [it] provides could facilitate the same benefits we have seen in other industries for the CAHPS family of instruments (i.e., quality improvement, reimbursement, public reporting, and benchmarking)” (Castle et al., 2018, pp. 1241–1242). However, as most of this survey’s question domains are the same as the Long-Stay Resident Survey, and considering that most discharged residents are short stay, testing is needed to determined necessary adjustments for the Discharged Resident Survey (Baskin et al., 2012; Castle et al., 2018). A 2012 analysis of the Family Member Survey found that “the final family member survey, using formative research to develop the draft, cognitive testing to refine the items, psychometric analyses, and technical expert input, represents a well tested, valid, and reliable survey” (Frentzel et al., 2012, p. S26). Higher family ratings on nursing home care are associated with better care, resident outcomes, and several risk-adjusted quality measures (e.g., lower rates of pressure ulcers, hospital admission, and mortality) (Li et al., 2016). These reflect similar findings of psychometric testing for Family Member Surveys conducted in hospital-based CAHPS surveys (Li et al., 2016).

State-Based Surveys of Resident and Family Experience and Satisfaction

Some states have supported surveys of resident experience and satisfaction. In Michigan, nursing homes can receive incentive payments for submitting resident satisfaction surveys, though the exact design and content of those surveys is determined by the facility or chain (Michigan DHHS, 2021a). A few states have created their own surveys which combine satisfaction and experience items, and some have required nursing homes to collect these for over a decade. For example, the Ohio Long-Term Care Ombudsman Program began conducting nursing home resident and family surveys in 2002 (Ejaz et al., 2003; Straker et al., 2016). Posted surveys show variation in scores across facilities and across domains (Pyle, 2017). Similarly, evaluations of consumer experience surveys in Maryland have shown disparities in ratings of all care domains that persist over time (Li et al., 2014) as well as differences in experiences of care that are associated with proprietary status and nursing home chain size (You et al., 2016).

Since 2006, Minnesota has posted a report card that includes long-stay resident and family QOL and short-stay resident experience survey results (Minnesota DHHS, 2021). These surveys are administered annually and are also part of the Minnesota Performance-Based Incentive Program for nursing home quality improvement (Kane et al., 2007). The Long-Stay Resident Survey is conducted in person by an independent research firm; the other surveys are completed by mail (Kane et al., 2007; Minnesota DHHS, 2021).

Industry Measures of Resident and Family Experience and Satisfaction

In parallel with federal and state efforts, the nursing home industry has developed and implemented its own measures of resident and family satisfaction. For example, CoreQ, endorsed by the American Health Care Association, has three versions: long-term care residents, long-term care family, and short-stay discharged patients (Castle et al., 2020; CoreQ, 2019; Schwartz, 2021). Each version consists of three or four general questions that focus less on rating the quality of resident experience and more on summative satisfaction ratings. Another example of an industry-developed tool is NRC Health’s My Inner View Customer Satisfaction Survey (NRC Health, 2021). Many nursing homes promote and advertise high scores from self-designed and administered surveys of their residents. However, consumer advocates and survey methodologists have raised concerns that item wording and the choice of response formats may increase the tendency of respondents to provide socially appropriate response choices and thus provide only minimal variation in the scale (Bowling, 2005; Dillman et al., 2014; Nadash et al., 2019).

OVERVIEW OF QUALITY IMPROVEMENT

In his remarks to this committee, Donald Berwick, the president emeritus and a senior fellow at the Institute for Healthcare Improvement, said that “most improvement agendas [have] excessive reliance on control, metrics, accountability, and fixing damage, with little energy left over for learning and not much energy left over for invention.” Technical assistance is one of the primary mechanisms of quality improvement. The role of technical assistance depends in part upon the nursing home recognizing its need for additional knowledge and expertise. Without the motivation to improve quality, little change may occur. The following sections highlight key historical and current efforts to improve the quality of care in nursing homes. (See Appendix B for more details on some of the quality improvement programs discussed below.)

History of National Quality Improvement Efforts

The Joint Commission, founded in 1951, accredits and certifies health care organizations and programs in the United States (Joint Commission, 2021). In 1966, The Joint Commission initiated the Long-Term Care Accreditation Program that stimulated interest from nurses and other leaders in nursing homes to examine the quality of care delivered in their organizations (Rantz et al., 2002). In 1973, the American Nurses Association and the American Hospital Association sponsored a conference that challenged nurses to take responsibility for quality assurance activities within their health care organizations (Lang and Clinton, 1983).

In 1972, the federal government mandated professional standards review organizations (later called peer-review organizations) to oversee the quality of care delivered by health care organizations to Medicare beneficiaries, including those in nursing homes. In 2002, these programs were renamed as Quality Improvement Organizations (QIOs) and are currently overseen by CMS. The 2006 IOM report Medicare’s Quality Improvement Organization Program: Maximizing Potential (IOM, 2006) was prepared in response to a request from the U.S. Congress for an evaluation of the QIO program. The IOM committee recommended that CMS should assess the QIO program in several ways, including (1) the program as a whole, (2) individual QIOs, and (3) selected quality improvement interventions implemented by QIOs. Additionally, the committee recommended independent, external evaluations of the QIO program’s overall contributions and effectiveness.

As the QIO core contract concluded in 2006, CMS staff conducted a national evaluation of the improvements achieved by QIOs working with nursing homes, home health, and physician offices. While there were some measurable improvements, the evaluation’s conclusions were limited because of design flaws in the analysis (Rollow et al., 2006; Shortell and Peck, 2006). Congress continued to question the effectiveness of the QIOs, noting documented problems with the program (Freedland, 2009; Reichard, 2007). The QIO program’s 9th Scope of Work (2008–2011) targeted poorly performing nursing homes, but an independent evaluation highlighted the limited accuracy of the targeting methods used to select nursing homes to work with and the specific performance measures selected for improvement (Stevenson and Mor, 2009).

A 25-year review of external evaluations of the QIO program, including the internal monitoring and evaluation protocols, found that all such evaluations have found the impact of the QIO program to be small or difficult to interpret; additionally, the review found that inconsistencies in data collection and measurement create difficulties in monitoring and evaluations of effectiveness (Shaw-Taylor, 2014). This review also noted

that the QIO program cost at that time was about $200 million per year and that the ninth scope of work cost more than $1 billion. These costs and continued questions about the effectiveness of the program prompted the Office of the Inspector General to examine the duplicative efforts of QIOs with other CMS quality improvement efforts and the relative contribution of each effort (OIG, 2015). To date, the effectiveness and relative contributions of the QIO program have not been fully determined.

Standardized CMS quality measures have been used to guide quality improvement, in large part because of their prominent use in payment, policy, consumer choice, and accountability. The extent to which individual facilities engage in quality improvement and the effectiveness of such activities are unknown. Furthermore, many facilities lack adequate expertise and resources to engage in effective quality improvement. Various groups have established quality improvement coalitions and initiatives to support these efforts. Coalitions may involve the QIOs, academic partnerships, and state-level initiatives. The following sections give a brief overview of some of the larger initiatives as examples.

FEDERAL INITIATIVES FOR QUALITY IMPROVEMENT

The federal government has played a key role in improving the quality of care in nursing homes. Notable efforts include implementation of quality assurance and performance improvement (QAPI) programs and various CMS-supported campaigns and interventions.

Quality Assurance Performance Improvement Program

Federal efforts to embed the use of quality improvement methods include regulations of the Omnibus Budget Reconciliation Act of 1987⁸ that required facilities to have quarterly meetings to identify needed quality assurance activities and to develop and implement plans to correct quality deficiencies (Rantz et al., 2002). In 2016, the Patient Protection and Affordable Care Act⁹ (ACA) required that all skilled nursing facilities implement QAPI programs as a condition of reimbursement by Medicare and Medicaid. The QAPI activities expand the team members to include representatives of all staff in the nursing home as well as residents and families or significant others in continuously identifying opportunities for improvement (AHCA and NCAL, 2021).

___________________

⁸ Omnibus Budget Reconciliation Act of 1987, Public Law 100-203; 100th Cong., 1st sess. (December 22, 1987).

⁹ Patient Protection and Affordable Care Act, Public Law 111-148; 111th Cong., 2nd sess. (March 23, 2010).

QAPI seeks to achieve two critical aspects of quality management: quality assurance (the enforcement of standards for the quality of services and outcomes) (see Chapter 8) and performance improvement through the continuous quality improvement process (CMS, 2016). CMS oversees several quality improvement initiatives for nursing homes, but QAPI is meant to be ongoing and more comprehensive, so enrolling in another initiative does not necessarily mean a nursing home is meeting QAPI regulations. QAPI identifies five key elements of quality management:

1. (1) The design and scope of the program address all systems of care, including clinical care, QOL, choice, and safety;
2. (2) Leadership is engaged and actively seeks input from all stakeholders including staff, residents, and their families;
3. (3) Facilities have a data collection and monitoring system;
4. (4) Facilities have performance-improvement projects to assess and address particular problem areas; and
5. (5) Facilities employ systemic analyses and approaches to identify and address issues, develop policies, and look toward sustainable improvement (CMS, 2016).

In spite of these regulations and efforts, the COVID-19 pandemic revealed an absence of QAPI practices in many nursing homes. Moreover, when a QAPI is absent during annual inspections, it is often not cited as a deficiency (Bonner, 2021).

CMS-Supported Campaigns and Interventions

CMS has supported and continues to support several nursing home quality improvement programs (CMS, 2017a). Despite the large investment, some of these large-scale CMS-supported initiatives have not been evaluated rigorously. The next section gives an overview to some of these initiatives.

Interventions to Reduce Acute Care Transfers

The Interventions to Reduce Acute Care Transfers (INTERACT) program was created by CMS to reduce transfers to hospitals through improved identification, evaluation, management, and communication about acute changes in nursing home resident conditions (Ouslander et al., 2014). The identification of common, nonspecific changes in condition may “help guide decisions about further evaluation of changes in resident condition, when to communicate with primary care clinicians, when to consider transfer to the hospital, and provide suggestions on how to manage some

conditions in the facility without hospital transfer when it is safe and feasible” (Ouslander et al., 2014, p. 5). Studies of the impact of the INTERACT program have shown mixed results; however, measurable improvement has been demonstrated in avoidable hospitalizations, cost savings to Medicare, and care process improvements (Huckfeldt et al., 2018; Kane et al., 2017; Mochel et al., 2021; Ouslander et al., 2011, 2014; Vasilevskis et al., 2017).

The National Nursing Home Quality Improvement Campaign

The National Nursing Home Quality Improvement Campaign, which began in 2006 as the Advancing Excellence in America’s Nursing Homes Campaign, provides long-term care providers, consumers, advocates, and quality improvement professionals with freely accessible, evidence-based resources to support quality improvement activities (National Consumer Voice, 2021). CMS funding for the campaign ended in 2019, but a group of nongovernmental stakeholders continues to support the website (Great Plains Quality Innovation Network, 2019; NNHQI, 2021).¹⁰

The National Nursing Home Quality Care Collaborative

Modeled after the Institute for Healthcare Improvement breakthrough series approach (IHI, 2003) and led by Quality Innovation Network–QIOs, the goal of the collaborative is to rapidly spread best practices that are identified from high-performing nursing homes to other facilities. The program has had limited evaluation, but one case study did report success in learning to apply quality improvement techniques (Gillespie et al., 2016).

National Partnership to Improve Dementia Care in Nursing Homes

Launched in 2012, the National Partnership to Improve Dementia Care in Nursing Homes has the mission of improving the quality of care provided to individuals with dementia living in nursing homes. Membership varies across states but often involves the Quality Innovation Network–QIO, the survey agency, and other related state offices (e.g., Department of Aging); industry (e.g., Leading Age, American Health Care Association affiliates); professional associations; resident advocacy groups; academic institutions; nursing homes; and others. The initial target of the partnership was to reduce antipsychotic use and has now expanded to promote the use of nonpharmacologic, person-centered dementia care practices (CMS, 2021d). The partnership met its initial goal of reducing the prevalence of antipsychotic use in long-stay nursing home residents by 30 percent by the

___________________

¹⁰ See www.nhQualityCampaign.org (accessed November 1, 2021).

end of 2016 (CMS, 2017b). Overall, the rates of antipsychotic use among long-stay residents has decreased from 23.9 percent in 2011 to 15.7 percent in 2017 (CMS, 2017b).

Initiative to Reduce Avoidable Hospitalizations Among Nursing Facility Residents

CMS’ Medicare–Medicaid Coordination Office, in collaboration with the Center for Medicare and Medicaid Innovation, ran the Initiative to Reduce Avoidable Hospitalizations Among Nursing Facility Residents from 2012 to 2020. Partnering programs and organizations included

• The Optimizing Patient Transfers, Impacting Medical Quality, and Improving Symptoms: Transforming Institutional Care (OPTIMISTIC) project;
• The Reduce Avoidable Hospitalizations using Evidence-Based Interventions for Nursing Facilities (RAVEN) program;
• The Missouri Quality Initiative (MOQI);
• The New York–Reducing Avoidable Hospitalizations (NY–RAH) project;
• The Nevada Admissions and Transitions Optimization Program initiative;
• CHI/Alegent Creighton Health (Nebraska); and
• The Alabama Quality Assurance Foundation’s Nursing Facility Initiative.

Each of the seven participating sites was individually evaluated against key metrics and in aggregate (in comparison to a national comparison group) by an independent team that performed extensive quantitative and qualitative analyses. Phase One (2012–2016) involved the delivery of evidence-based practices to “improve the overall health and healthcare of participating long-stay nursing facility residents with the primary goals of reducing potentially avoidable hospitalizations, improving quality of care, and decreasing health care spending” (RTI International, 2017, p. 13). All partnering initiatives were “required to employ staff, such as registered nurses (RNs), or advanced practice registered nurses (APRNs) . . . to provide full- or part-time support” (RTI International, 2017, p. 13). Additionally, the nurses focused on improving care processes and communication among providers using INTERACT and assisted in training staff on how to use quality improvement methods to improve all key aspects of care delivery (e.g., nutrition, hydration, mobility, communication, end-of-life care planning). For the intervention period of 2014–2016, the evaluation team

found “persuasive evidence of the Initiative’s effectiveness in reducing hospital inpatient admissions, ED visits, and hospitalization-related Medicare expenditures” (RTI International, 2017, p. 223).

Phase Two of the initiative (2016–2020) addressed six key conditions: pneumonia, congestive heart failure, chronic obstructive pulmonary disease/asthma, skin infection, dehydration, and urinary tract infection. During this phase, CMS offered payment incentives for each of these conditions in an effort to reduce hospitalizations (RTI International, 2021, p. ES 3). Phase Two included two different intervention groups: one in which the payment intervention was added to the existing clinical supports (i.e., RN and APRN support) and the other a “payment only” intervention group, which did not have the RN and APRN supports. Overall, providing payment incentives to nursing facilities to reduce use and expenditures was not effective (RTI International, 2017).

Four of the participating sites (MOQI, NY–RAH, OPTIMISTIC, and RAVEN) had a strong clinical focus. Details of these programs, including their key components and outcomes, are provided in Appendix B.

STATE AND LOCAL INITIATIVES FOR QUALITY IMPROVEMENT

In addition to federal efforts, different approaches to quality improvement in nursing homes have been implemented at the state and local levels. These efforts include academic–provider partnerships and state-supported programs.

Academic–Provider Partnerships

Many targeted quality improvement efforts in nursing homes focus on such topics as pressure ulcers, falls, antibiotic use, pain, palliative care, and hospital transfers (Rantz et al., 2009, 2012a, 2012b; Sloane et al., 2020). These projects are often initiated by an academic research team and involve a single facility or smaller groups of facilities from a single geographic region. Funding for these projects comes from a variety of sources, including federal research grants, foundation grants, and government contracts, including those supported with civil monetary penalty funds. In general, the results of these studies have been mixed; however, the use of quality improvement methods to improve quality of care is well supported with significant results or trends in improvement. In addition to discrete quality improvement projects, academic researchers also have partnered with nursing homes and other stakeholders to form quality improvement programs. Two examples of this type of partnership are the Arkansas Coalition

for Nursing Home Excellence and the Quality Improvement Program for Missouri (QIPMO) (Beck et al., 2014; Rantz et al., 2003, 2009). Another example of academic–provider partnerships is the model of a teaching nursing home wherein students, faculty, and health care workers collaborate to improve care for residents (Mezey et al., 2008).

The Arkansas Coalition for Nursing Home Excellence

The Arkansas program was a broad coalition of nursing home organizations, state survey representatives, academics specializing in geriatrics, the Arkansas ombudsman, and advocates, all focusing on efforts to improve quality and support culture change in the state’s nursing homes. The program, which coincided in some years with the National Consumer’s Voice Advancing Excellence Campaign (2006–2016), was active for 10 years (2004–2014) and reported success with improvement in pressure ulcers, physical restraints, chronic pain, acute pain, and complaints from 2004–2011 (Beck et al., 2014; National Consumer Voice, 2021).

The Quality Improvement Program for Missouri

Developed and tested in the 1990s, and adopted in 1999 (and still operating today), QIPMO is a cooperative project between the Missouri Department of Health and Senior Services and the University of Missouri Sinclair School of Nursing whose goal is to improve quality of care in Missouri nursing homes (Popejoy et al., 2000; Rantz et al., 2001, 2003). In the program, quality improvement nurses (nurses with graduate education in geriatric nursing) and leadership coaches (nursing home administrators) contact and offer free, confidential clinical and operational consultation to long-term care facilities. Every facility in the state is contacted at least annually with offers for site visits to help with quality improvement. In 2019, QIPMO nurses contacted 696 different facilities, organizations, and stakeholders and conducted 561 onsite visits in 342 different facilities to assist with the improvement of clinical systems and care. QIPMO leadership coaches contacted 605 different facilities, organizations, and stakeholders and conducted 258 onsite visits in 172 different facilities to assist with leadership quality improvement and education (Sinclair School of Nursing, 2021a,b). There were 527 active providers in Missouri in fiscal year 2019, which means the five QIPMO gerontological nurses and three leadership coaches reached, onsite, 71 and 63 percent, respectively, of active providers.¹¹

___________________

¹¹ Quality Improvement Program for Missouri (QIPMO) testimony presented to the National Academies of Sciences, Engineering, and Medicine Committee on the Quality of Care in Nursing Homes; Nicky Martin, M.P.A., B.S., LNHA, CDP, IP, LTC leadership coach, program team leader; and Wendy Boren, B.S.N., R.N., IP, clinical educator/consultant; May 10, 2021.

QIPMO has been found to improve the quality of care outcomes of nursing home residents and to reduce the cost of care, including annual care cost savings of more than $4.7 million statewide, several times more than the program costs (Rantz et al., 2003, 2009; University of Missouri Interdisciplinary Center on Aging, 2008). A 2018 evaluation of the administrator coaching service for survey readiness assistance found that use of the service resulted in a reduction of severe citations from 52.3 percent pre-training to 39 percent post-training and a reduction of administrator turnover from 22.1 percent pre-training to 19.5 percent post-training (Phillips et al., 2018).

Academic–provider partnerships such as QIPMO offer several advantages. With the program being managed within the state, it is easier to build support for the use of the clinical and administrative services. Communication among the nursing home associations, the nursing homes (and other long-term care settings), and state agency staff can be facilitated more readily so that trusting relationships can be built. Working onsite with the QIPMO nurse or leadership coach, facility staff can improve care systems in ways that are not possible with their individual efforts alone. Affiliation with a well-recognized and respected state university adds credibility to the educational offerings and increases consultation acceptance (Popejoy et al., 2020; Rantz et al., 2009).

The statewide infrastructure and rapid communication network of QIPMO was critical during the COVID pandemic (Pool and Boren, 2020; Popejoy et al., 2020). QIPMO served as a communication network to get consistent, helpful, accurate information to all long-term care settings in the state, even helping with the distribution of personal protective equipment. Virtual support groups were held at least weekly for administrative and clinical staff to ask questions and seek clarification about rapidly changing guidance on best practices. With daily (or as often as needed) electronic communication to all long-term care facilities in the state it was possible to quickly distribute “condensed, practical” implementation guidelines and revisions of changing national guidelines (Pool and Boren, 2020; Popejoy et al., 2020).

While QIPMO has shown success, this specific model has not been adopted or tested in other states, and its success may be dependent on several factors that may not exist elsewhere, such as state support and the involvement of a strong nursing and health sciences school with substantial expertise and commitment to nursing homes.

Teaching Nursing Homes

In the 1980s the National Institute on Aging and the Robert Wood Johnson Foundation funded teaching nursing home models that linked nursing homes, nursing programs, and academic medicine (Mezey et al., 2008). The

teaching nursing home was conceptualized as a way to increase research, improve resident health outcomes, expand staff training, and improve knowledge about geriatric care. The model was considered to be successful and has been shown to improve attitudes about nursing homes and working with older adults, decrease staff turnover rates, improve outcomes, and lower costs (Mezey and Lynaugh, 1989; Mezey et al., 1988; Shaughnessy et al., 1995).

In March 2005, a summit of experts in geriatrics endorsed the teaching nursing home model (with a reciprocal nature that encouraged a “culture of learning” for both nursing homes and the academic programs) over a more typical model of a nursing home collaboration with academia in which a student is simply placed into a nursing home for a clinical rotation (Mezey et al., 2008). The summit participants saw value in teaching nursing homes for interdisciplinary team training, faculty development, and enhancing the educating and credentialing of nursing home staff.

In 2021, the Jewish Healthcare Foundation, The John A. Hartford Foundation, and the Henry L. Hillman Foundation provided funding for the Pennsylvania Teaching Nursing Home project to test an updated version of the teaching nursing home in three different partnerships (Jewish Healthcare Foundation, 2021). In its announcement of the pilot project, the Jewish Healthcare Foundation stated:

State-Supported Quality Improvement Programs

States may also take the initiative to develop or test quality improvement programs within their own borders.

Minnesota Performance-Based Incentive Payment Program

The Minnesota Performance-Based Incentive Payment Program (PIPP) uses an alternative approach to pay-for-performance to fund nursing home–initiated quality improvement projects (Arling et al., 2013, 2014). (See later in this chapter as well as Chapter 7 for more on market-based incentives

to improve the quality of care in nursing homes.) Under a competitive process, nursing homes submit proposals that an expert panel evaluates. The program funds a select group of proposals, all of which must include clear and measurable performance targets. In addition to funding, the state offers technical assistance, particularly during the proposal development stage. Projects that do not meet their performance targets can lose up to 20 percent of the project funding.

From 2007 to 2010, PIPP supported 66 projects at 174 of the state’s 373 nursing facilities (Arling et al., 2013, 2014). The projects had a broad range of goals, with many focused on clinical quality and technology and many targeting discrete areas such as culture change, art therapy, and resident psychological well-being. Only 3 of the 66 projects lost funding because they failed to meet performance improvement targets. Moreover, participating facilities demonstrated significantly greater gains than facilities not participating in PIPP according to a multidimensional composite measure of quality as well as in targeted areas during years 2008–2010 as compared with the baseline of 2006–2007. Finally, these facilities also maintained their quality advantage during years 2011–2013 after their quality improvement projects were completed. PIPP in Minnesota has not been adopted or tested in other states and may be dependent on factors that may not exist elsewhere. However, PIPP has been shown to be an effective example of a state-based quality improvement initiative (Arling et al., 2013, 2014).

The Systems Change Tracking Tool^SM

The Systems Change Tracking Tool (SCTT) was developed by Altarum to “capture the adoption of culture change practices over time” in nursing homes (Perry et al., 2021). The tool was based on the idea that culture change to prioritize person-centered care is fundamental to improving the quality of care. The tool seeks to track and measure improvements in person-centered care in order to determine the impact of culture change practices on quality of care.

SCTT was funded as a grant from the Michigan Department of Health and Human Services in 2019 using funds from civil monetary penalties; the project is expected to conclude in early 2022 (Perry et al., 2021). The project was implemented in six nursing homes in Michigan (with varying characteristics, including star rating) and provided culture change training and curriculum from The Eden Alternative¹² to nursing home staff as part of a quality improvement initiative. The participating nursing homes admitted

___________________

¹² For more information, see https://www.edenalt.org (accessed November 16, 2021).

that the COVID-19 pandemic affected their ability to fully implement culture change practices. However, “throughout 2020 and 2021, homes made steady progress towards re-establishing more person-centered care practices and unifying them with infection control protocols to better serve their residents” (Perry et al., 2021). Altarum plans to pilot a shortened version of the tool for nursing home residents in eight nursing homes in Tennessee. While these projects have not been fully evaluated, they may provide insight as to how focusing on person-centered care can lead to demonstrable changes in quality.

OTHER APPROACHES TO QUALITY IMPROVEMENT

In addition to specific efforts at the federal, state, and local levels, other approaches have been used to improve the quality of care in nursing homes. Specific examples include the use of market-based incentives (e.g., value-based purchasing) and the development of age-friendly health systems.

Market-Based Incentives to Improve Quality in Nursing Homes

Market-based incentives to improve care appear to have had positive impacts on quality measures. However, the gains have been largely confined to a narrow range of targeted measures and do not necessarily reflect overall better quality of care (Arling et al., 2020; Werner and Konetzka, 2010). Also, there are likely some unintended consequences affecting nursing homes that serve higher numbers of minority residents. For example, Hefele and colleagues (2019) found that “hospitals serving racial/ethnic minority groups and low-income people perform worse” under the value-based purchasing programs they studied (Hefele et al., 2019, p. 1130). Furthermore, they raised concerns about using incentive-based programs in nursing homes, particularly because “vulnerable populations may be disproportionately affected by penalties” (Hefele et al., 2019, p. 1130). Value-based purchasing, as a mechanism to improve quality of care, is discussed more fully in Chapter 7.

Age-Friendly Health Systems

The Age-Friendly Health System initiative represents a collaboration of the John A. Hartford Foundation, the Institute for Healthcare Improvement, the American Hospital Association, and the Catholic Health Association of the United States. The initiative uses a 4 M’s framework—know what matters to the individual; prevent, identify, treat, and manage mentation (e.g., dementia, depression, delirium); encourage mobility; and use medications that do not interfere with preferences, mobility, or mentation)

(Fulmer, 2018; IHI, 2021). That is, the framework is designed to ensure that care follows evidence-based practices, does not cause harm, and aligns with older adults’ (and their family caregivers’) preferences.

The person-centeredness of the Age-Friendly Health System initiative aligns with the committee’s vision of high-quality care (see Chapter 1). Furthermore, Edelman and colleagues (2021) suggested how the 4 M’s framework could be applied to the nursing home setting in order to improve the quality of care. Such an approach could include the integration of what matters to the resident into QAPI monitoring and the care plan, the provision of activities and treatments that improve cognition and mobility, and medication management (Edelman et al., 2021). (See Chapter 4 for more on person-centered care.)

TECHNICAL ASSISTANCE FOR QUALITY IMPROVEMENT

Technical assistance can have different meanings but generally refers to the process by which an entity works with providers to build capacity, implement innovations, and enhance competence in order to improve outcomes (IOM, 2006; Wandersman et al., 2012). Technical assistance to implement advances in science originated in the land-grant universities established under the Morrill Act of 1862 to meet the growing need for people with expertise in science and agriculture. The Morrill Act of 1890 provided for regular appropriation to these institutions so they could continue to function in that capacity (Encyclopaedia Britannica, 2017). Technical assistance can help a provider to detect areas in need of improvement, identify root causes of problems, implement interventions and systems changes, teach process improvement methods, promote best practices, facilitate knowledge transfer, analyze performance data, and coordinate quality improvement efforts (IOM, 2006). In its 2006 evaluation of the QIO program, IOM concluded that the public sector needs to play a substantial role in improving care quality for all Americans, especially those who depend on Medicare and Medicaid, and “some level of technical assistance should be available through the federal government as a public good” (IOM, 2006, p. 63).

Technical assistance leading to successful implementation of quality improvement initiatives partly depends on an organization’s willingness and readiness to change and capacity to implement change (Holt et al., 2010; Le et al., 2014; Weiner, 2009; Weiner et al., 2008). For example, one study of instituting quality improvement methods in nursing homes identified “readiness indicators” among those that were most likely to improve, including

• A leadership team (e.g., nursing home administrator, director of nursing) interested in learning about how to use quality reports to improve care,

• A change champion within the nursing home,
• Willingness to involve all staff in educational activities,
• Plans for continuous education of new staff, and
• Continuous involvement of all staff to encourage “ownership” of the process and responsibility for change (Rantz et al., 2012b).

However, Wandersman and colleagues (2012) discussed the importance of proactive technical assistance, which they describe as “a strategic approach to bringing specific knowledge and skills to recipients, and then helping recipients to adopt and use the information and skills effectively” (Wandersman et al., 2012, p. 451). They add that proactive technical assistance can be both anticipatory and responsive:

The Value of Technical Assistance in Nursing Homes

The examples of national, state, and local approaches to quality improvement in nursing homes discussed earlier in this chapter all have their foundation in assisting workers to increase their knowledge, skills, and capacity to deliver up-to-date, necessary care. The examples do have positive outcome evaluations of their services to improve the quality of nursing home care. Furthermore, a 2010 analysis of the Special Focus Facility (SFF) program noted that “according to some states, the SFF Program is more effective when combined with state-based quality improvement activities,” citing QIPMO as a noteworthy program (GAO, 2010). However, many of the specific state and local programs have not been replicated in other states. Consistent, regular funding has been shown to be necessary for them to be sustainable, effectively adopted, and consistently used.

Some features of effective technical assistance from the evidence of state and local programs include building a trusting relationship between the nursing home staff and people offering the technical assistance, modifying the assistance to best fit current needs and skills of each nursing home (it

is not “one size fits all”), and making sure the scientific content (for example, specific care of complex residents with health, physical, mental, and social-behavioral needs) is the most up to date and accurate. State and local programs may be particularly well suited to provide technical assistance due to familiarity with the local community and the ability to be seen as a trusted peer. Such programs may also help integrate nursing homes into their local communities and the broader health care system. Additionally, technical assistance staff must assess each staff member’s readiness to learn and successfully engage in quality improvement activities (Le et al., 2014; Rantz et al., 2012b; Wandersman et al., 2012). When some nursing homes are not ready to operationalize quality improvement initiatives but others are, technical assistance programs need to consider readiness in prioritizing their efforts.

QUALITY IMPROVEMENT AND DISPARITIES

Braverman and colleagues (2017) noted that “for the purposes of measurement, health equity means reducing and ultimately eliminating disparities in health and its determinants that adversely affect excluded or marginalized groups.”

The existence of racial and socioeconomic disparities in nursing homes is well known. In 2004, it was reported that

Today, this disparity still exists, as African American and other minority residents more often live in poorer-quality nursing homes with higher Medicaid populations than White nursing home residents (Gorges and Konetzka, 2021; Sharma et al., 2020). Early in the COVID-19 pandemic, nursing homes with greater percentages of African American and other minority residents were more likely to have COVID-19 cases (Abrams et al., 2020) and had two to four times the proportion of cases and deaths from COVID-19 than with higher proportions of White residents (Gorges and Konetzka, 2021; Li et al., 2020b). Nursing homes serving underserved populations (such as those with higher proportions of Hispanic or Latinx residents, Black residents, and people who are funded by Medicaid) are more likely to have been penalized under value-based purchasing (Hefele et al., 2019). Additionally, nursing homes with more residents with serious mental

illness are more likely to have lower star ratings, lower direct-care staffing, and for-profit ownership than all other nursing homes (Jester et al., 2020). Clearly, with these sustained disparities there is a critical need to ensure that residents of diverse racial and ethnic backgrounds, as well as those with serious mental illness, can access and receive high-quality nursing home care when they need it.

Additionally, it is important that quality improvement initiatives carefully and intentionally include measurement and reporting of demographic variables as well as the structural factors that drive inequities. Data on sociodemographic characteristics need to be collected consistently so that quality improvement outcome measures can be evaluated on any differences across these characteristics. This would then enable the determination of when the degree of difference warrants action for targeted interventions for disparities. These interventions, “tailored to overcome barriers and meet the needs of populations” (Mutha et al., 2012), can ultimately address inequities and disparities in access to high-quality nursing home care (Green, 2017; Hirschhorn et al., 2021; Weinick and Hasnain-Wynia, 2011).

COORDINATED EFFORTS DURING THE COVID-19 PANDEMIC

While not framed as quality improvement efforts, several initiatives during the COVID-19 pandemic were intended to assist nursing homes with infection control and prevention issues. The state and federal resources in these efforts included strike teams that provided expertise, personal protective equipment, testing, vaccinations, and other resources important to nursing homes.

Rapid Response Network

The Rapid Response Network, supported and promoted by the Institute for Healthcare Improvement, The John A. Hartford Foundation, and Age-Friendly Health Systems, held 20-minute web-based “huddles” twice a week for 11 weeks (IHI, 2020). The topics presented included a range of pragmatic clinical and administrative issues such as screening and testing for COVID, infection control, advanced care planning during COVID, tending to the emotional well-being of residents and staff during the pandemic, and addressing pandemic-related staffing and workforce shortages. The series ran from August through October 2020 and has not yet been rigorously evaluated for its impact on the quality of care. Descriptive and participant reports summarized by the program sponsors concluded that the huddles were helpful to nursing home participants (Brandes et al., 2021).

Project ECHO^®

Project ECHO (Extension for Community Healthcare Outcomes) uses teams of experts to mentor local clinicians virtually to help reduce health disparities affecting underserved areas. In response to the COVID-19 pandemic, Project ECHO was used to disseminate information quickly to nursing homes with just-in-time learning, short presentations, exemplars, and group discussion (Lingum et al., 2021). AHRQ and the Institute for Healthcare Improvement partnered with Project ECHO to create the AHRQ–ECHO National Nursing Home COVID-19 Action Network, which offers quality improvement training programs for CMS-certified nursing homes aimed at stopping the spread of COVID-19 (AHRQ, 2021). The effort was supported through a $237 million contract made to AHRQ under the Coronavirus Aid, Relief, and Economic Security (CARES) Act.¹³ Key areas targeted by the initiative include keeping COVID-19 out of unaffected nursing homes, early identification of infections among residents and staff, prevention of spread, caring for residents with mild cases, sharing information on how to protect residents and staff, and reducing social isolation (Project ECHO, 2021).

Although Project ECHO has not been evaluated for its impact on facility and resident outcomes, the participation of nursing homes in the program during the COVID pandemic has been reported to be high. A systematic review of prior Project ECHO program evaluations concluded that the method is effective and potentially cost-saving, although this review did not include engaging nursing homes in ECHO (Zhou et al., 2016).

THE FUTURE OF QUALITY IMPROVEMENT IN NURSING HOMES

The science of quality improvement has been enhanced in part by recent trends in clinical trials, most notably the recognition that strictly controlled efficacy trials of complex interventions (such as quality improvement) require efforts to ensure that interventions can be tailored to the local site and that barriers to dissemination and implementation can be addressed. Commonly used methods in implementation science, such as the use of clinical champions, stakeholder engagement, feedback reports, action planning, and coaching, also are important quality improvement strategies. Ultimately, quality improvement has to involve all of the interdisciplinary team in nursing homes—leadership, direct-care workers, nurses, social workers, and all staff delivering service to residents. Quality improvement

___________________

¹³ Coronavirus Aid, Relief, and Economic Security (CARES) Act, Public Law 116-136; 116th Cong., 2nd Sess., (March 27, 2020).

cannot be accomplished by making one or two staff members accountable for the process; it truly needs a team effort and a persistent, long-term commitment to examining all aspects of the nursing home operation—direct care, business practices, facility maintenance, infection control, and team and management practices.

KEY FINDINGS AND CONCLUSIONS

Quality Measurement

• Inspections receive the most weight in the calculation of the five-star rating.
• The staffing component of the five-star rating may not fully reflect the adequacy of staffing in nursing homes.
• Although the five-star composite measure appears to distinguish nursing homes at the extremes (five stars versus one star), the rating offers little to distinguish among nursing homes rated at two, three, or four stars.
• More work is needed in regards to the individual measures within Care Compare, including better approaches to risk adjustment and improved correlation among measures that should be correlated.
• Gaming of self-reported data needs to be minimized by strategies such as auditing and evaluating the data sources used in public reporting of the quality measures.
• Several key domains of high-quality care are not measured directly in Care Compare, including resident and family satisfaction and experience, effectiveness of behavior and mental health services, and the quality of palliative and end-of-life care.
• Obtaining residents’ assessment of their care experience becomes even more important in the nursing home setting, where residents have high levels of support needs and rely on the nursing home staff and environment to meet their needs on a continuing basis for weeks, months, or even years.
• Not implementing surveys of resident and family satisfaction and experience in nursing homes disadvantages nursing home residents and families from providing feedback about their care experiences, and making informed decisions when choosing a nursing home. Nursing homes are also disadvantaged by not having consumer reports of their experiences to improve services and care delivery.
• While many nursing home administrators report using resident satisfaction surveys, and satisfaction information is reported as being useful, the surveys being used vary widely and may not be adequately validated.
• The nursing home CAHPS survey had extensive item development and testing for reliability and validity.

Quality Improvement

• Evidence about the effectiveness and relative contribution of QIOs to quality improvement in health care, particularly in nursing homes, is lacking.
• Technical assistance leading to successful implementation of quality improvement initiatives depends, in part, on an organization’s willingness and readiness to change as well as having the capacity to implement change.
• State programs that focus on helping nursing home staff with quality improvement activities within nursing homes using onsite assistance by expert clinical staff and collaborating groups are effective in improving quality of care, and their help is widely accepted by nursing homes.
• Features of effective technical assistance from state and local programs include building a trusting relationship between the nursing home staff and people offering the technical assistance, modifying the assistance to best fit current needs and skills of each nursing home, and making sure the scientific content is the most up to date and accurate.
• Dedicated funding streams are needed to sustain initiatives shown to improve quality in demonstration projects.
• The COVID-19 pandemic revealed an absence of QAPI practices in many nursing homes.
• Quality improvement cannot be accomplished by making one or two staff members accountable for the process; effective quality improvement initiatives need team effort and a persistent, long-term commitment to examining all aspects of the nursing home operation.

Quality and Disparities

• Nursing homes in low-income neighborhoods, with high numbers of African American and other minority residents, and nursing homes primarily serve Medicaid residents have lower quality of care ratings and lower direct-care staffing.
• Early in the pandemic, nursing homes with greater percentages of racial and ethnic minority residents experienced higher probably of COVID-19 cases and two to four times the proportion of cases and deaths from COVID-19.
• Nursing homes with more residents with serious mental illness are more likely to have lower star ratings, lower direct-care staffing, and for-profit ownership than all other nursing homes.
• There is limited, but mixed, evidence on the relationship between COVID-19 cases among residents in nursing homes and the home’s quality ratings.

• Quality improvement initiatives need to carefully and intentionally include measurement and reporting of the structural factors that drive inequities as well as demographic variables.

REFERENCES

Baskin, R. M., J. Sangl, and M. W. Zodet. 2012. Effect of different imputation methods on factor analyses of CAHPS nursing home survey. Rockville, MD: Agency for Healthcare, Research and Quality.

Beck, C., K. J. Gately, S. Lubin, P. Moody, and C. Beverly. 2014. Building a state coalition for nursing home excellence. The Gerontologist 54(Suppl 1):S87–S97.

Bolarinwa, O. A. 2015. Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Nigerian Postgraduate Medical Journal 22(4):195–201.

Bonner, A. 2021. Hope springs eternal: Can Project ECHO transform nursing homes? Journal of the American Medical Directors Association 22(2):225–227.

Bose, J., and S. Wilson. 2019. Nursing homes’ star ratings significantly impacted by new CMS updates. https://www.claconnect.com/resources/articles/2019/nursing-homes-star-ratings-significantly-impacted-by-new-cms-updates (accessed February 1, 2022).

Bowling, A. 2005. Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health 27(3):281–291.

Brandes, R., E. Miranda, A. Bonner, J. Baehrend, T. Fulmer, and J. Lenoci-Edwards. 2021. Leveraging national nursing home huddles for rapid COVID-19 response. Geriatrics 6(2):62.

Brauner, D., R. M. Werner, T. P. Shippee, J. Cursio, H. Sharma, and R. T. Konetzka. 2018. Does Nursing Home Compare reflect patient safety in nursing homes? Health Affairs (Millwood) 37(11):1770–1778.

Braverman, P., E. Arkin, T. Orleans, D. Proctor, and A. Plough. 2017. What is health equity? And what difference does a definition make? Princeton, NJ: Robert Wood Johnson Foundation.

Bui, D. P., I. See, E. M. Hesse, K. Varela, R. R. Harvey, E. M. August, A. Winquist, S. Mullins, S. McBee, E. Thomasson, and A. Atkins. 2020. Association between CMS quality ratings and COVID-19 outbreaks in nursing homes - West Virginia, March 17-June 11, 2020. Morbidity and Mortality Weekly Report 69(37):1300–1304.

Burstin, H., S. Leatherman, and D. Goldmann. 2016. The evolution of healthcare quality measurement in the United States. Journal of Internal Medicine 279(2):154–159.

Çalıkoğlu, Ş., C. S. Christmyer, and B. U. Kozlowski. 2012. My eyes, your eyes—the relationship between CMS five-star rating of nursing homes and family rating of experience of care in Maryland. Journal for Healthcare Quality 34(6):5–12.

Castle, N., J. Engberg, and A. Men. 2018. Satisfaction of discharged nursing home residents. Journal of Applied Gerontology 37(10):1225–1243.

Castle, N. G. 2007. A review of satisfaction instruments used in long-term care settings. Journal of Aging & Social Policy 19:9–41.

Castle, N. G., T. J. Lowe, J. Lucas, J. Robinson, and S. Crystal. 2004. Use of satisfaction surveys in New Jersey nursing homes and assisted living facilities. Journal of Applied Gerontology 34:156–171.

Castle, N. G., D. Gifford, and L. B. Schwartz. 2020. The CoreQ: Development and testing of a nursing facility resident satisfaction survey. Journal of Applied Gerontology 40(6):629–637. https://doi.org/10.1177/0733464820940871.

Chatterjee, P., S. Kelly, M. Qi, and R. M. Werner. 2020. Characteristics and quality of U.S. nursing homes reporting cases of coronavirus disease 2019 (COVID-19). JAMA Network Open 3(7):e2016930.

Chen, M. K., J. Chevalier, and E. Long. 2020. Nursing home staff networks and COVID-19. https://www.nber.org/papers/w27608 (accessed February 1, 2022).

Cleary, P. D., and B. J. McNeil. 1988. Patient satisfaction as an indicator of quality care. Inquiry 25(1):25–36.

CMS (Centers for Medicare & Medicaid Services). 2003. Nursing Home Quality Initiative overview. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/NursingHomeQualityInits/Downloads/NHQIOverView20030731.pdf (accessed July 29, 2021).

CMS. 2012. Minimum Data Set 3.0 public reports. https://www.cms.gov/Research-Statistics-Data-and-Systems/Computer-Data-and-Systems/Minimum-Data-Set-3-0-Public-Reports (accessed November 1, 2021).

CMS. 2016. QAPI description and background. https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/QAPI/qapidefinition (accessed July 29, 2021).

CMS. 2017a. Nursing home quality initiatives questions and answers. https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/QAPI/Downloads/Nursing-Home-Quality-Initiatives-FAQ.pdf (accessed March 2, 2021).

CMS. 2017b. Data show National Partnership to Improve Dementia care achieves goals to reduce unnecessary antipsychotic medications in nursing homes. https://www.cms.gov/newsroom/fact-sheets/data-show-national-partnership-improve-dementia-care-achieves-goals-reduce-unnecessary-antipsychotic (accessed July 29, 2021).

CMS. 2018. Transition to payroll-based journal (PBJ) staffing measures on the Nursing Home Compare tool on medicare.Gov and the five star quality rating system. https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertificationGenInfo/downloads/QSO18-17-NH.pdf (accessed November 4, 2021).

CMS. 2019. Updates to the Nursing Home Compare website and the five star quality rating system. https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertificationGenInfo/Downloads/QSO-20-02-NH.pdf (accessed November 5, 2021).

CMS. 2020a. Quality measures. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures (accessed May 24, 2021).

CMS. 2020b. MDS 3.0 quality measures user’s manual (v14.0). https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/NursingHomeQualityInits/NHQIQualityMeasures (accessed November 4, 2021).

CMS. 2020c. Consumer Assessment of Healthcare Providers and Systems (CAHPS). https://www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS (accessed August 16, 2021).

CMS. 2021a. Design for care compare nursing home five-star quality rating system: Technical users’ guide. https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/CertificationandComplianc/Downloads/usersguide.pdf (accessed November 1, 2021).

CMS. 2021b. Brief explanation of five star rating methodology. https://www.cms.gov/medicare/provider-enrollment-and-certification/certificationandcomplianc/downloads/brieffivestartug.pdf (accessed November 1, 2021).

CMS. 2021c. Provider information. https://data.cms.gov/provider-data/dataset/4pq5-n9py (accessed October 8, 2021).

CMS. 2021d. National Partnership to Improve Dementia Care in Nursing Homes. https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertificationGenInfo/National-Partnership-to-Improve-Dementia-Care-in-Nursing-Homes (accessed April 8, 2021).

Cockburn, A. 2020. Elder abuse: Nursing homes, the coronavirus, and the bottom line. https://harpers.org/archive/2020/09/elder-abuse-nursing-homes-COVID-19 (accessed March 15, 2021).

Conway, P. H., F. Mostashari, and C. Clancy. 2013. The future of quality measurement for improvement and accountability. JAMA 309(21):2215–2216.

CoreQ. 2019. CoreQ. http://www.coreq.org (accessed March 28, 2021).

Cornell, P. Y., D. C. Grabowski, E. C. Norton, and M. Rahman. 2019. Do report cards predict future quality? The case of skilled nursing facilities. Journal of Health Economics 66:208–221.

Davila, H., T. P. Shippee, Y. S. Park, D. Brauner, R. M. Werner, and R. T. Konetzka. 2021. Inside the black box of improving on nursing home quality measures. Medical Care Research and Review 78(6):758–770.

Dillman, D. A., J. D. Smyth, and L. M. Christian. 2014. Internet, phone, mail, and mixed-mode surveys: The tailored design method, 4th ed. New York: John Wiley & Sons.

Donabedian, A. 1988. The quality of care: How can it be assessed? JAMA 260(12):1743–1748.

Edelen, M. O., and D. Saliba. 2010. Correspondence of verbal descriptor and numeric rating scales for pain intensity: An item response theory calibration. Journals of Gerontology, Series A: Biological and Medical Sciences 65(7):778–785.

Edelman, L. S., J. Drost, R. P. Moone, K. Owens, G. L. Towsley, G. Tucker-Roghi, and J. E. Morley. 2021. Editorial: Applying the age-friendly health system framework to long term care settings. Journal of Nutrition, Health & Aging 25(2):141–145.

Edelman, T. 2015. Changes to nursing home compare and the five star quality rating system. https://medicareadvocacy.org/changes-to-nursing-home-compare-and-the-five-star-quality-rating-system (accessed February 1, 2022).

Ejaz, F. K., J. K. Straker, K. Fox, and S. Swami. 2003. Developing a satisfaction survey for families of Ohio’s nursing home residents. The Gerontologist 43(4):447–458.

Encyclopaedia Britannica. 2017. Land-grant universities. https://www.britannica.com/topic/land-grant-university (accessed February 7, 2022).

Ersek, M., J. Thorpe, H. Kim, A. Thomasson, and D. Smith. 2015. Exploring end-of-life care in Veterans Affairs community living centers. Journal of the American Geriatrics Society 63(4):644–650.

Figueroa, J. F., R. K. Wadhera, I. Papanicolas, K. Riley, J. Zheng, E. J. Orav, and A. K. Jha. 2020. Association of nursing home ratings on health inspections, quality of care, and nurse staffing with COVID-19 cases. Journal of the American Medical Association 324(11).

Fleuren, N., M. F. I. A. Depla, D. J. A. Janssen, M. Huisman, and C. M. P. M. Hertogh. 2020. Underlying goals of advance care planning (ACP): A qualitative analysis of the literature. BMC Palliative Care 19(1):27.

Freedland, S. 2009. Grassley questions CMS’ use of Defense Audit Agency to oversee QIOs. Inside CMS 12(12):10–12.

Frentzel, E. M., J. A. Sangl, C. T. Evensen, C. Cosenza, J. A. Brown, S. Keller, and S. A. Garfinkel. 2012. Giving voice to the vulnerable: The development of a CAHPS nursing home survey measuring family members’ experiences. Medical Care 50(Suppl):S20–S27.

Fuller, R. L., N. I. Goldfield, J. S. Hughes, and E. C. McCullough. 2019. Nursing Home Compare star rankings and the variation in potentially preventable emergency department visits and hospital admissions. Population Health Management 22(2):144–152.

Fulmer, T. 2018. Discovering the 4Ms: A framework for creating age-friendly health systems. https://www.johnahartford.org/blog/view/discovering-the-4ms-a-framework-for-creating-age-friendly-health-systems (accessed February 25, 2021).

GAO (U.S. Government Accountability Office). 2010. Poorly performing nursing homes: Special focus facilities are often improving, but CMS’s program could be strengthened. Washington, DC: U.S. Government Accountability Office.

GAO. 2015. Nursing home quality: CMS should continue to improve data and oversight. Washington, DC: U.S. Government Accountability Office.

GAO. 2016. Nursing homes: Consumers could benefit from improvements to the Nursing Home Compare website and five-star quality rating system. Washington, DC: U.S. Government Accountability Office.

Geng, F., D. G. Stevenson, and D. C. Grabowski. 2019. Daily nursing home staffing levels highly variable, often below CMS expectations. Health Affairs 38(7):1095–1100.

Gillespie, S. M., T. Olsan, D. Liebel, X. Cai, R. Stewart, P. R. Katz, and J. Karuza. 2016. Pioneering a nursing home quality improvement learning collaborative: A case study of method and lessons learned. Journal of the American Medical Directors Association 17(2):136–141.

Glazier, S. R., J. Schuman, E. Keltz, A. Vally, and R. H. Glazier. 2004. Taking the next steps in goal ascertainment: A prospective study of patient, team, and family perspectives using a comprehensive standardized menu in a geriatric assessment and treatment unit. Journal of the American Geriatrics Society 52(2):284–289.

Gorges, R. J., and R. T. Konetzka. 2021. Factors associated with racial differences in deaths among nursing home residents with COVID-19 infection in the U.S. JAMA Network Open 4(2):1–10.

Gozalo, P., J. M. Teno, S. L. Mitchell, J. Skinner, J. Bynum, D. Tyler, and V. Mor. 2011. End-of-life transitions among nursing home residents with cognitive issues. New England Journal of Medicine 365(13):1212–1221.

Grabowski, D. C. 2020. Strengthening nursing home policy for the postpandemic world: How can we improve residents’ health outcomes and experiences? https://www.commonwealthfund.org/publications/issue-briefs/2020/aug/strengthening-nursing-home-policy-postpandemic-world (accessed November 5, 2021).

Great Plains Quality Innovation Network. 2019. National Nursing Home Quality Improvement Campaign changes. https://greatplainsqin.org/blog/23950 (accessed November 30, 2021).

Green, A. R. 2017. Time for nursing homes to recognize and address disparities in care. The Joint Commission Journal on Quality and Patient Safety 43(11):551–553.

Han, X., N. Yaraghi, and R. Gopal. 2016. Five-star ratings for sub-par service: Evidence of inflation in nursing home ratings. https://www.brookings.edu/research/five-star-ratings-for-sub-par-service-evidence-of-inflation-in-nursing-home-ratings (accessed February 25, 2021).

Harris, Y., and S. B. Clauser. 2002. Achieving improvement through nursing home quality measurement. Health Care Financing Review 23(4):5–18.

He, M., Y. Li, and F. Fang. 2020. Is there a link between nursing home reported quality and COVID-19 cases? Evidence from California skilled nursing facilities. Journal of the American Medical Directors Association 21(7):905–908.

Hefele, J. G., X. J. Wang, and E. Lim. 2019. Fewer bonuses, more penalties at skilled nursing facilities serving vulnerable populations. Health Affairs 38(7):1127–1131.

Hirschhorn, L. R., H. Magge, and A. Kiflie. 2021. Aiming beyond equality to reach equity: The promise and challenge of quality improvement. BMJ (Clinical Research Ed.) 374:n939.

Holt, D. T., C. D. Helfrich, C. G. Hall, and B. J. Weiner. 2010. Are you ready? How health professionals can comprehensively conceptualize readiness for change. Journal of General Internal Medicine 25(Suppl 1):50–55.

Housen, P., G. R. Shannon, B. Simon, M. O. Edelen, M. P. Cadogan, L. Sohn, M. Jones, J. L. Buchanan, and D. Saliba. 2008. What the resident meant to say: Use of cognitive interviewing techniques to develop questionnaires for nursing home residents. The Gerontologist 48(2):158–169.

Housen, P., G. R. Shannon, B. Simon, M. O. Edelen, M. P. Cadogan, M. Jones, J. Buchanan, and D. Saliba. 2009. Why not just ask the resident? Journal of Gerontological Nursing 35(11):40–49.

Huckfeldt, P. J., R. L. Kane, Z. Yang, G. Engstrom, R. Tappen, C. Rojido, D. Newman, B. Reyes, and J. G. Ouslander. 2018. Degree of implementation of the Interventions to Reduce Acute Care Transfers (INTERACT) quality improvement program associated with number of hospitalizations. Journal of the American Geriatrics Society 66(9):1830–1837.

IHI (Institute for Healthcare Improvement). 2003. The breakthrough series: IHI’s collaborative model for achieving breakthrough improvement. Boston MA: Institute for Healthcare Improvement.

IHI. 2020. COVID-19 rapid response network for nursing homes: Tackling high-priority COVID-19 challenges for nursing homes. http://www.ihi.org/Engage/Initiatives/COVID-19-Rapid-Response-Network-for-Nursing-Homes/Pages/default.aspx (accessed November 1, 2021).

IHI. 2021. What is an age-friendly health system? http://www.ihi.org/Engage/Initiatives/Age-Friendly-Health-Systems/Pages/default.aspx (accessed November 18, 2021).

Integra Med Analytics. 2020. Underreporting in nursing home quality measures. https://www.nursinghomereporting.com/post/underreporting-in-nursing-home-quality-measures (accessed February 5, 2022).

IOM (Institute of Medicine). 1986. Improving the quality of care in nursing homes. Washington, DC: National Academy Press.

IOM. 2000. To err is human: Building a safer health systems. Washington, DC: National Academy Press.

IOM. 2001a. Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.

IOM. 2001b. Improving the quality of long-term care. Washington, DC: National Academy Press.

IOM. 2006. Medicare’s Quality Improvement Organization program: Maximizing potential. Washington, DC: The National Academies Press.

Ivers, N., G. Jamtvedt, S. Flottorp, J. M. Young, J. Odgaard-Jensen, S. D. French, M. A. O’Brien, M. Johansen, J. Grimshaw, and A. D. Oxman. 2012. Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2012(6):CD000259.

Ivers, N. M., A. Sales, H. Colquhoun, S. Michie, R. Foy, J. J. Francis, and J. M. Grimshaw. 2014. No more “business as usual” with audit and feedback interventions: Towards an agenda for a reinvigorated intervention. Implementation Science 9(1):14.

Jester, D. J., K. Hyer, and J. R. Bowblis. 2020. Quality concerns in nursing homes that serve large proportions of residents with serious mental illness. The Gerontologist 60(7):1312–1321.

Jevaji, S. 2016. The Q series: What is health care quality assurance? https://blog.ncqa.org/the-q-series-what-is-health-care-quality-assurance (accessed November 16, 2021).

Jewish Healthcare Foundation. 2021. Pennsylvania teaching nursing home pilot aims to transform care model. https://www.jhf.org/press-releases/entry/pennsylvania-teaching-nursing-home-pilot-aims-to-transform-care-model (accessed November 19, 2021).

Joint Commission. 2021. Joint Commission FAQs. https://www.jointcommission.org/about-us/facts-about-the-joint-commission/joint-commission-faqs (accessed November 1, 2021).

Kane, R. A., K. C. Kling, B. Bershadsky, R. L. Kane, K. Giles, H. B. Degenholtz, J. Liu, and L. J. Cutler. 2003. Quality of life measures for nursing home residents. Journals of Gerontology: Series A, Biological and Medical Sciences 58(3):240–248.

Kane, R. A., R. L. Kane, B. Bershadsky, L. J. Cutler, K. Giles, J. J. Liu, K. Kang, L. Zhang, K. C. Kling, and H. B. Degenholtz. 2004. Measures, indicators, and improvement of quality of life in nursing homes (RFP: HCFA-98-002/PK). https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/NursingHomeQualityInits/Downloads/NHQIVol2-ContentsPreface.pdf (accessed April 9, 2021).

Kane, R. L., R. A. Kane, B. Bershadsky, H. Degenholtz, K. Kling, A. Totten, and K. Jung. 2005. Proxy sources for information on nursing home residents’ quality of life. Journals of Gerontology, Series B 60(6):S318–S325.

Kane, R. L., G. Arling, C. Mueller, R. Held, and V. Cooke. 2007. A quality-based payment strategy for nursing home care in Minnesota. The Gerontologist 47(1):108–115.

Kane, R. L., P. Huckfeldt, R. Tappen, G. Engstrom, C. Rojido, D. Newman, Z. Yang, and J. G. Ouslander. 2017. Effects of an intervention to reduce hospitalizations from nursing homes: A randomized implementation trial of the INTERACT program. JAMA Internal Medicine 177(9):1257–1264.

Karlawish, J., D. C. Grabowski, and A. K. Hoffman. 2020. Opinion: Continued bans on nursing home visitors are unhealthy and unethical. https://www.washingtonpost.com/opinions/2020/07/13/residents-good-nursing-homes-should-consider-re-allowing-visitors (accessed April 6, 2021).

Katz, S., and C. A. Akpom. 1976. A measure of primary sociobiological functions. International Journal of Health Services 6(3):493–508.

Khairat, S., L. C. Zalla, J. Adler-Milstein, and C. E. Kistler. 2021. U.S. nursing home quality ratings associated with COVID-19 cases and deaths. Journal of the American Medical Directors Association 22(10):2021–2025.

Kim, S. J., E. C. Park, S. Kim, S. Nakagawa, J. Lung, J. B. Choi, W. S. Ryu, T. J. Min, H. P. Shin, K. Kim, and J. W. Yoo. 2014. The association between quality of care and quality of life in long-stay nursing home residents with preserved cognition. Journal of the American Medical Directors Association 15(3):220–225.

Klein, D. J., M. N. Elliott, A. M. Haviland, D. Saliba, Q. Burkhart, C. Edwards, and A. M. Zaslavsky. 2011. Understanding nonresponse to the 2007 Medicare CAHPS survey. The Gerontologist 51(6):843–855.

Konetzka, R. T., and Z. S. Gray. 2017. The role of socioeconomic status in nursing home quality ratings. Seniors Housing & Care Journal 25(1):3–14.

Konetzka, R. T., and M. C. Perraillon. 2016. Use of Nursing Home Compare website appears limited by lack of awareness and initial mistrust of the data. Health Affairs 35(4):706–713.

Konetzka, R. T., D. J. Brauner, M. Coca Perraillon, and R. M. Werner. 2015. The role of severe dementia in nursing home report cards. Medical Care Research and Review 72(5):562–579.

Konetzka, R. T., K. Yan, and R. M. Werner. 2020. Two decades of Nursing Home Compare: What have we learned? Medical Care Research and Review 78(4):295–310.

Krosnick, J. A. 1999. Survey research. Annual Review of Psychology 50(1):537–567.

Landrum, M., C. Nguyen, and M. Chernew. 2019. Measurement systems: A framework for next generation measurement of quality in healthcare. National Quality Forum. http://www.qualityforum.org/Story/Measurement_Systems_White_Paper.aspx (accessed February 25, 2021).

Lang, N. M., and J. F. Clinton. 1983. Assessment and assurance of the quality of nursing care: A selected overview. Evaluation & the Health Professions 6(2):211–231.

Lawton, M. P. 1991. A multidimensional view of quality of life in frail elders. In J. Birren, J. Lubben, J. Rowe, and D. Deutchman (eds.), The concept and measurement of quality of life in the frail elderly. San Diego: Academic Press. Pp. 3–27.

Le, L. T., B. J. Anthony, S. M. Bronheim, C. M. Holland, and D. F. Perry. 2014. A technical assistance model for guiding service and systems change. The Journal of Behavioral Health Sciences & Research 43(3).

LeadingAge. 2015. What the 5-star rating system changes mean for nursing homes. https://leadingage.org/members/what-5-star-rating-system-changes-mean-nursing-homes (accessed February 1, 2022).

Li, Y., X. Cai, L. G. Glance, W. D. Spector, and D. B. Mukamel. 2009. National release of the nursing home quality report cards: Implications of statistical methodology for risk adjustment. Health Services Research 44(1):79–102.

Li, Y., J. Schnelle, W. D. Spector, L. G. Glance, and D. B. Mukamel. 2010. The “Nursing Home Compare” measure of urinary/fecal incontinence: Cross-sectional variation, stability over time, and the impact of case mix. Health Services Research 45(1):79–97.

Li, Y., Z. Ye, L. G. Glance, and H. Temkin-Greener. 2014. Trends in family ratings of experience with care and racial disparities among Maryland nursing homes. Medical Care 52(7):641–648.

Li, Y., Q. Li, and Y. Tang. 2016. Associations between family ratings on experience with care and clinical quality-of-care measures for nursing home residents. Medical Care Research and Review 73(1):62–84.

Li, Y., H. Temkin-Greener, G. Shan, and X. Cai. 2020a. COVID-19 infections and deaths among Connecticut nursing home residents: Facility correlates. Journal of the American Geriatrics Society 68(9):1899–1906.

Li, Y., X. Cen, X. Cai, and H. Temkin-Greener. 2020b. Racial and ethnic disparities in COVID-19 infections and deaths across U.S. nursing homes. Journal of the American Geriatrics Society 68(11):2454–2461.

Lingum, N. R., L. G. Sokoloff, R. M. Meyer, S. Gingrich, D. J. Sodums, A. T. Santiago, S. Feldman, S. Guy, A. Moser, S. Shaikh, C. J. Grief, and D. K. Conn. 2021. Building long-term care staff capacity during COVID-19 through just-in-time learning: Evaluation of a modified ECHO model. Journal of the American Medical Directors Association 22(2):238–244.e231.

Martino, S. C., M. N. Elliott, A. M. Haviland, D. Saliba, Q. Burkhart, and D. E. Kanouse. 2016. Comparing the health care experiences of Medicare beneficiaries with and without depressive symptoms in Medicare managed care versus fee-for-service. Health Services Research 51(3):1002–1020.

McGlynn, E. A. 2003. Introduction and overview of the conceptual framework for a national quality measurement and reporting system. Medical Care 41(1 Suppl):I1–I7.

MedPAC (Medicare Payment Advisory Commission). 2021. Report to Congress: Medicare and the health care delivery system. http://www.medpac.gov/docs/default-source/reports/jun21_medpac_report_to_congress_sec.pdf (accessed August 16, 2021).

Mezey, M. D., and J. E. Lynaugh. 1989. The teaching nursing home program. Outcomes of care. The Nursing Clinics of North America 24(3):769–780.

Mezey, M. D., J. E. Lynaugh, and M. M. Cartier. 1988. The teaching nursing home program, 1982–1987: A report card. Nursing Outlook 36:285–292.

Mezey, M. D., E. L. Mitty, and S. G. Burger. 2008. Rethinking teaching nursing homes: Potential for improving long-term care. The Gerontologist 48(1):8–15.

Michigan DHHS (Michigan Department of Health and Human Services). 2021a. MDHHS—Nursing facility quality measure initiative. https://www.michigan.gov/mdhhs (accessed February 2, 2021).

Minnesota DHHS (Minnesota Department of Health and Minnesota Department of Human Services). 2021. Minnesota nursing home report card technical user guide. https://nhreportcard.dhs.mn.gov/technicaluserguide.pdf (accessed July 6, 2021).

Mochel, A. L., C. L. Holle, J. L. Rudolph, J. G. Ouslander, D. Saliba, V. Mor, and B. S. Mittman. 2021. Influencing factors associated with implementation of INTERACT (Interventions to Reduce Acute Care Transfers) in VA community living centers (CLCs) using the consolidated framework. Journal of Aging & Social Policy, June 4:1–17.

Mor, V., J. Zinn, J. Angelelli, J. M. Teno, and S. C. Miller. 2004. Driven to tiers: Socioeconomic and racial disparities in the quality of nursing home care. The Milbank Quarterly 82(2):227–256.

Morris, J. N., C. Hawes, B. E. Fries, C. D. Phillips, V. Mor, S. Katz, K. Murphy, M. L. Drugovich, and A. S. Friedlob. 1990. Designing the national resident assessment instrument for nursing homes. The Gerontologist 30(3):293–307.

Mukamel, D. B., L. G. Glance, Y. Li, D. L. Weimer, W. D. Spector, J. S. Zinn, and L. Mosqueda. 2008. Does risk adjustment of the CMS quality measures for nursing homes matter? Medical Care 46(5):532–541.

Mukamel, D. B., T. Caprio, R. Ahn, N. T. Zheng, S. Norton, T. Quill, and H. Temkin-Greener. 2012. End-of-life quality-of-care measures for nursing homes: Place of death and hospice. Journal of Palliative Medicine 15(4):438–446.

Mukamel, D. B., H. Ladd, T. Caprio, and H. Temkin-Greener. 2016. Prototype end-of-life quality measures based on MDS 3 data. Medical Care 54(11):1024–1032.

Mukamel, D. B., D. L. Weimer, Y. Shi, H. Ladd, and D. Saliba. 2020. Comparison of consumer rankings with Centers for Medicare & Medicaid Services five-star rankings of nursing homes. JAMA Network Open 3(5):e204798.

Mukamel, D. B., D. Saliba, D. L. Weimer, and H. Ladd. 2021. Families’ and residents’ perspectives of the quality of nursing home care: Implications for composite quality measures. Journal of the American Medical Directors Association 22(8):1609–1614.e1601.

Mutha, S., A. Marks, I. Bau, and M. Regenstein. 2012. Bringing equity into quality improvement: An overview and opportunities ahead. https://healthforce.ucsf.edu/sites/healthforce.ucsf.edu/files/publication-pdf/6.1%20Part%201%20_Equity%20into%20QI.pdf (accessed November 11, 2021).

Nadash, P., J. G. Hefele, E. A. Miller, A. Barooah, and X. Wang. 2019. A national-level analysis of the relationship between nursing home satisfaction and quality. Research on Aging 41(3):215–240.

National Consumer Voice (National Consumer Voice for Quality Long-Term Care). 2021. National Nursing Home Quality Improvement (NNHQI) campaign. https://theconsumervoice.org/issues/recipients/nursing-home-residents/advancing-excellence (accessed May 13, 2021).

Neuman, M. D., C. Wirtalla, and R. M. Werner. 2014. Association between skilled nursing facility quality indicators and hospital readmissions. JAMA 312(15):1542–1551.

New York State Department of Health. 2021. Factors associated with nursing home infections and fatalities in New York State during the COVID-19 global health crisis. https://www.health.ny.gov/press/releases/2020/docs/nh_factors_report.pdf (accessed February 1, 2022).

Nightingale, F. 1859. Notes on nursing: What it is, and what it is not. London: Harrison. Reprint: New York: Dover Publications, 1969.

NNHQI (National Nursing Home Quality Improvement Campaign). 2021. Resources & downloads. https://nhqualitycampaign.org/resources-downloads/ (accessed November 30, 2021).

Norton, S. A., S. Ladwig, T. V. Caprio, T. E. Quill, and H. Temkin-Greener. 2018. Staff experiences forming and sustaining palliative care teams in nursing homes. The Gerontologist 58(4):e218–e225.

NQF (National Quality Forum). 2012a. Palliative care and end-of-life care—A consensus report. https://www.qualityforum.org/Publications/2012/04/Palliative_Care_and_End-of-Life_Care%e2%80%94A_Consensus_Report.aspx (accessed May 24, 2021).

NQF. 2012b. Performance measurement coordination strategy for hospice and palliative care. https://www.qualityforum.org/Publications/2012/06/Performance_Measurement_Coordination_Strategy_for_Hospice_and_Palliative_Care.aspx (accessed May 24, 2021).

NQF. 2013. Patient-reported outcomes in performance measurement. https://www.qualityforum.org/Publications/2012/12/Patient-Reported_Outcomes_in_Performance_Measurement.aspx (accessed February 25, 2021).

NQF. 2020. Measure Applications Partnership 2020 considerations for implementing measures. Final report: Post-acute care and long-term care. http://www.qualityforum.org/Publications/2020/02/MAP_2020_Considerations_for_Implementing_Measures_Final_Report_-_PAC_LTC.aspx (accessed February 25, 2021).

NQF. 2021. The ABCs of measurement https://www.qualityforum.org/Measuring_Performance/ABCs_of_Measurement.aspx (accessed December, 31, 2020).

NRC Health. 2021. Post-acute customer experience. https://nrchealth.com/solutions-old/post-acute-customer-experience (accessed November 1, 2021).

Ochieng, N., P. Chidambaram, R. Garfield, and T. Neuman. 2021. Factors associated with COVID-19 cases and deaths in long-term care facilities: Findings from a literature review. https://www.kff.org/coronavirus-covid-19/issue-brief/factors-associated-with-covid-19-cases-and-deaths-in-long-term-care-facilities-findings-from-a-literature-review (accessed February 1, 2022).

OIG (Office of the Inspector General). 2015. Quality Improvement Organizations provide support to more than half of hospital but overlap with other programs. Washington, DC: Department of Health and Human Services Office of the Inspector General.

OIG. 2021. CMS use of data on nursing home staffing: Progress and opportunities to do more. Washington, DC: Department of Health and Human Services Office of the Inspector General.

Ouslander, J. G., and D. C. Grabowski. 2020. COVID-19 in nursing homes: Calming the perfect storm. Journal of the American Geriatrics Society 68:2153–2162.

Ouslander, J. G., G. Lamb, R. Tappen, L. Herndon, S. Diaz, B. A. Roos, D. C. Grabowski, and A. Bonner. 2011. Interventions to reduce hospitalizations from nursing homes: Evaluation of the INTERACT II collaborative quality improvement project. Journal of the American Geriatrics Society 59(4):745–753.

Ouslander, J. G., A. Bonner, L. Herndon, and J. Shutes. 2014. The Interventions to Reduce Acute Care Transfers (INTERACT) quality improvement program: An overview for medical directors and primary care clinicians in long term care. Journal of the American Medical Directors Association 15(3):162–170.

Perraillon, M. C., D. J. Brauner, and R. T. Konetzka. 2019a. Nursing home response to Nursing Home Compare: The provider perspective. Medical Care Research and Review 76(4):425–443.

Perraillon, M. C., R. T. Konetzka, D. He, and R. M. Werner. 2019b. Consumer response to composite ratings of nursing home quality. American Journal of Health Economics 5(2):165–190.

Perry, M., C. Stanik, A. Montgomery, and S. Slocum. 2021. Perspective: The case for person-centered residential long-term care in 2021: Measure it, move it forward. https://altarum.org/news/case-person-centered-residential-long-term-care-2021-measure-it-move-it-forward (accessed November 16, 2021).

Phillips, C. D., C. Hawes, T. Lieberman, and M. J. Koren. 2007. Where should momma go? Current nursing home performance measurement strategies and a less ambitious approach. BMC Health Services Research 7:93–93.

Phillips, L., C. Oyewusi, N. Martin, L. Youse, and M. Rantz. 2018. Impact of survey readiness training on nursing home quality of care. Innovation in Aging 2(Suppl 1):723–723.

Pool, D., and W. Boren. 2020. Second responders: Answering the call for help in long-term care. https://www.mcknights.com/blogs/second-responders-answering-the-call-for-help-in-long-term-care (accessed April 15, 2021).

Popejoy, L. L., M. J. Rantz, V. Conn, D. Wipke-Tevis, V. T. Grando, and R. Porter. 2000. Improving quality of care in nursing facilities. Gerontological clinical nurse specialist as research nurse consultant. Journal of Gerontological Nursing 26(4):6–13.

Popejoy, L., A. Vogelsmeier, W. Boren, N. Martin, S. Kist, K. Canada, S. J. Miller, and M. Rantz. 2020. A coordinated response to the COVID-19 pandemic in Missouri nursing homes. Journal of Nursing Care Quality 35(4):287–292.

Powell, B. J., T. J. Waltz, M. J. Chinman, L. J. Damschroder, J. L. Smith, M. M. Matthieu, E. K. Proctor, and J. E. Kirchner. 2015. A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science 10(1):21.

Project ECHO. 2021. Partnering with nursing homes: AHRQ ECHO national nursing home COVID-19 action network. https://hsc.unm.edu/echo/institute-programs/nursing-home/pages/ (accessed May 12, 2021).

Pyle, E. 2017. State website shows where Ohio nursing homes excel, fall short. https://www.dispatch.com/news/20170611/state-website-shows-where-ohio-nursing-homes-excel-fall-short (accessed February 2, 2021).

Rantz, M. J., L. Popejoy, G. F. Petroski, R. W. Madsen, D. R. Mehr, M. Zwygart-Stauffacher, L. L. Hicks, V. Grando, D. D. Wipke-Tevis, J. Bostick, R. Porter, V. S. Conn, and M. Maas. 2001. Randomized clinical trial of a quality improvement intervention in nursing homes. The Gerontologist 41(4):525–538.

Rantz, M. J., T. V. Miller, L. L. Popejoy, and M. Zwygart-Stauffacher. 2002. Outcome-based quality improvement for long-term care: Using MDS, process, and outcomes measures. 2nd ed. New York: Aspen Publishers, Inc.

Rantz, M. J., A. Vogelsmeier, P. Manion, D. Minner, B. Markway, V. Conn, M. A. Aud, and D. R. Mehr. 2003. Statewide strategy to improve quality of care in nursing facilities. The Gerontologist 43(2):248–258.

Rantz, M. J., M. Zwygart-Stauffacher, D. R. Mehr, G. F. Petroski, S. V. Owen, R. W. Madsen, M. Flesner, V. Conn, J. Bostick, R. Smith, and M. Maas. 2006. Field testing, refinement, and psychometric evaluation of a new measure of nursing home care quality. Journal of Nursing Measurement 14(2):129–148.

Rantz, M. J., D. Cheshire, M. Flesner, G. F. Petroski, L. Hicks, G. Alexander, M. A. Aud, C. Siem, K. Nguyen, C. Boland, and S. Thomas. 2009. Helping nursing homes “at risk” for quality problems: A statewide evaluation. Geriatric Nursing 30(4):238–249.

Rantz, M. J., M. Zwygart-Stauffacher, L. Hicks, D. Mehr, M. Flesner, G. F. Petroski, R. W. Madsen, and J. Scott-Cawiezell. 2012a. Randomized multilevel intervention to improve outcomes of residents in nursing homes in need of improvement. Journal of the American Medical Directors Association 13(1):60–68.

Rantz, M. J., M. Zwygart-Stauffacher, M. Flesner, L. Hicks, D. Mehr, T. Russell, and D. Minner. 2012b. Challenges of using quality improvement methods in nursing homes that “need improvement.” Journal of the American Medical Directors Association 13(8):732–738.

Rantz, M. J., M. Zwygart-Stauffacher, M. Flesner, L. Hicks, D. Mehr, T. Russell, and D. Minner. 2013. The influence of teams to sustain quality improvement in nursing homes that “need improvement.” Journal of the American Medical Directors Association 14(1):48–52.

Rau, J., and A. Almendrala. 2020. COVID-plagued California nursing homes often had problems in past. https://khn.org/news/covid-plagued-california-nursing-homes-often-had-problems-in-past/ (accessed February 1, 2022).

Reape, P. 2019. April 2019 five-star ratings decrease for 37% of nursing facilities. https://hwco.cpa/april-2019-five-star-ratings-decrease-for-37-of-nursing-facilities/ (accessed February 1, 2022).

Reichard, J. 2007. Medicare quality improvement stagnating, Senators complain. https://www.commonwealthfund.org/publications/newsletter-article/medicare-qualityimprovement-stagnating-senators-complain (accessed July 6, 2021).

Roberts, T. J., A. Gilmore-Bykovskyi, M. Lor, D. Liebzeit, C. J. Crnich, and D. Saliba. 2018. Important care and activity preferences in a nationally representative sample of nursing home residents. Journal of the American Medical Directors Association 19(1):25–32.

Rollow, W., T. R. Lied, P. McGann, J. Poyer, L. LaVoie, R. T. Kambic, D. W. Bratzler, A. Ma, E. D. Huff, and L. D. Ramunno. 2006. Assessment of the Medicare Quality Improvement Organization Program. Annals of Internal Medicine 145(5):342–353.

Rowan, P., R. Gupta, and R. S. Lester. 2020. A study of the COVID-19 outbreak and response in Connecticut long-term care facilities: Final report. Princeton, NJ: Mathematica.

RTI International. 2012. MDS 3.0 quality measures user’s manual (v5.0). https://www.cms.gov/medicare/quality-initiatives-patient-assessment-instruments/nursinghomequalityinits/downloads/mds30qm-manual.pdf (accessed November 4, 2021).

RTI International. 2017. Evaluation of the Initiative to Reduce Avoidable Hospitalizations among Nursing Facility Residents: Final report. Waltham, MA: RTI International.

RTI International. 2021. Evaluation of the Initiative to Reduce Avoidable Hospitalizations among Nursing Facility Residents—Payment reform: Fourth annual report. Baltimore, MD: RTI International.

Ryskina, K. L., R. T. Konetzka, and R. M. Werner. 2018. Association between 5-star nursing home report card ratings and potentially preventable hospitalizations. Inquiry 55:46958018787323.

Saliba, D., and J. Buchanan. 2008. Development & validation of a revised nursing home assessment tool: MDS 3.0. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/NursingHomeQualityInits/downloads/MDS30FinalReport.pdf (accessed February 25, 2021).

Saliba, D., and J. Buchanan. 2012. Making the investment count: Revision of the Minimum Data Set for nursing homes, MDS 3.0. Journal of the American Medical Directors Association 13(7):602–610.

Saliba, D., and J. F. Schnelle. 2002. Indicators of the quality of nursing home residential care. Journal of the American Geriatrics Society 50(8):1421–1430.

Saliba, D., S. DiFilippo, M. O. Edelen, K. Kroenke, J. Buchanan, and J. Streim. 2012. Testing the PHQ-9 interview and observational versions (PHW-9 OV) for MDS 3.0. Journal of the American Medical Directors Association 13(7):618–625.

Saliba, D., D. L. Weimer, Y. Shi, and D. B. Mukamel. 2018. Examination of the new short-stay nursing home quality measures: Rehospitalizations, emergency department visits, and successful returns to the community. Inquiry 55:46958018786816.

Sanders, J. J., J. R. Curtis, and J. A. Tulsky. 2018. Achieving goal-concordant care: A conceptual model and approach to measuring serious illness communication and its impact. Journal of Palliative Medicine 21(S2):S17–S27.

Sangl, J., D. Saliba, D. R. Gifford, and D. F. Hittle. 2005. Challenges in measuring nursing home and home health quality: Lessons from the first National Healthcare Quality Report. Medical Care 43(3):I-24–I-32.

Sangl, J., J. Buchanan, C. Cosenza, S. Bernard, S. Keller, N. Mitchell, J. Brown, N. Castle, E. Sekscenski, and D. Larwood. 2007. The development of a CAHPS instrument for nursing home residents (NHCAHPS). Journal of Aging and Social Policy 19(2):63–82.

Schapira, M. M., J. A. Shea, K. A. Duey, C. Kleiman, and R. M. Werner. 2016. The Nursing Home Compare report card: Perceptions of residents and caregivers regarding quality ratings and nursing home choice. Health Services Research 51(Suppl 2):1212–1228.

Schwartz, L. 2021. CoreQ measures receive re-endorsement from the National Quality Forum (NQF). https://www.ahcancal.org/News-and-Communications/Blog/Pages/CoreQ-Measures-Receive-Re-Endorsement-from-the-National-Quality-Forum-(NQF-.aspx) (accessed July 29, 2021).

Sharma, H., R. T. Konetzka, and F. Smieliauskas. 2019. The relationship between reported staffing and expenditures in nursing homes. Medical Care Research and Review 76(6):758–783.

Sharma, H., M. C. Perraillon, R. M. Werner, D. C. Grabowski, and R. T. Konetzka. 2020. Medicaid and nursing home choice: Why do duals end up in low-quality facilities? Journal of Applied Gerontology 39(9):981–990.

Shaughnessy, P. W., A. M. Kramer, D. F. Hittle, and J. F. Steiner. 1995. Quality of care in teaching nursing homes: Findings and implications. Health Care Financing Review 16(4):55–83.

Shaw-Taylor, Y. 2014. Making quality improvement programs more effective. International Journal of Health Care Quality Assurance 27(4):264–270.

Shortell, S. M., and W. A. Peck. 2006. Enhancing the potential of Quality Improvement Organizations to improve quality of care. Annals of Internal Medicine 145(5):388–389.

Silow-Carroll, S., D. Peartree, S. Tucker, and A. Pham. 2021. Fundamental nursing home reform: Evidence on single-resident rooms to improve personal experience and public health. https://www.healthmanagement.com/wp-content/uploads/HMA.Single-Resident-Rooms-3.22.2021_final.pdf (accessed November 5, 2021).

Silver-Greenberg, J., and R. Gebeloff. 2021. Maggots, rape and yet five stars: How U.S. Ratings of nursing homes mislead the public. https://www.nytimes.com/2021/03/13/business/nursing-homes-ratings-medicare-covid.html?campaign_id=9&emc=edit_nn_20210313&instance_id=28051&nl=the-morning&regi_id=98348276&segment_id=53394&te=1&user_id=d3b6076e56d56335bd158430d6fd0d63 (accessed March 15, 2021).

Simpson, A. 2020. Nursing home coronavirus rules provoke backlash. https://www.pewtrusts.org/en/research-and-analysis/blogs/stateline/2020/04/10/nursing-home-coronavirus-rules-provoke-backlash (accessed April 6, 2021).

Sinclair School of Nursing University of Missouri. 2021a. QIPMO program statistics. https://nursinghomehelp.org/qipmo-program/statistics (accessed October 15, 2021).

Sinclair School of Nursing University of Missouri. 2021b. Leadership coaching statistics. https://nursinghomehelp.org/leadership-coaching/statistics (accessed October 15, 2021).

Sloane, P. D., S. Zimmerman, K. Ward, C. E. Kistler, D. Paone, D. J. Weber, C. J. Wretman, and J. S. Preisser. 2020. A 2-year pragmatic trial of antibiotic stewardship in 27 community nursing homes. Journal of the American Geriatrics Society 68(1):46–54.

Snyder, D. J., T. R. Kroshus, A. Keswani, E. B. Garden, K. M. Koenig, K. J. Bozic, D. S. Jevsevar, J. Poeran, and C. S. Moucha. 2019. Are Medicare’s Nursing Home Compare ratings accurate predictors of 90-day complications, readmission, and bundle cost for patients undergoing primary total joint arthroplasty? Journal of Arthroplasty 34(4):613–618.

Steel, K., G. Ljunggren, E. Topinková, J. N. Morris, C. Vitale, J. Parzuchowski, S. Nonemaker, D. H. Frijters, T. Rabinowitz, K. M. Murphy, M. W. Ribbe, and B. E. Fries. 2003. The RAI-PC: An assessment instrument for palliative care in all settings. American Journal of Hospice and Palliative Care 20(3):211–219.

Stevenson, D. G., and V. Mor. 2009. Targeting nursing homes under the Quality Improvement Organization Program’s 9th statement of work. Journal of the American Geriatrics Society 57(9):1678–1684.

Straker, J., K. McGrew, J. Dibert, C. Burch, and A. Raymore. 2016. Ohio nursing home and residential care facility satisfaction: Survey testing and development for residents and families. https://sc.lib.miamioh.edu/xmlui/bitstream/handle/2374.MIA/5925/Ohio%27s%20Nursing%20Homes%20and%20Residential%20Care%20Family%20Satisfaction.pdf (accessed February 25, 2021).

Sugg, M. M., T. J. Spaulding, S. J. Lane, J. D. Runkle, S. R. Harden, A. Hege, and L. S. Iyer. 2021. Mapping community-level determinants of COVID-19 transmission in nursing homes: A multi-scale approach. Science of the Total Environment 752:141946.

Temkin-Greener, H., Q. Li, Y. Li, M. Segelman, and D. B. Mukamel. 2016. End-of-life care in nursing homes: From care processes to quality. Journal of Palliative Medicine 19(12):1304–1311.

Teno, J. M., P. Gozalo, A. N. Trivedi, J. Bunker, J. Lima, J. Ogarek, and V. Mor. 2018. Site of death, place of care, and health care transitions among U.S. Medicare beneficiaries, 2000–2015. JAMA 320(3):264–271.

Thomas, K. 2014. Medicare star ratings allow nursing homes to game the system. https://www.nytimes.com/2014/08/25/business/medicare-star-ratings-allow-nursing-homes-to-game-the-system.html?searchResultPosition=1 (accessed February 1, 2022).

Thomas, K. 2015. Government will change how it rates nursing homes. https://www.nytimes.com/2015/02/13/business/government-will-change-how-it-rates-nursing-homes.html?searchResultPosition=2 (accessed February 1, 2022).

Thompson, S., M. Bott, D. Boyle, B. Gajewski, and V. P. Tilden. 2011. A measure of palliative care in nursing homes. Journal of Pain and Symptom Management 41(1):57–67.

Thorpe, J. M., D. Smith, N. Kuzla, L. Scott, and M. Ersek. 2016. Does mode of survey administration matter? Using measurement invariance to validate the mail and telephone versions of the Bereaved Family Survey. Journal of Pain and Symptom Management 51(3):546–556.

University of Missouri Interdisciplinary Center on Aging. 2008. Improvements in Missouri nursing homes using QIPMO services. https://agingmo.com/wp-content/uploads/2017/10/QIPMO08.pdf (accessed May 12, 2021).

Unroe, K. T., M. A. Greiner, C. Colón-Emeric, E. D. Peterson, and L. H. Curtis. 2012. Associations between published quality ratings of skilled nursing facilities and outcomes of Medicare beneficiaries with heart failure. Journal of the American Medical Directors Association 13(2):188.e1–e6.

Vasilevskis, E. E., J. G. Ouslander, A. S. Mixon, S. P. Bell, J. M. L. Jacobsen, A. A. Saraf, D. Markley, K. C. Sponsler, J. Shutes, E. A. Long, S. Kripalani, S. F. Simmons, and J. F. Schnelle. 2017. Potentially avoidable readmissions of patients discharged to post-acute care: Perspectives of hospital and skilled nursing facility staff. Journal of the American Geriatrics Society 65(2):269–276.

Vest, J. R., H. Y. Jung, K. Wiley, Jr., H. Kooreman, L. Pettit, and M. A. Unruh. 2019. Adoption of health information technology among U.S. nursing facilities. Journal of the American Medical Directors Association 20(8):995–1000.e1004.

Vogelsmeier, A., L. Popejoy, K. Canada, C. Galambos, G. Petroski, C. Crecelius, G. L. Alexander, and M. Rantz. 2021. Results of the Missouri Quality Initiative in sustaining changes in nursing home care: Six-year trends of reducing hospitalizations of nursing home residents. Journal of Nutrition, Health, and Aging 25(1):5–12.

Wandersman, A., V. H. Chien, and J. Katz. 2012. Toward an evidence-based system for innovation support for implementing innovations with quality tools: Tools, training, technical assistance, and quality assurance/quality improvement. American Journal of Community Psychology 50(3-4):445–459.

Weech-Maldonado, R., R. Pradhan, N. Dayama, J. Lord, and S. Gupta. 2019. Nursing home quality and financial performance: Is there a business case for quality? Inquiry 56:46958018825191.

Weimer, D. L., D. Saliba, H. Ladd, Y. Shi, and D. B. Mukamel. 2019. Using contingent valuation to develop consumer-based weights for health quality report cards. Health Services Research 54(4):947–956.

Weiner. B. J. 2009. A theory of organizational readiness for change. Implementation Science 4:67.

Weiner, B. J., H. Amick, and S. D. Lee. 2008. Conceptualization and measurement of organizational readiness for change. Medical Care Research and Review 65(4):379–436.

Weinick, R. M., and R. Hasnain-Wynia. 2011. Quality improvement efforts under health reform: How to ensure that they help reduce disparities—not increase them. Health Affairs 30(10):1837–1843.

Werner, R. M., and R. T. Konetzka. 2010. Advancing nursing home quality through quality improvement itself. Health Affairs (Millwood) 29(1):81–86.

Werner, R. M., R. T. Konetzka, E. A. Stuart, and D. Polsky. 2011. Changes in patient sorting to nursing homes under public reporting: Improved patient matching or provider gaming? Health Services Research 46(2):555–571.

Werner, R. M., R. T. Konetzka, and M. M. Kim. 2013. Quality improvement under Nursing Home Compare: The association between changes in process and outcome measures. Medical Care 51(7):582–588.

White, E. M., C. M. Kosar, R. A. Feifer, C. Blackman, S. Gravenstein, J. Ouslander, and V. Mor. 2020a. Variation in SARS-CoV-2 prevalence in U.S. Skilled nursing facilities. Journal of the American Geriatrics Society 68(10):2167–2173.

White, E. M., L. H. Aiken, D. M. Sloane, and M. D. McHugh. 2020b. Nursing home work environment, care quality, registered nurse burnout and job dissatisfaction. Geriatric Nursing 41(2):158–164.

White, E. M., C. M. Kosar, M. Rahman, and V. Mor. 2020c. Trends in hospitals and skilled nursing facilities sharing medical providers, 2008–16. Health Affairs 39(8):1312–1320.

Williams, A., J. K. Straker, and R. Applebaum. 2016. The nursing home five star rating: How does it compare to resident and family views of care? The Gerontologist 56(2):234–242.

Williams, C. S., Q. Zheng, A. J. White, A. I. Bengtsson, E. T. Shulman, K. R. Herzer, and L. A. Fleisher. 2021. The association of nursing home quality ratings and spread of COVID-19. Journal of the American Geriatrics Society 69(8):2070–2078.

Xu, D., R. Kane, and G. Arling. 2019. Relationship between nursing home quality indicators and potentially preventable hospitalisation. BMJ Quality and Safety 28(7):524–533.

Zhou, C., A. Crawford, E. Serhal, P. Kurdyak, and S. Sockalingam. 2016. The impact of Project ECHO on participant and patient outcomes: A systematic review. Academic Medicine 91(10):1439–1461.

Zimmerman, D. R., S. L. Karon, G. Arling, B. R. Clark, T. Collins, R. Ross, and F. Sainfort. 1995. Development and testing of nursing home quality indicators. Health Care Financing Review 16(4):107–127.

Zinn, J. S., W. D. Spector, D. L. Weimer, and D. B. Mukamel. 2008. Strategic orientation and nursing home response to public reporting of quality measures: An application of the Miles and Snow typology. Health Services Research 43(2):598–615.