National Academies Press: OpenBook

Measuring Meaningful Outcomes for Adult Hearing Health Interventions (2025)

Chapter: Appendix C: Measure Evaluation Worksheet

« Previous: Appendix B: Measure Inventory
Suggested Citation:"Appendix C: Measure Evaluation Worksheet." National Academies of Sciences, Engineering, and Medicine. 2025. Measuring Meaningful Outcomes for Adult Hearing Health Interventions. Washington, DC: The National Academies Press. doi: 10.17226/29104.
×

Appendix C

Measure Evaluation Worksheet

The criteria and rating system presented in Table C-1 was adopted from Terwee and colleagues (2007) and modified to fit the committee’s process for evaluating measures of interest. For many of the considered measures, the level of evidence needed to evaluate each criterion was not available and therefore not every component was applicable. However, the committee used the existing evidence to assess each candidate measure based on as many of these criteria as possible to select the best outcome for each measure.

Suggested Citation:"Appendix C: Measure Evaluation Worksheet." National Academies of Sciences, Engineering, and Medicine. 2025. Measuring Meaningful Outcomes for Adult Hearing Health Interventions. Washington, DC: The National Academies Press. doi: 10.17226/29104.
×

TABLE C-1 Committee Criteria and Rating System for Outcome Measure Evaluation

Measurement Property Rating* Criteria
Content validity + All items refer to relevant aspects of the construct to be measured AND are relevant for the target population AND are relevant for the context of use AND together comprehensively reflect the construct to be measured
(including face validity) ? Not all information for ‘+’ reported
Criteria for ‘+’ not met
Structural validity + Classical Test Theory (CTT):
Unidimensionality: Exploratory factor analysis: First factor accounts for at least 20% of the variability AND ratio of the variance explained by the first to the second factor greater than 4 OR Bi-factor model: Standardized loadings on a common factor > 0.30 AND correlation between individual scores under a bi-factor and unidimensional model > 0.90
Structural validity: Comparative fit index (CFI) or Tucker-Lewis Index (TLI) comparable measure > 0.95 AND root mean square error of approximation (RMSEA) < 0.06 OR standardized root mean residuals (SRMR) < 0.08
Rasch/Item Response Theory (IRT):
At least limited evidence for unidimensionality or positive structural validity AND no evidence for violation of local independence: Rasch: standardized item-person fit residuals between −2.5 and 2.5; OR IRT: residual correlations among the items after controlling for the dominant factor < 0.20 OR Q3s < 0.37 AND no evidence for violation of monotonicity: adequate looking graphs OR item scalability > 0.30 AND adequate model fit: Rasch: infit and outfit mean squares ≥ 0.5 and ≤ 1.5 OR Z-standardized values > −2 and < 2; OR IRT: G2 > 0.01;
Optional additional evidence:
Adequate targeting; Rasch: adequate person-item threshold distribution; IRT: adequate threshold range
No important differential item functioning for relevant subject characteristics (such as age, gender, education), McFadden’s R2 < 0.02
? CTT: Not all information for ‘+’ reported
IRT: Model fit not reported
Criteria for ‘+’ not met
No data available If the element hasn’t been tested/reported, leave the cell blank (this is different from having a negative finding).
Suggested Citation:"Appendix C: Measure Evaluation Worksheet." National Academies of Sciences, Engineering, and Medicine. 2025. Measuring Meaningful Outcomes for Adult Hearing Health Interventions. Washington, DC: The National Academies Press. doi: 10.17226/29104.
×
Measurement Property Rating* Criteria
Internal consistency + At least limited evidence for unidimensionality or positive structural validity AND Cronbach’s alpha(s) ≥ 0.70 and ≤ 0.95
? Not all information for ‘+’ reported OR conflicting evidence for unidimensionality or structural validity OR evidence for lack of unidimensionality or negative structural validity
Criteria for ‘+’ not met
Reliability + Intraclass correlation coefficient (ICC) or weighted Kappa ≥ 0.70
? ICC or weighted Kappa not reported
Criteria for ‘+’ not met
Measurement error + Smallest detectable change (SDC) or limits of agreement (LOA) < minimal important change (MIC)
? MIC not defined
Criteria for ‘+’ not met
Hypotheses testing + At least 75% of the results are in accordance with the hypotheses
? No correlations with instrument(s) measuring related construct(s) AND no differences between relevant groups reported
Criteria for ‘+’ not met
Criterion validity + Convincing arguments that gold standard is “gold” AND correlation with gold standard ≥ 0.70
? Not all information for ‘+’ reported
Criteria for ‘+’ not met
Responsiveness + At least 75% of the results are in accordance with the hypotheses
? No correlations with changes in instrument(s) measuring related construct(s) AND no differences between changes in relevant groups reported
Criteria for ‘+’ not met

NOTE: * “+” = positive rating, “?” = indeterminate rating, “−” = negative rating.

SOURCE: Modified from Terwee et al., 2007. Reprinted with permission from Elsevier.

REFERENCE

Terwee, C. B., S. D. Bot, M. R. de Boer, D. A. van der Windt, D. L. Knol, J. Dekker, L. M. Bouter, and H. C. de Vet. 2007. Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology 60(1):34–42.

Suggested Citation:"Appendix C: Measure Evaluation Worksheet." National Academies of Sciences, Engineering, and Medicine. 2025. Measuring Meaningful Outcomes for Adult Hearing Health Interventions. Washington, DC: The National Academies Press. doi: 10.17226/29104.
×

This page intentionally left blank.

Suggested Citation:"Appendix C: Measure Evaluation Worksheet." National Academies of Sciences, Engineering, and Medicine. 2025. Measuring Meaningful Outcomes for Adult Hearing Health Interventions. Washington, DC: The National Academies Press. doi: 10.17226/29104.
×
Page 223
Suggested Citation:"Appendix C: Measure Evaluation Worksheet." National Academies of Sciences, Engineering, and Medicine. 2025. Measuring Meaningful Outcomes for Adult Hearing Health Interventions. Washington, DC: The National Academies Press. doi: 10.17226/29104.
×
Page 224
Suggested Citation:"Appendix C: Measure Evaluation Worksheet." National Academies of Sciences, Engineering, and Medicine. 2025. Measuring Meaningful Outcomes for Adult Hearing Health Interventions. Washington, DC: The National Academies Press. doi: 10.17226/29104.
×
Page 225
Suggested Citation:"Appendix C: Measure Evaluation Worksheet." National Academies of Sciences, Engineering, and Medicine. 2025. Measuring Meaningful Outcomes for Adult Hearing Health Interventions. Washington, DC: The National Academies Press. doi: 10.17226/29104.
×
Page 226
Next: Appendix D: WIN vs. QuickSIN Table »
Measuring Meaningful Outcomes for Adult Hearing Health Interventions Get This Book
×
 Measuring Meaningful Outcomes for Adult Hearing Health Interventions
Buy Paperback | $30.00 Buy Ebook | $24.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

As people get older, they become more likely to have hearing difficulties in certain situations; for example, having conversations in public settings where there is naturally more noise. This can sometimes lead to isolation, depression, or even fatigue from the effort needed to communicate with others. Clinicians and researchers often do not use standardized outcome measures for hearing interventions that reflect patients' perceptions of real-life improvements.

The National Academies of Sciences, Engineering, and Medicine convened a committee of experts to identify a core set of outcomes and corresponding measures that researchers and clinicians should use each time they assess the effectiveness of hearing aids and other treatments in addressing hearing difficulties. The resulting report, Measuring Meaningful Outcomes for Adult Hearing Health Interventions, presents those outcomes and measures, as well as recommendations for promoting their use within the hearing health community.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!