Finding What Works in Health Care Standards for Systematic Reviews (2011) / Chapter Skim
Currently Skimming:
Summary
Summary
Pages 1-16
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
to learn what is known and not known about the potential benefits and harms of alternative drugs, devices, and other healthcare services. An SR is a scientific investigation that focuses on a specific question and uses explicit, prespecified scientific methods to identify, select, assess, and summarize the findings of similar but separate studies.
|
From page 2... ...
. This report presents methodological standards for SRs that are designed to inform everyday healthcare decision making, especially for patients, clinicians and other healthcare providers, and devel BOX S-1 Charge to the Committee on Standards for Systematic Reviews of Comparative Effectiveness Research An ad hoc committee will conduct a study to recommend methodologi cal standards for systematic reviews (SRs)
|
From page 3... ...
PURPOSE OF SETTING STANDARDS Organizations establish standards to set performance expectations and to promote accountability for meeting these expectations. For SRs in particular, the principal objective of setting standards is to minimize bias in identifying, selecting, and interpreting evidence.
|
From page 4... ...
All of the commit tee's recommended standards are based on current evidence, expert guidance, and thoughtful reasoning, and are actively used by many experts, and thus are reasonable "best practices" for reducing bias and for increasing the scientific rigor of SRs of CER. However, all of the recommended standards must be considered provisional pending better empirical evidence about their scientific validity, feasibil ity, efficiency, and ultimate usefulness in medical decision making.
|
From page 5... ...
The importance of the review questions and analytic framework in guiding the entire review process demands a rigorous approach to formulating the research questions and analytic framework. Requiring a research protocol that prespecifies the research methods at the outset of the SR process helps to prevent the effects of author bias, allows feedback at an early stage in the SR, and tells readers of the review about protocol changes that occur as the SR develops.
|
From page 6... ...
The search should be systematic, use prespecified search parameters, and access an array of information sources that provide both published and unpublished research reports. Screening and selecting
|
From page 7... ...
7 SUMMARY 2.5.2 Develop an analytic framework that clearly lays out the chain of logic that links the health intervention to the outcomes of interest and defines the key clinical questions to be ad dressed by the systematic review 2.5.3 Use a standard format to articulate each clinical question of interest 2.5.4 State the rationale for each clinical question 2.5.5 Refine each question based on user and stakeholder input Standard 2.6 Develop a systematic review protocol Required elements: 2.6.1 Describe the context and rationale for the review from both a decision-making and research perspective 2.6.2 Describe the study screening and selection criteria (inclu sion/exclusion criteria) 2.6.3 Describe precisely which outcome measures, time points, interventions, and comparison groups will be addressed 2.6.4 Describe the search strategy for identifying relevant evidence 2.6.5 Describe the procedures for study selection 2.6.6 Describe the data extraction strategy 2.6.7 Describe the process for identifying and resolving dis agreement between researchers in study selection and data extraction decisions 2.6.8 Describe the approach to critically appraising individual studies 2.6.9 Describe the method for evaluating the body of evi dence, including the quantitative and qualitative synthesis strategies 2.6.10 Describe and justify any planned analyses of differential treatment effects according to patient subgroups, how an intervention is delivered, or how an outcome is measured 2.6.11 Describe the proposed timetable for conducting the review Standard 2.7 Submit the protocol for peer review Required element: 2.6.9 Provide a public comment period for the protocol and pub licly report on disposition of comments Standard 2.8 Make the final protocol publicly available, and add any amendments to the protocol in a timely fashion studies should use methods that address the pervasive problems of SR author bias, errors, and inadequate documentation of the study selection process in SRs.
|
From page 8... ...
8 FINDING WHAT WORKS IN HEALTH CARE BOX S-3 Recommended Standards for Finding and Assessing Individual Studies Standard 3.1 Conduct a comprehensive systematic search for evidence Required elements: 3.1.1 Work with a librarian or other information specialist trained in performing systematic reviews to plan the search strategy 3.1.2 Design the search strategy to address each key research question 3.1.3 Use an independent librarian or other information specialist to peer review the search strategy 3.1.4 Search bibliographic databases 3.1.5 Search citation indexes 3.1.6 Search literature cited by eligible studies 3.1.7 Update the search at intervals appropriate to the pace of generation of new information for the research question be ing addressed 3.1.8 Search subject-specific databases if other databases are unlikely to provide all relevant evidence 3.1.9 Search regional bibliographic databases if other databases are unlikely to provide all relevant evidence Standard 3.2 Take action to address potentially biased reporting of research results Required elements: 3.2.1 Search grey-literature databases, clinical trial registries, and other sources of unpublished information about studies 3.2.2 Invite researchers to clarify information about study eligibil ity, study characteristics, and risk of bias 3.2.3 Invite all study sponsors and researchers to submit unpub lished data, including unreported outcomes, for possible inclusion in the systematic review 3.2.4 Handsearch selected journals and conference abstracts 3.2.5 Conduct a web search 3.2.6 Search for studies reported in languages other than English if appropriate Standard 3.3 Screen and select studies Required elements: 3.3.1 Include or exclude studies based on the protocol's prespeci fied criteria 3.3.2 Use observational studies in addition to randomized clinical trials to evaluate harms of interventions
|
From page 9... ...
For other types of data, one individual could extract the data while the second individual independently checks for accuracy and completeness. Establish a fair pro cedure for resolving discrepancies -- do not simply give final decision-making power to the senior reviewer 3.5.2 Link publications from the same study to avoid including data from the same study more than once 3.5.3 Use standard data extraction forms developed for the spe cific SR 3.5.4 Pilot-test the data extraction forms and process Standard 3.6 Critically appraise each study Required elements: 3.6.1 Systematically assess the risk of bias, using predefined criteria 3.6.2 Assess the relevance of the study's populations, interven tions, and outcome measures 3.6.3 Assess the fidelity of the implementation of interventions
|
From page 10... ...
Many SRs lack a qualitative synthesis altogether or simply recite the facts about the studies without examining them for patterns or characterizing the strengths and weaknesses BOX S-4 Recommended Standards for Synthesizing the Body of Evidence Standard 4.1 Use a prespecified method to evaluate the body of evidence Required elements: 4.1.1 For each outcome, systematically assess the following char acteristics of the body of evidence: • Risk of bias • Consistency • Precision • Directness • Reporting bias 4.1.2 For bodies of evidence that include observational research, also systematically assess the following characteristics for each outcome: • Dose–response association • Plausible confounding that would change the observed effect • Strength of association 4.1.3 For each outcome specified in the protocol, use consistent language to characterize the level of confidence in the esti mates of the effect of an intervention Standard 4.2 Conduct a qualitative synthesis Required elements: 4.2.1 Describe the clinical and methodological characteristics of the included studies, including their size, inclusion or exclu sion of important subgroups, timeliness, and other relevant factors
|
From page 11... ...
For readers to have a clear understanding of how the evidence applies to real-world clinical circumstances and specific patient populations, SRs should describe -- in easy-to-understand language -- the clinical and methodological characteristics of the individual studies, including their strengths and weaknesses and their relevance to particular populations and clinical settings. It should also describe how flaws in the design or execution of the individual studies could bias the results.
|
From page 12... ...
Research organizations such as the AHRQ Effective Health Care Program, CRD, and the Cochrane Collaboration have published standards, but none of these are universally accepted and consistently applied during planning, conducting, reporting, and peer review of SRs. Furthermore, the SR enterprise in the United States lacks both adequate funding and
|
From page 13... ...
The recommended standards are an appropriate starting point for publicly funded reviews in the United States (including PCORI, federal, state, and local funders) because of the heightened attention and potential clinical impact of major reviews sponsored by public agencies.
|
From page 14... ...
Describe the following: • Research protocol* • Eligibility criteria (criteria for including and excluding studies in the systematic review)
|
From page 15... ...
• Gaps in evidence • Future research needs 5.1.9 Include a section describing funding sources* and COI Standard 5.2 Peer review the draft report Required elements: 5.2.1 Use a third party to manage the peer review process 5.2.2 Provide a public comment period for the report and publicly report on disposition of comments Standard 5.3 Publish the final report in a manner that ensures free public access *
|
From page 16... ...
The committee assigns these groups with responsibility and accountability for coordinating and moving the agenda ahead. The committee found compelling evidence that having highquality SRs based on rigorous standards is a topic of international concern, and that individual colleagues, professional organizations, and publicly funded agencies in other countries make up a large proportion of the world's expertise on the topic.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.