Traumatic Injury Research at NIOSH Reviews of Research Programs of the National Institute for Occupational Safety and Health (2009) / Chapter Skim
Currently Skimming:
Appendix A: Framework for the Review of Research Programs of the National Institute for Occupational Safety and Health
Appendix A: Framework for the Review of Research Programs of the National Institute for Occupational Safety and Health
Pages 137-181
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 137... ...
This document is not a formal report of the National Academies -- rather, it is a framework proposed for use by multiple National Academies evaluation committees to review up to 15 National Institute for Occupational Safety and Health (NIOSH) research programs.
|
From page 138... ...
. • A new table was added to provide explicit guidance to evaluation com mittees on how to weigh differences in the observed levels of "research priority" and "engagement in appropriate transfer activities" in arriv ing at a single integer score for relevance (see Table 6)
|
From page 139... ...
Evaluation Committee Report Template Figure 1 The NIOSH operational plan presented as a logic model Figure 2 Flowchart for the evaluation of the NIOSH research program Table 1 NORA High-Priority Research by Category Table 2 Evaluation Committee Information Needs Table 3 Examples of NIOSH Program Research and Transfer Activities Table 4 Examples of Research-Program Outputs to Be Considered Table 5 Background Context for Program Relevance and Impact Table 6 Guidance for Weighting Research Priority and Engagement in Appropriate Transfer Activities in the Application of Relevance Score
|
From page 140... ...
140 T r a u m a t ic I n j u r y R e s e a rc h at N I OSH Table 7 Targeting of New Research and Identification of Emerging Issues Box 1 The Evaluation Process Box 2 Scoring Criteria for Relevance Box 3 Scoring Criteria for Impact Box 4 Suggested Outline for Evaluation Committee Reports
|
From page 141... ...
A p p e n d i x A 141 ABBREVIATIONS AND ACRONYMS ABLES Adult Blood Lead Epidemiology and Surveillance AOEC Association of Occupational and Environmental Clinics BLS Bureau of Labor Statistics CDC Centers for Disease Control and Prevention CSTE Council of State and Territorial Epidemiologists DOD U.S. Department of Defense EC Evaluation Committee EPA Environmental Protection Agency FACE Fatality Assessment Control and Evaluation FC Framework Committee HHE Health Hazard Evaluation MSHA Mine Safety and Health Administration NIH National Institutes of Health NIOSH National Institute for Occupational Safety and Health NORA National Occupational Research Agenda NORA1 National Occupational Research Agenda 1996-2005 NORA2 National Occupational Research Agenda 2005-forward OSH Review Occupational Safety and Health Review Commission Commission OSHA Occupational Safety and Health Administration OSHAct Occupational Safety and Health Act of 1970 PART Performance Assessment Rating Tool PEL permissible exposure limit RFA request for applications SENSOR Sentinel Event Notification System of Occupational Risks
|
From page 142... ...
142 T r a u m a t ic I n j u r y R e s e a rc h at N I OSH TMT tools, methods, or technologies USDA U.S. Department of Agriculture
|
From page 143... ...
This second version was finalized after the Framework Committee's May 30-31, 2007, meeting, based on feedback received on the framework from the first two independent evaluation committees, NIOSH leadership, and National Academies' staff, as well as discussions during an earlier FC meeting in April 2006. This document is not a report of the National Academies; rather, it presents the evaluation framework developed by the FC to guide and provide common structure for the reviews of as many as 15 NIOSH programs during a 5-year period by independent evaluation committees (ECs)
|
From page 144... ...
2. Assessment of the program's effectiveness in targeting new research areas and identifying emerging issues in occupational safety and health most relevant to future improvements in workplace protection.
|
From page 145... ...
The OSHAct identifies workplace safety and health as having high national priority and gives employers the responsibility for controlling hazards and preventing workplace injury and illness. The act creates an organizational framework for doing that, assigning complementary roles and responsibilities to employers and employees, the Occupational Safety and Health Administration (OSHA)
|
From page 146... ...
° Build global professional capacity to address workplace hazards through training, information sharing, and research experience. In 1994, NIOSH embarked on a national partnership effort to identify research priorities to guide occupational health and safety research for the next decade.
|
From page 147... ...
A p p e n d i x A 147 TABLE 1 NORA High-Priority Research by Category Category Priority Research Area Disease and injury Allergic and irritant dermatitis Asthma and chronic obstructive pulmonary disease Fertility and pregnancy abnormalities Hearing loss Infectious diseases Low-back disorders Musculoskeletal disorders of upper extremities Trauma Work environment and workforce Emerging technologies Indoor environment Mixed exposures Organization of work Special populations at risk Research tools and approaches Cancer research methods Control technology and personal protective equipment Exposure-assessment methods Health-services research Intervention-effectiveness research Risk-assessment methods Social and economic consequences of workplace illness and injury Surveillance research methods I.D. Evaluation Committees' Information Needs Each NIOSH program under review will provide information to the relevant EC, including that outlined in Table 2.
|
From page 148... ...
148 New Figure A-1, bitmapped FIGURE 1 The NIOSH operational plan presented as a logic model.
|
From page 149... ...
• Interactions with stakeholders and with other NIOSH programs: ° The role of program research staff in NIOSH policy-setting, OSHA and MSHA standard setting, voluntary standard-setting and other government policy functions. ° Interactions and working relationships with other NIOSH programs.
|
From page 150... ...
◊ Transfer activities. ◊ Capacity-building activities.
|
From page 151... ...
The major components to be evaluated are • major program challenges, • strategic goals and objectives, • inputs (such as budget, staff, facilities, the institute's research manage ment, the NIOSH Board of Scientific Counselors, the NORA process, and NORA work groups) , • activities (efforts by NIOSH staff, contractors, and grantees, such as hazard surveillance; surveillance for injury, illness, and biomarkers of effect; exposure-measurement research; safety-systems research; injury-prevention research; health-effects research; intervention 4The PART focuses on assessing program-level performance and is one of the measures of success of the budget and performance integration initiative of the president's management agenda (see CDC Occupational Safety and Health at http://www.whitehouse.gov/omb/budget/fy2006/pma/hhs.pdf)
|
From page 152... ...
152 bitmap image New Figure A-2.eps FIGURE 2 Flowchart for the evaluation of the NIOSH research program.
|
From page 153... ...
Examples of external factors are research activities of industry and other federal agencies and the political and regulatory environment. For purposes of this review, the results of inputs and external factors are the program research activities, outputs, and associated transfer activities that may result in intermediate outcomes and possibly end outcomes.
|
From page 154... ...
. Identification of external factors by an EC is essen tial because it provides the context for evaluation of the NIOSH program.
|
From page 155... ...
The ECs should also independently identify emerging workplace issues for which the NIOSH program under review should be prepared.
|
From page 156... ...
Evaluation of the impact of NIOSH research outputs on worker health and safety may require consideration of external factors that might impede or aide implementation, measurement, and so on. ECs might consider whether or not the following are true: • Regulatory end points are unachievable because of obstacles to regula tion or because of differing priorities of the regulatory agencies.
|
From page 157... ...
• The structure, in addition to the content, of the research program: A relevant research program is more than a set of unrelated research projects; it is an integrated program involving interrelated surveillance, research, and transfer activities. • Appropriate NIOSH consideration of stakeholder input.
|
From page 158... ...
, external advisory inputs, and its own expert judg ment. The EC will then be able to compare its own assessment of workplace chal lenges with the NIOSH program goals and objectives.
|
From page 159... ...
a. Did past program goals and objectives (research and dissemination and transfer activities)
|
From page 160... ...
Sources of qualitative inputs include the following: • Federal advisory committees (such as the Board of Scientific Coun selors, the Mine Safety and Health Research Advisory Committee, and the National Advisory Committee on Occupational Safety and Health) • NORA research partners, initial NORA stakeholder meetings, later NORA team efforts (especially strategic research plans)
|
From page 161... ...
, the American So ciety of Safety Engineers, and the American College of Occupational and Environmental Medicine • Appropriate data from investigator-initiated extramural research funded by NIOSH Production Inputs For the research program under review, NIOSH should identify portions of the NIOSH intramural budget, staff, facilities, and management that play major roles in the research program. Production inputs should be described primarily in terms of intramural research projects, relevant extramural projects (particularly cooperative agreements and contracts)
|
From page 162... ...
Research activities in clude safety research, health-outcomes research, safety-design research, and safety systems research. Transfer activities include information dissemination, training, technical assistance, and education designed to translate research outputs into content and formats that are designed for application in the workplace.
|
From page 163... ...
hazards Safer-design and safety-systems research Intervention research: Control technologies Engineering controls and alternatives Administrative controls Personal protective equipment Work organization Community participation Policy (such as alternative approaches to targeting inspections) Design for safety Emergency preparedness and disaster response Diffusion and dissemination research: Training effectiveness Information-dissemination effectiveness Diffusion of technology Health-services and other research: Access to occupational health care Infrastructure -- delivery of occupational-health services, including international health and safety Socioeconomic consequences of work-related injuries and illnesses Worker compensation Technology-transfer and other transfer activities: Information dissemination Training programs Technical assistance
|
From page 164... ...
The EC should review project-level research and transfer activities (including surveillance activities) that have been completed, are in progress, or planned by the program under review.
|
From page 165... ...
Questions to Guide the Evaluation Committee in Assessing Transfer Activities 1. Is there a coherent planned program of transfer activities?
|
From page 166... ...
The NIOSH program should provide information on all relevant outputs of the program under review produced during the chosen period. Questions to Guide the Evaluation Committee 1.
|
From page 167... ...
Peer-reviewed publications by external researchers funded by NIOSH: Number of NIOSH-funded original research articles by external researchers Number of NIOSH-funded review articles by external researchers (including best-practices articles) Complete citation for each written report Complete copies of the "top five" articles Collaboration with other government or academic researchers NIOSH reports in the research program: Number of written reports Complete citation for each written report Complete copies of the "top five" reports Sponsored conferences and workshops: Number of sponsored conferences Number of sponsored workshops Description of conferences and workshops (title, date, sponsors, target audience, number of participants, and resulting products)
|
From page 168... ...
Complete citation Percentage of target audience that has used product 1, 5, and 10 years later Up to three examples of implementation in the field Identification of "top five" patents to date Miscellaneous: Any other important program outputs 3. Does the research program produce outputs that address high-priority areas?
|
From page 169... ...
Intermediate outcomes are important indicators of stakeholder response to NIOSH outputs. They reflect the impact of program activities and may lead to the desired end outcome of improved workplace safety and health.
|
From page 170... ...
Assessment Only a qualitative assessment of product development, usefulness, and impact is required at this point in the EC report. Some thought should be given to the relative value of intermediate outcomes, and the FC recommends applying the well accepted hierarchy-of-controls model.
|
From page 171... ...
If there is no direct evidence of improvements in health and safety, intermediate outcomes may be used as proxies for end outcomes in assessing impact as long as the EC qualifies its findings. The EC will describe the realized or potential benefits of the NIOSH program.
|
From page 172... ...
data on fatal occupational injuries (the Census of Fatal Occupational Injuries) and nonfatal occupational injuries and illnesses (the annual Survey of Occupational Injury and Illnesses)
|
From page 173... ...
To the extent that resources allocated to safety and health are limiting factors, the EC should explicitly assess NIOSH performance in the context of constraints. Questions to Guide the Evaluation Committee 1.
|
From page 174... ...
Assessment The EC may discuss other outcomes, including beneficial changes that have not yet occurred; social, economic, security, or environmental outcomes; and the impact that NIOSH has had on international occupational safety and health.
|
From page 175... ...
High impact assessments require the EC's judgment that the research program has contributed to outcomes; for example, outcomes have occurred earlier than they would have or are better than they would have been in the absence of the research program, TABLE 5 Background Context for Program Relevance and Impact Assess the following for each subprogram: 1. Relevance of current and recently completed research and transfer activities to objective improvements in workplace safety and health.
|
From page 176... ...
As discussed in previous sections, numerous factors can be con sidered in assessing relevance. The scoring criteria focus on two: the EC assessment BOX 2 Scoring Criteria for Relevance 5 = Research is in high-priority subject areas and NIOSH is significantly engaged in appro priate transfer activities for completed research projects/reported research results.
|
From page 177... ...
1 = Research activities and outputs do not result in or are NOT likely to have any application. of whether the program appropriately sets priorities among research needs and the EC assessment of how engaged the program is in appropriate transfer activities.
|
From page 178... ...
III.C. Assessment of NIOSH Process for Targeting Priority Research Needs and Committee Assessment of Emerging Issues The second charge to the EC is the assessment of the research program's effec tiveness in targeting new research and identifying emerging issues in occupational safety and health most relevant to future improvements in workplace protection.
|
From page 179... ...
2. Effectiveness in targeting research in fields most relevant to future improvements in occupational safety and health.
|
From page 180... ...
IV. EVALUATION COMMITTEE REPORT TEMPLATE Consistency and comparability among EC report formats is desirable, but the FC recognizes that each NIOSH research program is different and that each EC is independent.
|
From page 181... ...
The EC should assess the progress that the NIOSH program has made in targeting new research in occupational safety and health. The EC should assess whether the NIOSH pro gram has identified important emerging issues that appear especially important in terms of relevance to the mission of NIOSH.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.