Hearing Loss Research at NIOSH Reviews of Research Programs of the National Institute for Occupational Safety and Health (2006) / Chapter Skim
Currently Skimming:
Appendix A Framework for the Review of Research Programs of the National Institute for Occupational Safety and Health
Appendix A Framework for the Review of Research Programs of the National Institute for Occupational Safety and Health
Pages 125-167
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 125... ...
1Members of the National Academies' Committee for the Review of NIOSH Research Programs include: David Wegman (Chair; University of Massachusetts Lowell School of Health and Environment) , William Bunn, III (International Truck and Engine Corporation)
|
From page 126... ...
126 H EA R I N GL OS SR E S E A R C H A TN I O S H relevance on the basis of responses received from evaluation committee members, NIOSH, stakeholders, and the general public before and during the course of the assessments conducted by independent evaluation committees of up to 15 re search programs and health-outcomes programs. All public comments submitted to the Committee for the Review of NIOSH Research Programs will be included in the Public Access File for this study as provided in the National Academies Terms of Use (www.nationalacademies.org/ legal/terms.html)
|
From page 127... ...
Framework Committee Final Report Figure 1 The NIOSH operational plan presented as a logic model Figure 2 Flow chart for the evaluation of the NIOSH research program Table 1 NORA High-priority Research Areas by Category Table 2 Examples of NIOSH Program Research and Transfer Activities Table 3 Examples of a Variety of Scientific Information Outputs Table 4 Evaluation Committee Worksheet to Assess Research Programs and Subprograms
|
From page 128... ...
128 H E A R I N GL OS SRE S E A R C H A TN I O S H ACRONYMS ABLES Adult Blood Lead Epidemiology and Surveillance ACOEM American College of Occupational and Environmental Medicine AOEC Association of Occupational and Environmental Clinics BLS Bureau of Labor Statistics CDC Centers for Disease Control and Prevention EC Evaluation Committee FACE Fatality Assessment Control and Evaluation FC Framework Committee HHE Health Hazard Evaluations MSHA Mine Safety and Health Administration NEISS National Electronic Injury Surveillance System NIOSH National Institute for Occupational Safety and Health NORA National Occupational Research Agenda NORA1 National Occupational Research Agenda 1996-2005 NORA2 National Occupational Research Agenda 2005-Forward OSHA Occupational Safety and Health Administration OSHAct Occupational Safety and Health Act of 1970 OSH Review Occupational Safety and Health Review Commission Commission PART Performance Assessment Rating Tool PEL Permissible Exposure Limits SENSOR Sentinel Event Notification System of Occupational Risks TMT Tools, Methods, or Technologies
|
From page 129... ...
The National Academies appointed a committee of 14 members, including persons with expertise in occupational medicine and health, industrial health and safety, industrial hygiene, epidemiology, civil and mining engineering, sociology, program evaluation, communication, and toxicology; representatives of industry and of the workforce; and a scientist experienced in international occupationalhealth issues. The Committee on the Review of NIOSH Research Programs, referred to as the Framework Committee (FC)
|
From page 130... ...
3. Evaluation committee identification of significant emerging research ar eas which appear especially important in terms of their relevance to the mission of NIOSH.
|
From page 131... ...
. NORA was intended not only for NIOSH but for the entire occupational health community.
|
From page 132... ...
Figure 1 is the NIOSH operational plan presented as a logic model3 of the path from inputs to outcomes for each NIOSH research program. The FC adapted the model to develop its framework.
|
From page 133... ...
133 and in vementso ety saf health orkplaces Outcomes Impr in w Injuries , , , eesy ersy ,y and customers industr Emplo emplo educators regulators Final (implementation) Illness Outputs , and , , , , and ready and/or amsr Outputs etk media Pilot mar technologies ainingtr education prog guidance releases regulations standards aluationve ork-Related Activities W , and, , er and customers SH, vent mation)
|
From page 134... ...
The committees will conduct appropriate information-gathering sessions to obtain information from the sponsor (a NIOSH research program) , stakeholders affected directly by the NIOSH research, and relevant independent parties.
|
From page 135... ...
Interventions, recommendations, and information-dissemination and technology-transfer activities designed to get research findings used to improve occupational safety and health. Outcomes of research, alerts, standard-setting, investigations, and consultations; for example -- documented reductions in risk after program-supported interventions, employer and industry behavior changes made in response to research outputs, and worker behavior changes in response to research outputs.
|
From page 136... ...
. Drawing on the program logic model, the flow chart, and EC members' exper tise, the ECs will delineate important determinants of a NIOSH research program's agenda and the consequences of the NIOSH research activity.
|
From page 137... ...
137 other , , , ts , and and research, patents, research, C w technology activities and eillancev ect repor, tools, vices er vie Activities D w guidelines, Sur entionv Outputs Re Assessment of ansftr vie of lications health-eff inter research, Re Assessment databases health-ser Pub methods recommendations , and and ucture and data, inputs a- acilitiesf str B w eillancev intr funding, al Inputs ysical sur vie entionv ur , of eholder ph am s s Re Assessment inter stak xtre of Production: management and/or Planning: staffing, eholders actorF and impact, use actorF stak guidelines E w ted yb vie policy Outcomes lic Re Intermediate aining/education,tr kaging program. External Assessment Pub External and self-repor implemented , EC of repac NIOSH Driving ogram goals of select goals of NIOSH, with challenges research A ysis Goals Pr to yb of amr ision Anal prog aluation process NIOSH ev selected Current compar Assessment Strategic Objectives assessment the of of , ies evaluation es EC and injur to yb F w xposurese, the goals kplacesro NIOSH vie Outcomes w for ogram- assessment in Pr with area Re Reduced members Challeng Assessment End illnesses chart amr EC Major by prog Flow Area compare Determined 2 Independent FIGURE
|
From page 138... ...
. For purposes of this review, the results of inputs and external factors are the program research activities, outputs, and associated transfer activities that may result in intermediate outcomes and possibly eventual end outcomes.
|
From page 139... ...
10. Identify significant emerging research areas (Section III.C)
|
From page 140... ...
In some cases, they may be quantifiable. It is possible, however, to evaluate the impact of a NIOSH research program whether the outcomes are intermediate outcomes or end outcomes.
|
From page 141... ...
EVALUATION OF NIOSH RESEARCH PROGRAMS -- THE PROCESS III.A. Analysis of External Factors Relevant to the NIOSH Research Program As depicted in the logic model (Figure 1)
|
From page 142... ...
· NIOSH resources are inadequate to tackle the key questions. Evaluation of the impact of NIOSH research outputs on outcomes may re quire consideration of external factors that might have impeded or aided imple mentation, measurement, and so on.
|
From page 143... ...
III.B. Evaluating NIOSH Research Programs (Addressing Charges 1 and 2)
|
From page 144... ...
The EC should recognize that NIOSH research priorities may be strategic rather than based on the assessment of the state of knowledge. Some aspects of the NIOSH research program's strategic goals and objectives would have been already subjected to evaluation by internal or external bodies.
|
From page 145... ...
Assessment The EC will provide a qualitative assessment discussing the relevance of the area's goals, objectives, and strategies as related to the research program's major challenges and emerging problems.
|
From page 146... ...
Planning Inputs Planning inputs can be qualitative or quantitative. Sources of qualitative in puts include · Federal Advisory Committee Act panels (Board of Scientific Counselors, Mine Safety and Health Research Advisory Committee, National Advisory Committee on Occupational Safety and Health, and so on)
|
From page 147... ...
Production Inputs For each research program under review, NIOSH should specify an identifiable portion of the NIOSH intramural budget, staff, facilities, and management that has been allocated by divisions and offices that play a major role in the research program. Production inputs should be described primarily in terms of intramural research projects and staff, relevant extramural projects (particularly cooperative agreements and contracts)
|
From page 148... ...
. Transfer activities include information dissemination, training, technical assistance, and education designed to translate research outputs into content and formats designed for application in the workplace to produce improvements in occupational safety and health.
|
From page 149... ...
Diffusion and dissemination research Training effectiveness Information-dissemination effectiveness Diffusion of technology Health-services and other research Access to occupational health care Infrastructure research -- delivery of occupational-health services, including international health and safety Socioeconomic consequences of work-related injuries and illnesses Worker compensation Technology-transfer and other transfer activities Information dissemination Training programs The ECs should review the list of research and transfer activities (projects) for the research program under review that have been completed, are in progress, or have been planned.
|
From page 150... ...
Questions to Guide the Evaluation Committee in Assessing Transfer Activities 1. Is there a coherent planned program of transfer activities?
|
From page 151... ...
As shown in Figure 1, research inputs and activities lead to outputs. An output is a direct product of a NIOSH research program that is logically related to the achievement of desirable and intended outcomes.
|
From page 152... ...
Peer-reviewed publications by external researchers funded by NIOSH Total number of NIOSH-funded original research articles by external researchers Total number of NIOSH-funded review articles by external researchers (including best-practices articles) Complete citation for each written report Complete copies of the "top five" articles Collaboration with other government or academic researchers NIOSH reports in the research program Total number of written reports Complete citation for each written report Complete copies of the "top five" reports Sponsored conferences and workshops Total number of sponsored conferences Total number of sponsored workshops For each sponsored conference or workshop, describe: Title, date, and location Partial vs complete sponsorship (if partial, who were cosponsors?
|
From page 153... ...
Complete citation Percentage of target audience that has used product 1, 5, and 10 years later Up to three examples of implementation in the field Identification of "top five" patents to date Miscellaneous Any other important program outputs The EC should ask NIOSH to provide information on all relevant outputs for the specific program for the chosen time period. Questions to Guide the Evaluation Committee 1.
|
From page 154... ...
; attendance at training and education programs sponsored by other organizations; use of publi cations by workers, industry, and occupational safety and health professionals in the field; and citations of NIOSH research by industrial and academic scientists. More difficult-to-collect intermediate outcomes that may be valid indicators of quality or utility include self-report measures by users and relevant nonusers of NIOSH outputs.
|
From page 155... ...
4. Has the program resulted in new personal protective equipment that is feasible for use or has been adopted in the workplace to reduce risk factors or exposures?
|
From page 156... ...
End outcomes are defined by measures of health and safety and of impact on process and programs. The FC recognizes that a major challenge in assessing the causal relationship between NIOSH research and specific occupational health and safety outcomes is that NIOSH does not have direct responsibility or authority for implementing its research findings in the workplace.
|
From page 157... ...
Sources of quantitative data include · Bureau of Labor Statistics (BLS) data on fatal occupational injuries (Cen sus of Fatal Occupational Injuries)
|
From page 158... ...
4. To what degree has the NIOSH program or subprogram been responsible for improvements in occupational safety or health?
|
From page 159... ...
Assessment Evaluation by the EC may consist of a discussion of other outcomes, including positive changes that have not yet occurred; other social, economic, security, or environmental outcomes; and the impact that NIOSH has had on international occupational safety and health. It might also consider the incorporation of international research results into the NIOSH program of knowledge transfer for industry sectors.
|
From page 160... ...
... n 1.3 Contributions of NIOSH research Research and transfer activities to changes in work-related practices Transfer 1.4 Contributions of NIOSH research Research and transfer activities to reductions in workplace exposure, Transfer illness, or injuries 1.5 Evidence of external factors Research preventing application of NIOSH research results Transfer 1.6 Contribution of NIOSH research Research to enhancement of capacity in government or other research Transfer institutions 1.7 Contributions of NIOSH research Research to productivity, security, or environmental quality (beneficial Transfer side effects)
|
From page 161... ...
and charge 2 (relevance) , the ECs will use their expert judgment, their responses to the questions in Table 4, and any other appropriate information to arrive at one overall rating for the impact of the research program and one for its relevance to the improvement of occupational safety and health.
|
From page 162... ...
Rating of Impact 5 = Research program has made a major contribution to worker health and safety on the basis of end outcomes or well-accepted intermediate outcomes. 4 = Research program has made a moderate contribution on the basis of end outcomes or well-accepted intermediate outcomes; research program gen erated important new knowledge and is engaged in transfer activities, but well-accepted intermediate outcomes or end outcomes have not been documented.
|
From page 163... ...
3 = Research focuses on lesser priorities and is loosely or only indirectly con nected to workplace protection; NIOSH is not significantly involved in transfer activities. 2 = Research program is not well integrated or well focused on priorities and is not clearly connected to workplace protection and inadequately connected to transfer activities.
|
From page 164... ...
In addition, appropriate federal advisory committees and other stakeholder groups should be consulted to provide qualitative information. The EC members should use their expert judgment both to evaluate what NIOSH has identified as emerging research targets (charge 2)
|
From page 165... ...
F Review of intermediate outcomes and causal impact: Describes assess ment of the intermediate outcomes and the causal attribution to NIOSH; includes the likely impacts and recent outcomes in the assessment.
|
From page 166... ...
: The EC should assess the progress that the NIOSH program has made in targeting new research in the fields of occupational safety and health. There should be a discussion of the assessment process and results.
|
From page 167... ...
E Coordinating NIOSH research activity with respect to other public and private research programs.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.