The Personal Protective Technology Program at NIOSH Reviews of Research Programs of the National Institute for Occupational Safety and Health (2008) / Chapter Skim
Currently Skimming:
Appendix A: Framework for the Review of Research Programs of the National Institute for Occupational Safety and Health
Appendix A: Framework for the Review of Research Programs of the National Institute for Occupational Safety and Health
Pages 131-175
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 131... ...
This document is not a formal report of the National Academies -- rather, it is a framework proposed for use by multiple National Academies evaluation committees to review up to 15 National Institute for Occupational Safety and Health (NIOSH) research programs.
|
From page 132... ...
. • A new table was added to provide explicit guidance to evaluation com mittees on how to weigh differences in the observed levels of "research priority" and "engagement in appropriate transfer activities" in arriv ing at a single integer score for relevance (see Table 6)
|
From page 133... ...
Evaluation Committee Report Template Figure 1 The NIOSH operational plan presented as a logic model Figure 2 Flowchart for the evaluation of the NIOSH research program Table 1 NORA High-Priority Research by Category Table 2 Evaluation Committee Information Needs Table 3 Examples of NIOSH Program Research and Transfer Activities Table 4 Examples of Research-Program Outputs to Be Considered Table 5 Background Context for Program Relevance and Impact Table 6 Guidance for Weighting Research Priority and Engagement in Appropriate Transfer Activities in the Application of Relevance Score
|
From page 134... ...
134 The Personal Protective Technology Program at N I OSH Table 7 Targeting of New Research and Identification of Emerging Issues Box 1 The Evaluation Process Box 2 Scoring Criteria for Relevance Box 3 Scoring Criteria for Impact Box 4 Suggested Outline for Evaluation Committee Reports
|
From page 135... ...
Appendix A 135 ABBREVIATIONS AND ACRONYMS ABLES Adult Blood Lead Epidemiology and Surveillance AOEC Association of Occupational and Environmental Clinics BLS Bureau of Labor Statistics CDC Centers for Disease Control and Prevention CSTE Council of State and Territorial Epidemiologists DOD U.S. Department of Defense EC Evaluation Committee EPA Environmental Protection Agency FACE Fatality Assessment Control and Evaluation FC Framework Committee HHE Health Hazard Evaluation MSHA Mine Safety and Health Administration NIH National Institutes of Health NIOSH National Institute for Occupational Safety and Health NORA National Occupational Research Agenda NORA1 National Occupational Research Agenda 1996-2005 NORA2 National Occupational Research Agenda 2005-forward OSH Review Occupational Safety and Health Review Commission Commission OSHA Occupational Safety and Health Administration OSHAct Occupational Safety and Health Act of 1970 PART Performance Assessment Rating Tool PEL permissible exposure limit RFA request for applications SENSOR Sentinel Event Notification System of Occupational Risks
|
From page 136... ...
136 The Personal Protective Technology Program at N I OSH TMT tools, methods, or technologies USDA U.S. Department of Agriculture
|
From page 137... ...
This second version was finalized after the Framework Committee's May 30-31, 2007 meeting, based on feedback received on the framework from the first two independent evaluation committees, NIOSH leadership, and National Academies' staff, as well as discussions during an earlier FC meeting in April 2006. This document is not a report of the National Academies; rather, it presents the evaluation framework developed by the FC to guide and provide common structure for the reviews of as many as 15 NIOSH programs during a 5-year period by independent evaluation committees (ECs)
|
From page 138... ...
2. Assessment of the program's effectiveness in targeting new research areas and identifying emerging issues in occupational safety and health most relevant to future improvements in workplace protection.
|
From page 139... ...
The OSHAct identifies workplace safety and health as having high national priority and gives employers the responsibility for controlling hazards and preventing workplace injury and illness. The act creates an organizational framework for doing that, assigning complementary roles and responsibilities to employers and employees, the Occupational Safety and Health Administration (OSHA)
|
From page 140... ...
The emphasis is on moving research to practice in workplaces through sector-based partnerships. Figure 1 is the NIOSH operational plan, presented as a logic model, of the path from inputs to outcomes for each NIOSH research program.
|
From page 141... ...
Appendix A 141 TABLE 1 NORA High-Priority Research by Category Category Priority Research Area Disease and injury Allergic and irritant dermatitis Asthma and chronic obstructive pulmonary disease Fertility and pregnancy abnormalities Hearing loss Infectious diseases Low-back disorders Musculoskeletal disorders of upper extremities Trauma Work environment and workforce Emerging technologies Indoor environment Mixed exposures Organization of work Special populations at risk Research tools and approaches Cancer research methods Control technology and personal protective equipment Exposure-assessment methods Health-services research Intervention-effectiveness research Risk-assessment methods Social and economic consequences of workplace illness and injury Surveillance research methods I.D. Evaluation Committees' Information Needs Each NIOSH program under review will provide information to the relevant EC, including that outlined in Table 2.
|
From page 142... ...
142 New Figure A-1, bitmapped FIGURE 1 The NIOSH operational plan presented as a logic model.
|
From page 143... ...
• Interactions with stakeholders and with other NIOSH programs: ° The role of program research staff in NIOSH policy-setting, OSHA and MSHA standard setting, voluntary standard-setting and other government policy functions. ° Interactions and working relationships with other NIOSH programs.
|
From page 144... ...
° Adoption and use of control or personal protective technologies developed by NIOSH. ° Evidence of industry, employer, or worker behavioral changes in response to research outputs.
|
From page 145... ...
The major components to be evaluated are • major program challenges, • strategic goals and objectives, • inputs (such as budget, staff, facilities, the institute's research manage ment, the NIOSH Board of Scientific Counselors, the NORA process, and NORA work groups) , • activities (efforts by NIOSH staff, contractors, and grantees, such as hazard surveillance; surveillance for injury, illness, and biomarkers of effect; exposure-measurement research; safety-systems research; injury-prevention research; health-effects research; intervention 4The PART focuses on assessing program-level performance and is one of the measures of success of the budget and performance integration initiative of the president's management agenda (see CDC Occupational Safety and Health at http://www.whitehouse.gov/omb/budget/fy2006/pma/hhs.pdf)
|
From page 146... ...
146 bitmap image New Figure A-2.eps FIGURE 2 Flowchart for the evaluation of the NIOSH research program.
|
From page 147... ...
Examples of external factors are research activities of industry and other federal agencies and the political and regulatory environment. For purposes of this review, the results of inputs and external factors are the program research activities, outputs, and associated transfer activities that may result in intermediate outcomes and possibly end outcomes.
|
From page 148... ...
. Identification of external factors by an EC is essen tial because it provides the context for evaluation of the NIOSH program.
|
From page 149... ...
The ECs should also independently identify emerging workplace issues for which the NIOSH program under review should be prepared.
|
From page 150... ...
Evaluation of the impact of NIOSH research outputs on worker health and safety may require consideration of external factors that might impede or aide implementation, measurement, and so on. ECs might consider whether or not the following are true: • Regulatory end points are unachievable because of obstacles to regula tion or because of differing priorities of the regulatory agencies.
|
From page 151... ...
• The structure, in addition to the content, of the research program: A relevant research program is more than a set of unrelated research projects; it is an integrated program involving interrelated surveillance, research, and transfer activities. • Appropriate NIOSH consideration of stakeholder input.
|
From page 152... ...
, external advisory inputs, and its own expert judg ment. The EC will then be able to compare its own assessment of workplace chal lenges with the NIOSH program goals and objectives.
|
From page 153... ...
a. Did past program goals and objectives (research and dissemination and transfer activities)
|
From page 154... ...
Sources of qualitative inputs include the following: • Federal advisory committees (such as the Board of Scientific Coun selors, the Mine Safety and Health Research Advisory Committee, and the National Advisory Committee on Occupational Safety and Health) • NORA research partners, initial NORA stakeholder meetings, later NORA team efforts (especially strategic research plans)
|
From page 155... ...
, the American So ciety of Safety Engineers, and the American College of Occupational and Environmental Medicine • Appropriate data from investigator-initiated extramural research funded by NIOSH Production Inputs For the research program under review, NIOSH should identify portions of the NIOSH intramural budget, staff, facilities, and management that play major roles in the research program. Production inputs should be described primarily in terms of intramural research projects, relevant extramural projects (particularly cooperative agreements and contracts)
|
From page 156... ...
Depend ing on the scope of the program under review, activities may also be grouped by research-program objectives or subprograms. Conventional occupational safety and health research focuses appropriately on injury, illness, or death; on biomarkers of exposure; and on health effects of new technology, personal protective equipment, and regulations.
|
From page 157... ...
hazards Safer-design and safety-systems research Intervention research: Control technologies Engineering controls and alternatives Administrative controls Personal protective equipment Work organization Community participation Policy (such as alternative approaches to targeting inspections) Design for safety Emergency preparedness and disaster response Diffusion and dissemination research: Training effectiveness Information-dissemination effectiveness Diffusion of technology Health-services and other research: Access to occupational health care Infrastructure -- delivery of occupational-health services, including international health and safety Socioeconomic consequences of work-related injuries and illnesses Worker compensation Technology-transfer and other transfer activities: Information dissemination Training programs Technical assistance
|
From page 158... ...
The EC should review project-level research and transfer activities (including surveillance activities) that have been completed, are in progress, or planned by the program under review.
|
From page 159... ...
Assessment For this part of the assessment, the EC will provide a qualitative assessment that discusses relevance. This assessment should include consideration of the external factors identified in Section III.A that constrain choices of research projects and the relevance and effectiveness of transfer activities.
|
From page 160... ...
The NIOSH program should provide information on all relevant outputs of the program under review produced during the chosen period. Questions to Guide the Evaluation Committee 1.
|
From page 161... ...
Peer-reviewed publications by external researchers funded by NIOSH: Number of NIOSH-funded original research articles by external researchers Number of NIOSH-funded review articles by external researchers (including best-practices articles) Complete citation for each written report Complete copies of the "top five" articles Collaboration with other government or academic researchers NIOSH reports in the research program: Number of written reports Complete citation for each written report Complete copies of the "top five" reports Sponsored conferences and workshops: Number of sponsored conferences Number of sponsored workshops Description of conferences and workshops (title, date, sponsors, target audience, number of participants, and resulting products)
|
From page 162... ...
Complete citation Percentage of target audience that has used product 1, 5, and 10 years later Up to three examples of implementation in the field Identification of "top five" patents to date Miscellaneous: Any other important program outputs 3. Does the research program produce outputs that address high-priority areas?
|
From page 163... ...
Intermediate outcomes include the production by those outside of NIOSH of guidelines or regulations based wholly or partly on NIOSH research (products adopted as national or international public policy or as policy or guidelines by private organizations or industry) ; contributions to training and education programs sponsored by other organizations; use of publications or other materials by workers, industry, and occupational safety and health professionals in the field; and citations of NIOSH research by industrial and academic scientists.
|
From page 164... ...
Assessment Only a qualitative assessment of product development, usefulness, and impact is required at this point in the EC report. Some thought should be given to the relative value of intermediate outcomes, and the FC recommends applying the well accepted hierarchy-of-controls model.
|
From page 165... ...
If there is no direct evidence of improvements in health and safety, intermediate outcomes may be used as proxies for end outcomes in assessing impact as long as the EC qualifies its findings. The EC will describe the realized or potential benefits of the NIOSH program.
|
From page 166... ...
data on fatal occupational injuries (the Census of Fatal Occupational Injuries) and nonfatal occupational injuries and illnesses (the annual Survey of Occupational Injury and Illnesses)
|
From page 167... ...
Those data are available from multiple databases and studies of exposure. Apart from air monitoring, such measures of exposure as biohazard controls, reduction in requirements for use of personal protective equipment, and reduction in ergonomic risks are important.
|
From page 168... ...
Assessment The EC may discuss other outcomes, including beneficial changes that have not yet occurred; social, economic, security, or environmental outcomes; and the impact that NIOSH has had on international occupational safety and health.
|
From page 169... ...
High impact assessments require the EC's judgment that the research program has contributed to outcomes; for example, outcomes have occurred earlier than they would have or are better than they would have been in the absence of the research program, TABLE 5 Background Context for Program Relevance and Impact Assess the following for each subprogram: 1. Relevance of current and recently completed research and transfer activities to objective improvements in workplace safety and health.
|
From page 170... ...
As discussed in previous sections, numerous factors can be con sidered in assessing relevance. The scoring criteria focus on two: the EC assessment BOX 2 Scoring Criteria for Relevance 5 = Research is in high-priority subject areas and NIOSH is significantly engaged in appro priate transfer activities for completed research projects/reported research results.
|
From page 171... ...
1 = Research activities and outputs do not result in or are NOT likely to have any application. of whether the program appropriately sets priorities among research needs and the EC assessment of how engaged the program is in appropriate transfer activities.
|
From page 172... ...
III.C. Assessment of NIOSH Process for Targeting Priority Research Needs and Committee Assessment of Emerging Issues The second charge to the EC is the assessment of the research program's effec tiveness in targeting new research and identifying emerging issues in occupational safety and health most relevant to future improvements in workplace protection.
|
From page 173... ...
2. Effectiveness in targeting research in fields most relevant to future improvements in occupational safety and health.
|
From page 174... ...
IV. EVALUATION COMMITTEE REPORT TEMPLATE Consistency and comparability among EC report formats is desirable, but the FC recognizes that each NIOSH research program is different and that each EC is independent.
|
From page 175... ...
The EC should assess the progress that the NIOSH program has made in targeting new research in occupational safety and health. The EC should assess whether the NIOSH pro gram has identified important emerging issues that appear especially important in terms of relevance to the mission of NIOSH.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.