In-Service Performance Evaluation: Guidelines for the Assembly and Analysis of Data (2022)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2022 N A T I O N A L C O O P E R A T I V E H I G H W A Y R E S E A R C H P R O G R A M NCHRP RESEARCH REPORT 1010 In-Service Performance Evaluation GUIDELINES FOR THE ASSEMBLY AND ANALYSIS OF DATA Christine E. Carrigan a n d Malcolm H. Ray RoadSafe LLC Canton, ME Subscriber Categories Operations and Trafc Management • Safety and Human Factors Research sponsored by the American Association of State Highway and Transportation Ofcials in cooperation with the Federal Highway Administration

Published research reports of the NATIONAL COOPERATIVE HIGHWAY RESEARCH PROGRAM are available from Transportation Research Board Business Office 500 Fifth Street, NW Washington, DC 20001 and can be ordered through the Internet by going to https://www.mytrb.org/MyTRB/Store/default.aspx Printed in the United States of America NCHRP RESEARCH REPORT 1010 Project 22-33 ISSN 2572-3766 (Print) ISSN 2572-3774 (Online) ISBN 978-0-309-68735-5 Library of Congress Control Number 2022945249 © 2022 by the National Academy of Sciences. National Academies of Sciences, Engineering, and Medicine and the graphical logo are trade- marks of the National Academy of Sciences. All rights reserved. COPYRIGHT INFORMATION Authors herein are responsible for the authenticity of their materials and for obtaining written permissions from publishers or persons who own the copyright to any previously published or copyrighted material used herein. Cooperative Research Programs (CRP) grants permission to reproduce material in this publication for classroom and not-for-profit purposes. Permission is given with the understanding that none of the material will be used to imply TRB, AASHTO, APTA, FAA, FHWA, FTA, GHSA, or NHTSA endorsement of a particular product, method, or practice. It is expected that those reproducing the material in this document for educational and not-for-profit uses will give appropriate acknowledgment of the source of any reprinted or reproduced material. For other uses of the material, request permission from CRP. NOTICE The research report was reviewed by the technical panel and accepted for publication according to procedures established and overseen by the Transportation Research Board and approved by the National Academies of Sciences, Engineering, and Medicine. The opinions and conclusions expressed or implied in this report are those of the researchers who performed the research and are not necessarily those of the Transportation Research Board; the National Academies of Sciences, Engineering, and Medicine; the FHWA; or the program sponsors. The Transportation Research Board does not develop, issue, or publish standards or speci- fications. The Transportation Research Board manages applied research projects which provide the scientific foundation that may be used by Transportation Research Board sponsors, industry associations, or other organizations as the basis for revised practices, procedures, or specifications. The Transportation Research Board; the National Academies of Sciences, Engineering, and Medicine; and the sponsors of the National Cooperative Highway Research Program do not endorse products or manufacturers. Trade or manufacturers’ names or logos appear herein solely because they are considered essential to the object of the report. NATIONAL COOPERATIVE HIGHWAY RESEARCH PROGRAM Systematic, well-designed, and implementable research is the most effective way to solve many problems facing state departments of transportation (DOTs) administrators and engineers. Often, highway problems are of local or regional interest and can best be studied by state DOTs individually or in cooperation with their state universities and others. However, the accelerating growth of highway transporta- tion results in increasingly complex problems of wide interest to high- way authorities. These problems are best studied through a coordinated program of cooperative research. Recognizing this need, the leadership of the American Association of State Highway and Transportation Officials (AASHTO) in 1962 ini- tiated an objective national highway research program using modern scientific techniques—the National Cooperative Highway Research Program (NCHRP). NCHRP is supported on a continuing basis by funds from participating member states of AASHTO and receives the full cooperation and support of the Federal Highway Administration (FHWA), United States Department of Transportation, under Agree- ment No. 693JJ31950003. The Transportation Research Board (TRB) of the National Academies of Sciences, Engineering, and Medicine was requested by AASHTO to administer the research program because of TRB’s recognized objectivity and understanding of modern research practices. TRB is uniquely suited for this purpose for many reasons: TRB maintains an extensive com- mittee structure from which authorities on any highway transportation subject may be drawn; TRB possesses avenues of communications and cooperation with federal, state, and local governmental agencies, univer- sities, and industry; TRB’s relationship to the National Academies is an insurance of objectivity; and TRB maintains a full-time staff of special- ists in highway transportation matters to bring the findings of research directly to those in a position to use them. The program is developed on the basis of research needs iden- tified by chief administrators and other staff of the highway and transportation departments, by committees of AASHTO, and by the FHWA. Topics of the highest merit are selected by the AASHTO Special Committee on Research and Innovation (R&I), and each year R&I’s recommendations are proposed to the AASHTO Board of Direc- tors and the National Academies. Research projects to address these topics are defined by NCHRP, and qualified research agencies are selected from submitted proposals. Administration and surveillance of research contracts are the responsibilities of the National Academies and TRB. The needs for highway research are many, and NCHRP can make significant contributions to solving highway transportation problems of mutual concern to many responsible groups. The program, however, is intended to complement, rather than to substitute for or duplicate, other highway research programs.

e National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, non- governmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president. e National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president. e National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president. e three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. e National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine. Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org. e Transportation Research Board is one of seven major programs of the National Academies of Sciences, Engineering, and Medicine. e mission of the Transportation Research Board is to provide leadership in transportation improvements and innovation through trusted, timely, impartial, and evidence-based information exchange, research, and advice regarding all modes of transportation. e Board’s varied activities annually engage about 8,000 engineers, scientists, and other transportation researchers and practitioners from the public and private sectors and academia, all of whom contribute their expertise in the public interest. e program is supported by state transportation departments, federal agencies including the component administrations of the U.S. Department of Transportation, and other organizations and individuals interested in the development of transportation. Learn more about the Transportation Research Board at www.TRB.org.

C O O P E R A T I V E R E S E A R C H P R O G R A M S AUTHOR ACKNOWLEDGMENTS The ISPE Data Set and Analysis Template was developed by Ethan Ray of RoadSafe LLC to aid in the calculations of each evaluation measure shown in this report and to support the documentation of an in-service performance evaluation (ISPE). A resource hub to support the exchange of ISPE findings was developed under this research effort by Archie Ray of RoadSafe LLC. Persaud and Lyon Inc. provided comments on the first draft of this report. The NCHRP project panel reviewed the alpha version of the report and provided suggestions for improvements. A pilot test of the research products (i.e., the beta version of the guidelines in this report, the ISPE Data Set and Analysis Template, and the ISPE Resource Hub) was conducted by personnel from the following transportation agencies: Arizona, Connecticut, Iowa, Maine, New Hampshire, South Carolina, Tennessee, Utah, and Washington. This report, the ISPE Data Set and Analysis Template, and the ISPE Resource Hub reflect the suggestions for improvements received from each of these groups. The research team is grateful to each of these groups for their invalu- able insight and suggestions for improvements. CRP STAFF FOR NCHRP RESEARCH REPORT 1010 Christopher J. Hedges, Director, Cooperative Research Programs Waseem Dekelbab, Deputy Director, Cooperative Research Programs, and Manager, National Cooperative Highway Research Program Sid Mohan, Associate Program Manager, Implementation and Technology Transfer, National Cooperative Highway Research Program David M. Jared, Senior Program Officer Traci Caldwell, Senior Program Assistant Natalie Barnes, Director of Publications Heather DiAngelis, Associate Director of Publications Janet M. McNaughton, Senior Editor NCHRP PROJECT 22-33 PANEL Field of Design—Area of Vehicle Barrier Systems Erik O. Emerson, Wisconsin Department of Transportation, Madison, WI (Chair) D. Lance Bullard, Jr., Texas A&M Transportation Institute, College Station, TX Keith A. Cota, Epsom, NH Randy Hiatt, California Department of Transportation (Caltrans), Sacramento, CA Chris Lindsey, Texas Department of Transportation, Austin, TX Kurt Sielbach, Washington State Department of Transportation, Olympia, WA Jason J. Siwula, Kentucky Transportation Cabinet, Frankfort, KY Ana Maria Eigen, FHWA Liaison Kelly K. Hardy, AASHTO Liaison Stephen F. Maher, TRB Liaison

NCHRP Research Report 1010: In-Service Performance Evaluation: Guidelines for the Assembly and Analysis of Data presents guidelines for conducting in-service performance evaluations (ISPEs) of both permanent and temporary safety features. In an effort to develop a unified format and nationally compatible ISPE methodology, various individual states’ ISPE databases were employed to leverage available data where possible. The guidelines will be of interest to state departments of transportation (DOTs) seeking to perform both individual and multi- state ISPEs on roadside safety hardware currently in service. Roadside safety hardware is installed along roadways to reduce the risk of serious and fatal injuries to motorists from roadway departures. Impact performance criteria for roadside hardware based on full-scale crash testing are detailed in the AASHTO Manual for Assessing Safety Hardware (MASH). MASH tests, however, represent only a fraction of the site condi- tions motorists may encounter and the crash types they may experience. For more than 40 years, in-service evaluation has been recommended as the final step in evaluating roadside hard- ware, in addition to MASH testing. NCHRP Report 490: In-Service Performance of Traffic Barriers provided detailed procedures and guidelines for performing in-service evaluations of roadside safety hardware. Although the roadside safety community has agreed for many years on the importance of in-service evaluations, and resources on these evaluations were available, relatively few ISPEs have been completed. The role of the evaluations in making decisions about roadside safety has not been well defined to date, and the evaluations that have been conducted were by state DOTs working independently. The purpose of this study was to establish a standardized methodology for ISPEs based on assessing the field performance of safety features, assembling and evaluating in-service data, uniformly documenting ISPEs, and interpreting and applying ISPE results. Results of this research will enable state DOTs to set and evaluate their level of safety risk and use quantitative information in the decision-making process. Under NCHRP Project 22-33, “Multi-State In-Service Performance Evaluations of Roadside Safety Hardware,” RoadSafe LLC was asked to (1) develop a unified format and nationally compatible ISPE methodology employing various individual state database parameters; (2) demonstrate and verify the developed ISPE methodology in a pilot effort, including a workshop; and (3) identify common issues and challenges of conducting a multistate ISPE on selected roadside safety hardware devices. In addition to NCHRP Research Report 1010, four other deliverables are available on the website of the National Academies Press (www.nap.edu) by searching for NCHRP Research F O R E W O R D By David M. Jared Staff Officer Transportation Research Board

Report 1010: In-Service Performance Evaluation: Guidelines for the Assembly and Analysis of Data. These deliverables are as follows: • NCHRP Web-Only Document 332: Multi-State In-Service Performance Evaluations of Roadside Safety Hardware, which is the research agency’s report documenting the develop- ment of the guidelines and the entire research effort; • The ISPE Data Set and Analysis Template, a spreadsheet tool to aid in the calculations of each evaluation measure shown in this report and to support the documentation of an ISPE; • A plan that identifies mechanisms and channels for implementing this research; and • A presentation introducing NCHRP Research Report 1010.

1 Chapter 1 Introduction 1 1.1 Purpose 2 1.2 Objectives 3 1.3 Underlying Philosophy 4 1.4 Organization 6 Chapter 2 ISPE Methodology and Scope 6 2.1 General 6 2.2 Safety Features Addressed 7 2.3 Methodology 8 2.4 Summary: ISPE Methodology and Scope 9 Chapter 3 ISPE Study Design 9 3.1 Identify Safety Feature Under Evaluation 10 3.2 Assemble Available Data 10 3.3 Compile the ISPE Data 11 3.3.1 Safety Feature Under Evaluation (SFUE) 11 3.3.2 Crash Record Number (CRN) 12 3.3.3 Crash Date (CRASH_DATE) 12 3.3.4 Number of Units (TOTAL_UNITS) 12 3.3.5 Maximum Severity of Crash (MAX_SEV) 13 3.3.6 Vehicle Type (VEH_TYPE) 15 3.3.7 Posted Speed Limit (SPEED_LIMIT) 15 3.3.8 Harmful Event Post Impact with the Safety Feature Under Evaluation (PostHE) 16 3.3.9 Most Harmful Event in Crash with the Safety Feature Under Evaluation (MHE) 16 3.3.10 First Harmful Event in Crash with the Safety Feature Under Evaluation (FHE) 16 3.3.11 Any Harmful Event in Crash with the Safety Feature Under Evaluation (AHE) 17 3.3.12 First and Only Harmful Event in Crash with the Safety Feature Under Evaluation (FOHE) 17 3.3.13 Breach of Safety Feature (BREACH) 17 3.3.14 Predictable Breakaway (BREAK) 18 3.3.15 Controlled Penetration, Redirection, or Stop (PRS) 18 3.3.16 Penetration of Safety Feature into Occupant Compartment (PEN) 18 3.3.17 Initial Contact Point (ICP) 19 3.3.18 Subfeature Name (NAME) 20 3.3.19 Installation Inspected and Documented (INSTALL) 20 3.3.20 Maintenance Inspection (MAINT) 20 3.3.21 Compilation into Worksheet 22 3.3.22 Summary: ISPE Data Compilation 22 3.4 Define Study Period and Region C O N T E N T S

26 Chapter 4 Evaluation Measures 26 4.1 General 26 4.2 Performance Outcome 27 4.3 Evaluation Measures 27 4.3.1 Evaluation Measures for Structural Adequacy 28 4.3.2 Evaluation Measures for Occupant Risk 29 4.3.3 Evaluation Measures for Assessing Vehicle Trajectory and Impact Orientation 30 4.4 Definition of Variables Used in Evaluation 32 4.4.1 Determination of the Confidence Interval for the Effect Size 32 4.4.2 Determination of the Confidence Interval for Point Estimates 32 4.5 Summary: Evaluation Measures 34 Chapter 5 Assessing, Interpreting, and Implementing Results 34 5.1 Conditional Assessment of ISPE Data Set 34 5.1.1 Condition 1: Statistical Power and Sample Bias 36 5.1.2 Conditions 2 and 3: Construction and Maintenance Practices 37 5.2 Performance Assessment of Safety Feature Under Evaluation 37 5.2.1 Performance Assessment Level 1: Analysis with No Limitations 38 5.2.2 Performance Assessment Level 2: Analysis Limited by Vehicle Type 38 5.2.3 Performance Assessment Level 3: Analysis Limited by Posted Speed Limit 39 5.2.4 Performance Assessment Level 4: Analysis Limited by Vehicle Type and Posted Speed Limit 39 5.3 Interpreting Results 41 5.4 Implementing Results 42 Chapter 6 Sharing and Comparing Results 44 Chapter 7 ISPE Documentation 44 7.1 General Reporting Recommendations 45 7.2 ISPE Report Contents 46 7.3 Meta-Analysis of ISPE Reports 46 7.4 Presentation of Data and Findings 46 7.4.1 ISPE Data Set Case Reduction Sheet 47 7.4.2 Preparation of ISPE Data Set 50 7.4.3 ISPE Summary of Data and Results Sheets 50 7.5 Electronic Data 55 References 56 Appendix A Example ISPE Summary of Data and Results Sheets for Each Evaluation Measure 68 Appendix B Example ISPE Summary of Data and Results Sheets for NAME Note: Photographs, figures, and tables in this report may have been converted from color to grayscale for printing. The electronic version of the report (posted on the web at www.nap.edu) retains the color versions.