Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
N A T I O N A L C O O P E R A T I V E H I G H W A Y R E S E A R C H P R O G R A M NCHRP REPORT 727 Effective Experiment Design and Data Analysis in Transportation Research Richard W. Lyles M. Abrar Siddiqui Neeraj Buch William C. Taylor Syed Waqar Haider Dennis C. Gilliland Bruce W. Pigozzi Michigan State UniverSity East Lansing, Michigan i n a s s o c i a t i o n w i t h Joseph E. Hummer north carolina State UniverSity Raleigh, North Carolina Subscriber Category Research TRANSPORTAT ION RESEARCH BOARD WASHINGTON, D.C. 2012 www.TRB.org Research sponsored by the American Association of State Highway and Transportation Officials in cooperation with the Federal Highway Administration
NATIONAL COOPERATIVE HIGHWAY RESEARCH PROGRAM Systematic, well-designed research provides the most effective approach to the solution of many problems facing highway administrators and engineers. Often, highway problems are of local interest and can best be studied by highway departments individually or in cooperation with their state universities and others. However, the accelerating growth of highway transportation develops increasingly complex problems of wide interest to highway authorities. These problems are best studied through a coordinated program of cooperative research. In recognition of these needs, the highway administrators of the American Association of State Highway and Transportation Officials initiated in 1962 an objective national highway research program employing modern scientific techniques. This program is supported on a continuing basis by funds from participating member states of the Association and it receives the full cooperation and support of the Federal Highway Administration, United States Department of Transportation. The Transportation Research Board of the National Academies was requested by the Association to administer the research program because of the Boardâs recognized objectivity and understanding of modern research practices. The Board is uniquely suited for this purpose as it maintains an extensive committee structure from which authorities on any highway transportation subject may be drawn; it possesses avenues of communications and cooperation with federal, state and local governmental agencies, universities, and industry; its relationship to the National Research Council is an insurance of objectivity; it maintains a full-time research correlation staff of specialists in highway transportation matters to bring the findings of research directly to those who are in a position to use them. The program is developed on the basis of research needs identified by chief administrators of the highway and transportation departments and by committees of AASHTO. Each year, specific areas of research needs to be included in the program are proposed to the National Research Council and the Board by the American Association of State Highway and Transportation Officials. Research projects to fulfill these needs are defined by the Board, and qualified research agencies are selected from those that have submitted proposals. Administration and surveillance of research contracts are the responsibilities of the National Research Council and the Transportation Research Board. The needs for highway research are many, and the National Cooperative Highway Research Program can make significant contributions to the solution of highway transportation problems of mutual concern to many responsible groups. The program, however, is intended to complement rather than to substitute for or duplicate other highway research programs. Published reports of the NATIONAL COOPERATIVE HIGHWAY RESEARCH PROGRAM are available from: Transportation Research Board Business Office 500 Fifth Street, NW Washington, DC 20001 and can be ordered through the Internet at: http://www.national-academies.org/trb/bookstore Printed in the United States of America NCHRP REPORT 727 Project 20-71 ISSN 0077-5614 ISBN 978-0-309-25849-4 Library of Congress Control Number 2012945556 © 2012 National Academy of Sciences. All rights reserved. COPYRIGHT INFORMATION Authors herein are responsible for the authenticity of their materials and for obtaining written permissions from publishers or persons who own the copyright to any previously published or copyrighted material used herein. Cooperative Research Programs (CRP) grants permission to reproduce material in this publication for classroom and not-for-profit purposes. Permission is given with the understanding that none of the material will be used to imply TRB, AASHTO, FAA, FHWA, FMCSA, FTA, or Transit Development Corporation endorsement of a particular product, method, or practice. It is expected that those reproducing the material in this document for educational and not-for-profit uses will give appropriate acknowledgment of the source of any reprinted or reproduced material. For other uses of the material, request permission from CRP. NOTICE The project that is the subject of this report was a part of the National Cooperative Highway Research Program, conducted by the Transportation Research Board with the approval of the Governing Board of the National Research Council. The members of the technical panel selected to monitor this project and to review this report were chosen for their special competencies and with regard for appropriate balance. The report was reviewed by the technical panel and accepted for publication according to procedures established and overseen by the Transportation Research Board and approved by the Governing Board of the National Research Council. The opinions and conclusions expressed or implied in this report are those of the researchers who performed the research and are not necessarily those of the Transportation Research Board, the National Research Council, or the program sponsors. The Transportation Research Board of the National Academies, the National Research Council, and the sponsors of the National Cooperative Highway Research Program do not endorse products or manufacturers. Trade or manufacturersâ names appear herein solely because they are considered essential to the object of the report.
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. On the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, on its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academyâs purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. The Transportation Research Board is one of six major divisions of the National Research Council. The mission of the Transporta- tion Research Board is to provide leadership in transportation innovation and progress through research and information exchange, conducted within a setting that is objective, interdisciplinary, and multimodal. The Boardâs varied activities annually engage about 7,000 engineers, scientists, and other transportation researchers and practitioners from the public and private sectors and academia, all of whom contribute their expertise in the public interest. The program is supported by state transportation departments, federal agencies including the component administrations of the U.S. Department of Transportation, and other organizations and individu- als interested in the development of transportation. www.TRB.org www.national-academies.org
C O O P E R A T I V E R E S E A R C H P R O G R A M S CRP STAFF FOR NCHRP REPORT 727 Christopher W. Jenks, Director, Cooperative Research Programs Crawford F. Jencks, Deputy Director, Cooperative Research Programs B. Ray Derr, Senior Program Officer Andréa Harrell, Senior Program Assistant Eileen P. Delaney, Director of Publications Sharon Lamberton, Assistant Editor NCHRP PROJECT 20-71 PANEL Field of Special ProjectsâArea of Research Donald L. Dean, California DOT, Sacramento, CA (Chair) Deniz Sandhu, New York State Education Department, formerly New York State DOT, Albany, NY Montasir M. Abbas, Virginia Polytechnic Institute and State University, Blacksburg, VA Tie He, Nevada DOT (retired), El Dorado Hills, CA Gary L. Robson, West Virginia DOT (retired), Hurricane, WV Vincent Van Der Hyde, Jr., Oregon DOT (retired), Salem, OR Zhongjie âDocâ Zhang, Louisiana DOTD, Baton Rouge, LA Peter A. Kopac, FHWA Liaison Richard Pain, TRB Liaison
This report describes the factors that should be considered in designing experiments and presents 21 typical transportation examples illustrating the experiment design process, including selection of appropriate statistical tests. The examples encompass a wide range of transportation disciplines and statistical methods. This report will be very beneficial to anyone with limited research experience needing to answer a question based on data (e.g., presenting ozone concentrations in a region, determining whether a contractorâs qual- ity assurance/quality control procedures are adequate, estimating the effect of automated enforcement on speeds, monitoring trends in the condition of bridge superstructures, devel- oping a user survey to determine the impact of transit fare changes). The report is a com- panion to NCHRP CD-22, Scientific Approaches to Transportation Research, Volumes 1 and 2, which were developed in NCHRP Project 20-45 and present detailed information on statis- tical methods. NCHRP CD-22 is available at http://www.trb.org/Main/Blurbs/152122.aspx Transportation agencies spend millions of dollars conducting research to improve their ability to plan, design, construct, maintain, and operate the transportation system. These research projects cover a broad range of topics and use approaches ranging from fully con- trolled laboratory experiments to field observational studies. Unfortunately, some research projects use inappropriate experimental designs or data analysis techniques, thereby increas- ing costs and decreasing the likelihood of success. There are many excellent university-level texts on experimental design and data analysis, but these are often not well suited to the needs of those involved in state DOT research. Principal investigators and DOT research program and project managers need practical information that focuses on common problems that DOTs face so that they can make bet- ter decisions when planning and conducting research. NCHRP Project 20-45, âScientific Approaches for Transportation Research,â produced NCHRP CD-22 that presents valuable material for transportation researchers. It does not, however, cover experimental design. In NCHRP Project 20-71, Michigan State University and North Carolina State University determined how state DOTs handle experimental design for contract and in-house research and to what extent the results from NCHRP Project 20-45 are in use. They then described basic principles and approaches that should guide experimental design and data analysis in transportation research. Example cases of experiment design were developed across a broad range of functional areas within transportation. The example cases include: (1) the research question being addressed, (2) the dependent and independent variables, (3) data that will be collected, (4) techniques for analyzing the data, (5) interpretation of results, (6) discussion of the process, and (7) application of the approach used in other areas of transportation. F O R E W O R D By B. Ray Derr Staff Officer Transportation Research Board
1 Chapter 1 Introduction 1 Developing Effective Experiment Designs and Data Analysis Plans 1 What This Guide Is and Is Not 2 Organization of the Guide 3 Chapter 2 Some Questions and Answers About Experiment Design 3 What Is the Research Question (What Do You Want to Know)? 5 What Else Needs to Be Asked? 6 What Are Some Typical Problems and Pitfalls? 7 What Factors Affect Outcomes? 9 Summary 10 Chapter 3 Examples of Effective Experiment Design and Data Analysis in Transportation Research 10 About this Chapter 10 Basic Outline for Examples 11 Techniques Covered in the Examples 12 Areas Covered in the Examples 12 Example 1: Structures/Bridges; Descriptive Statistics 16 Example 2: Public Transport; Descriptive Statistics 19 Example 3: Environment; Descriptive Statistics 23 Example 4: Traffic Operations; Goodness of Fit 26 Example 5: Construction; Simple Comparisons to Specified Values 29 Example 6: Maintenance; Simple Two-Sample Comparisons 31 Example 7: Materials; Simple Two-Sample Comparisons 35 Example 8: Laboratory Testing/Instrumentation; Simple Analysis of Variance (ANOVA) 38 Example 9: Materials; Simple Analysis of Variance (ANOVA) 41 Example 10: Pavements; Simple Analysis of Variance (ANOVA) 44 Example 11: Pavements; Factorial Design (ANOVA Approach) 47 Example 12: Work Zones; Simple Before-and-After Comparisons 50 Example 13: Traffic Safety; Complex Before-and-After Comparisons and Controls 52 Example 14: Work Zones; Trend Analysis 54 Example 15: Structures/Bridges; Trend Analysis 58 Example 16: Transportation Planning; Multiple Regression Analysis 64 Example 17: Traffic Operations; Regression Analysis 66 Example 18: Transportation Planning; Logit and Related Analysis 70 Example 19: Public Transit; Survey Design and Analysis 74 Example 20: Traffic Operations; Simulation 77 Example 21: Traffic Safety; Non-parametric Methods 79 Resources C O N T E N T S Note: Many of the figures and tables in this report have been converted from color to grayscale for printing. The electronic version of the report (posted on the Web at www.trb.org) retains the color versions.
