Developing the "Guide for the Process of Managing Risk on Rapid Renewal Projects" (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6C h a p t e r 2 Overview Many transportation projects experience budget and sched­ ule overruns and other types of undesirable performance issues (e.g., excessive disruption of traffic, short life span), which often result in other unfavorable outcomes (e.g., pub­ lic dissatisfaction, funding difficulties). These performance issues typically result from the occurrence of unexpected problems—risks; yet it is possible for some risks to be antici­ pated, managed, and effectively mitigated. Through literature search, interviews, and personal experi­ ence, the R09 research team found that few state DOTs have formal risk management programs to anticipate and pro­ actively manage risks. The team also found no comprehensive inventories (checklists) of rapid renewal elements, risks asso­ ciated with those elements, or actions for reducing those risks. This chapter describes the research approach introduced in Chapter 1 and details the methodology. This chapter also explains the findings of the research and how those findings were developed into the Guide for the Process of Managing Risk on Rapid Renewal Projects (the guide) and associated imple­ mentation materials. The research comprised five tasks. Each task is described in the subsections that follow. • Task 1: Develop a work plan. • Task 2: Develop a draft guide and related implementation materials. • Task 3: Conduct pilot workshops. • Task 4: Finalize the guide and related implementation materials. • Task 5: Manage the study. task 1: Develop a Work plan The first task was to develop an initial detailed work plan for the entire project consistent with the proposal. On the basis of this initial plan (submitted January 13, 2008, and approved with revisions June 4, 2008), sufficient research was con­ ducted to determine where significant gaps exist in the cur­ rent risk guidelines and associated implementation materials with respect to application to rapid renewal projects. This research included a literature review, interviews with select DOTs, drawing on research team experience, and review of other related research projects. The gaps were documented and a more­detailed plan developed to fill the gaps for rapid renewal projects. The Task 1 report, with input from the Tech­ nical Expert Task Group and staff, was finalized on January 14, 2009. The Task 1 report is unpublished, but its various subtasks are described in more detail below. Subtask 1.1: Conduct Team Kick-Off Meeting At project initiation, the team conducted an internal project kick­off meeting. The primary objectives were to (a) clarify project scope and approach, as well as establish communication methods and expectations; (b) share information; and (c) coor­ dinate related activities of the team members. The primary out­ come of this meeting was a preliminary detailed work plan. Subtask 1.2: Review the FHWA Risk Guide and Other Background Documents After the kick­off meeting, all team members reviewed the FHWA Risk Guide and considered other ongoing develop­ ments in their evolution. The team also reviewed other docu­ ments that address various innovative contracting methods and specifications (e.g., NCHRP Report 451: Guidelines for Warranty, Multi-Parameter, and Best Value Contracting) and cost estimating (e.g., NCHRP Report 574: Guidance for Cost Esti mation and Management for Highway Projects During Plan- ning, Programming, and Preconstruction) (Anderson and Russell 2001; Anderson et al. 2006). This review inventoried and exam­ ined the distinct variables of the available construction and contracting methods (e.g., insurance, finance, and safety). Findings and Applications

7 2002, the task force conducted the first two pilots of ACTT; since then FHWA and AASHTO have carried forward the effort as managers of the ACTT program. Although construction is in the ACTT title, the program addresses all phases of project delivery. As the FHWA described in its interim report on the ACTT program, “the goals of the program include minimizing the impact of ongoing highway construction on motorists and adjacent communities by streamlining project schedules and containing costs while enhancing safety and improving quality.” Under the ACTT program a corridor or project is selected because of its need for accelerated delivery; each one is reviewed in a 2­day workshop by experts from local, state, and federal agencies and private industries with a variety of skill sets relating to project accelera­ tion. The multidisciplinary team of 20 to 30 transportation experts works with local transportation agency professionals to evaluate all aspects of the project. Workshop participants present feasible recommendations for reducing roadway con­ struction time, enhancing safety, and delivering quality. The whole ACTT process is ultimately aimed at enabling agencies to save time and money while reducing construction­related congestion and improving work zone safety (FHWA 2004). The findings of these project reviews were documented in project reports, annual reports, training materials, and addi­ tional records (available on the ACTT website at http://www .fhwa.dot.gov/construction/accelerated/). The costs of proj­ ects examined ranged from $1 million to $3.5 billion. A rigor­ ous review of these reports by the R09 research team yielded a significant number of case studies that were later used to develop a preliminary inventory of rapid renewal methods, as well as some common recommendations for accelerating the projects. The ACTT project analysis and resulting recom­ mendations were organized by skill sets that offered a logical framework for the inventory of rapid renewal methods. The skill sets were the following: • Innovative contracting and financing; • Roadway geometric design; • Structures; • Traffic engineering, safety, and intelligent transportation systems (ITS); • Environment; • Construction; • Coordination of rights­of­way (ROWs), utilities, and railroads; • Geotechnical aspects, materials, and accelerated testing; • Long­life pavements and maintenance; and • Public relations. Figure 2.1 summarizes the skill sets that were applied to the 25 ACTT projects. The construction skill set was represented in all 25 ACTT workshops, which was not surprising given The team also reviewed the reports of the Accelerated Con­ struction Technology Transfer program, which is discussed further in the subsection below, Review of Accelerated Con­ struction Technology Transfer Program. Ultimately this prelim­ inary literature review was used to identify potential problems (risks) with various elements and possible ways to mitigate them (lessons learned)—the topic of Task 2. The content of these documents is briefly summarized in an annotated bib­ liography included in Appendix A. Furthermore, the team proposed to survey and interview FHWA and DOTs about their experience with risks in rapid renewal activities and their interest in being involved in this study; these topics are discussed further in the subsection below, Agency Surveys and Interviews. Similarly, team mem­ bers’ experience with rapid renewal projects is summarized in the subsection below, Summary of Industry Experience, and in Appendix C (see also Table C.9). Literature Review The research team reviewed available literature to identify information on risks related to rapid renewal projects. The search included TRB resources, academic engineering data­ bases, academic business databases, American Society of Civil Engineers (ASCE) and Project Management Institute (PMI) publications, and selected transportation agency websites. The resulting annotated bibliography of that literature search appears in Appendix A and is largely taken from the results of NCHRP Project 8­60 (Molenaar et al. 2010) on the same sub­ ject, which was conducted by R09 research team member Keith Molenaar concurrently with this research project. Review of Accelerated Construction Technology Transfer Program The methodology for developing the inventory of rapid renewal strategies and methods included a review of 25 case studies from the Accelerated Construction Technology Trans­ fer (ACTT) program; these cases represent the state of the art in rapid renewal construction. The area of rapid renewal has been evolving in the highway industry for more than 10 years. FHWA and American Association of State Highway and Transportation Officials (AASHTO) have been at the fore­ front of the effort through their work on the ACTT program. In 1996, TRB released Special Report 249, which called for formation of a strategic forum to accelerate innovation in the highway industry. In response, the TRB Task Force A5T60 (now AFH35T) was formed in 1999 to facilitate removal of barriers to innovation, advocate continuous quality improve­ ment and positive change, encourage development of strate­ gies that generate beneficial change, and create a framework for informed consideration of innovation (FHWA 2004). In

8that rapid renewal projects generally involve construction under traffic. Other skill sets—structures; traffic engineering, safety, and intelligent transportation systems; innovative con­ tracting and financing; and geotechnical aspects, materials, and accelerated testing—were all represented in more than 80% of the workshops. These areas could also be considered a primary focus of rapid renewal projects as they promote the “get in, get out, stay out” philosophy of rapid renewal proj­ ects. The long­life pavements and maintenance skill set was represented in less than one­third of the projects. While long­ life pavements and ongoing maintenance are considerations in the rapid renewal approach, they did not seem to be as urgent as the other issues being addressed. Agency Surveys and Interviews An important part of the R09 research in both Task 1 and Task 2 was to obtain relevant information from state DOTs. The R09 research team analyzed two surveys to complete the gap analysis. The first was a recent state­of­practice survey of various state DOTs on their risk management and cost esti­ mation programs, which was conducted under a separate contract. The second survey analysis was conducted by team members under the SHRP 2 R09 contract, which followed up with DOT respondents of the previous state­of­practice sur­ vey. The second survey inquired about respondents’ experi­ ence specifically with rapid renewal projects and the associated risks of those projects. State-of-Practice Survey Members of the R09 research team were simultaneously but separately working under a research contract with TRB’s National Cooperative Highway Research Program (NCHRP). Under this separate contract—NCHRP 08­60, Guidebook on Risk Analysis Tools and Management Practices to Control Transportation Project Costs—team members developed a state­of­practice survey to identify how different transporta­ tion agencies and organizations determine contingencies and manage risk­related costs throughout the process of project development (hereafter referred to as the state­of­practice survey). The survey received responses from 48 of the 52 DOTs (50 states, District of Columbia, and Puerto Rico), four Canadian agencies, and more than 130 individuals. Key results from this survey (those relating to risk management for rapid renewal projects) were as follows: • Only one in five agencies had a formal, published definition of contingency that was used consistently throughout the 0 5 10 15 20 25 Long-Life Pavements/ Maintenance ROW/Utilities/ Railroad Coordination Roadway/Geometric Design Environment Public Relations Geotechnical/ Materials/ Accelerated Testing Innovative Contracting/ Financing Traffic Engineering/ Safety/ITS Structures Construction Sk ill S et s Number of Projects Figure 2.1. Summary of skill sets used in 25 ACTT projects.

9 estimating process. Given the importance of contingency in managing budgets, this low proportion is unexpected, as is the fact that schedule contingency is generally even less defined. • Only one in 10 agencies had a formal, published project risk management policy or procedure. As risk management is an emerging (rather than established) trend in the highway sector, this low proportion is perhaps less unexpected than the result of the previous item. Risk Management Practices and Application of Contingency Risk is inherent in every capital transportation project. One definition of risk is “the possibility that something unpleasant or unwelcome will happen.” In this study, risk is defined as “an uncertain event or condition that, if it occurs, has a negative or positive effect on a project’s objectives” (Project Man­ agement Institute 2008). Risk management, then, involves several specific steps: risk identification, assessment, analysis (qualitative or quantitative), planning, allocation, monitoring, and control. While risk is inherent in every capital transportation proj­ ect, the state­of­practice survey found that only three of the 48 state agencies had a formal, published project risk man­ agement policy or procedures. California, Florida, and Wash­ ington had formal risk management procedures (Utah was in the process of establishing such procedures). In those three states, it was clear how the risk analysis related to controlling cost escalation. Representatives from the three state agencies that had formal management procedures were interviewed and case studies were written in NCHRP Report 658: Guide- book on Risk Analysis Tools and Management Practices to Con- trol Transportation Project Costs (Molenaar et al. 2010). In the other states, the manner in which the agencies set their proj­ ect and program contingency did recognize and incorporate risks into project estimates but not in a formalized risk man­ agement procedure. Appendix C includes short profiles of how 10 states set contingency and analyze risk in projects. Contingency is a future event that is possible but cannot be predicted with certainty. In project estimates, contingency is the tool that estimators and project managers use to address risk and uncertainty. In this study, contingency is defined as the “esti­ mate of costs associated with identified uncertainties and risks, the sum of which is added to the base estimate to com­ plete the project cost estimate. Contingency [funds are] expected to be expended during the project development and construction process” (Anderson et al. 2009). The transportation industry generally agrees that contin­ gency is necessary but disagrees significantly about which risk­associated costs should be included in a contingency amount and how that amount should be calculated. The results of the state­of­practice survey showed that only one in six (8 of the 48) responding agencies had a formal, published definition of contingency. Without that formal definition, agencies will have difficulty in consistently calculating appro­ priate contingency amounts or communicating the elements that constitute contingency in an estimate. Setting contingency The lack of a formal definition does not imply that agencies disregard contingencies in their estimates. Approximately four of five agencies responding to the state­of­practice sur­ vey stated that they apply contingency in at least one phase of the project development process. Agencies set contingency through use of three primary methods: (1) a standard pre­ determined contingency by percentage, (2) a unique project contingency set by individual estimators, and (3) formal risk analysis and associated contingency. The first method uses a standard predetermined contin­ gency by percentage across all projects. Sixteen of the 48 reporting state agencies employed some form of this method on their projects. Even when an agency applies a standard contingency, it can make exceptions for various reasons, including phase in the project development process, project type, project complexity, market conditions, geographic region, and estimated project value. The second method has a unique project contingency that the engineers, estimators, or project managers set. The major­ ity of agencies responding to the survey stated that they used this method. When a unique project contingency is applied, many tools are used to determine the contingency, including engineering judgment, statistical analysis of historical data, correlation of historical data with current market prices, and assignment of contingency for specifically identified risks. The third method uses formal risk analysis and its associ­ ated contingency. The survey responses from agencies in Cali­ fornia, Maryland, and Washington indicated that they use a combination of formal risk analysis and unique project con­ tingency. Furthermore, the FHWA response stated that it uses formal risk analysis. The research team knew of other agencies using formal risk analysis to set contingencies on a project­by­ project basis (e.g., Colorado, Florida, Minnesota, New York, and Texas), but the survey respondents did not enumerate that use in their answers. Tools for determining contingency through risk analysis include use of expected values through statistical analysis of historical data for assigning cost to risks, use of expected values through engineering judgment for assigning cost to risks, Monte Carlo or simulation methods, influence diagramming, and probability or decision trees. This third method, formal risk analysis, was a primary focus of this research and is explored in detail in the guide.

10 contingency at Project LeveL or Program LeveL State agencies can apply contingency at an individual project level or a program level. Applying contingency at a project level determines a contingency amount for an individual proj­ ect cost estimate on a project­by­project basis. Applying con­ tingency at a program level spreads contingency across projects (e.g., as set­aside amounts in a state transportation improve­ ment program). Depending on the phase of project develop­ ment, state agencies can choose to apply contingency at one or both of these levels. Over half of state agencies in the state­of­practice survey apply contingency at a project level for all three development phases. Only one state agency applied it at solely a program level, regardless of developmental phase. Just less than one in five agencies used a combination of project and program con­ tingencies (19% at the planning phase and the programming and preliminary design phase and 16% at the final design phase). The remaining responding agencies do not apply con­ tingency in their estimates (26% at the planning phase, 21% at the programming and preliminary design phase, and 21% at the final design phase). Range Estimates Versus Point Estimates One method of communicating estimates is as a single number (a point estimate). Point estimates can include a stated contin­ gency to help convey uncertainty. Another method of convey­ ing estimate uncertainty is through a range estimate, which may include simple best­case and worst­case points or may be shown graphically with a probability curve (probability mass function). Depending on the project phase, one method might be considered more appropriate than the other. Table C.3 in Appendix C summarizes the use of ranges by agencies to com­ municate estimates. The state­of­practice survey that was developed under the NCHRP 08­60 research project provided valuable informa­ tion for the R09 research team. It clearly demonstrated the need for guidance on risk management and estimation of contingencies. It highlighted common tools currently being used by agencies. When mapped against the literature review, the survey also revealed tools for risk analysis and risk man­ agement that are absent from the transportation sector. Finally, the survey pointed to the best agencies to interview in the next phase of the R09 research. For a more­detailed description of the state­of­practice survey, see Appendix C. Follow-Up Survey To narrow the focus of the research to rapid renewal projects, the R09 research team developed a focused, follow­up sur­ vey to the recent state­of­practice survey. It was developed for state DOTs to specifically address rapid renewal risks (here­ after referred to as the rapid renewal survey). This draft sur­ vey was developed to be as short and focused as possible to encourage participation. The survey was later revised and streamlined. The actual rapid renewal survey was conducted as part of Task 2 and is further discussed in the Task 2 sub­ section titled Research. AASHTO Subcommittee on Construction Annual Meeting On the recommendation of the SHRP 2 R09 program officer, a member of the research team attended the AASHTO Sub­ committee on Construction (SOC) annual meeting in San Antonio, Texas, in August 2008 to discuss AASHTO’s assis­ tance in conducting the rapid renewal survey. The SOC pub­ lishes the guide specifications for construction and coordinates the practices of the several member DOTs regarding construc­ tion procedures. It hosts a forum to exchange information on construction procedures and endeavors to reduce construc­ tion cost, promote quality in construction, provide coordi­ nated plans and specifications, mitigate traffic impacts, advocate environmental sensitivity in construction, promote safety for workers and travelers, and promote the best prac­ tices for administering construction contracts with all stake­ holders (AASHTO 2013). The SOC annual meeting brought together representatives from the 52 member departments of SOC to discuss the committee’s mission and agenda; 178 peo­ ple were in attendance. (The SOC annual meeting website is located at http://construction.transportation.org/?siteid= 58&pageid=732). To assist the R09 research team in developing the rapid renewal survey, the vice­chairman posed the following two questions relating to rapid renewal risks during a general question­and­answer session of the SOC conference: 1. Does your organization have an established policy regard­ ing rapid renewal? 2. Does your organization have established procedures for risk management? None of the DOTs that responded had an established policy related to rapid renewal. Despite that initial negative response, subsequent discussions found that a number of states had innovative contracting groups or leaders within their agencies that address accelerating construction and conducting con­ struction under traffic. The Utah DOT used construction man­ ager at risk, design–build, accelerated bridge construction, and other innovative methods; and it had policies covering the circumstances under which the use of innovative project delivery is appropriate (e.g., to accelerate projects through

11 rapid renewal techniques). California used design sequencing, A + B bidding, and lane rental. Washington used extensive design–build project delivery. Finally, Florida used all the alter­ native contracting approaches already mentioned, plus public­ private partnerships. It also used advanced techniques with respect to accelerating construction through innovative con­ tracting methods. The second question regarding established risk manage­ ment procedures confirmed the state­of­practice survey con­ ducted under NCHRP 08­60 by members of the research team. Only California, Washington, and Florida confirmed having formal risk management procedures. Utah stated that it had risk procedures regarding the selection of insurance but not relating to risk management for scope, cost, and schedule. The Utah DOT is currently in the process of estab­ lishing a risk management program (most likely modeled on the Washington program). Subsequent discussions with SOC members also identified a current report relating to the subject of rapid renewal: Primer on Contracting for the Twenty-first Century (AASHTO 2006). This report covers much of what was being addressed in the research team’s proposed survey. As a result, it was determined that the survey could be further streamlined. The research team later reviewed this report for rapid renewal techniques and related risks. interviewS During the SOC session, volunteers were solicited for inter­ views. Representatives from the DOTs of California, Florida, Utah, and Washington were identified as candidates for inter­ views or case studies on the basis of their policies and proce­ dures on risk management and their use of innovative contracting methods relating to rapid renewal. Several of these DOTs subsequently attended pilot workshops in Task 3. Only Utah could provide a brief interview during the con­ ference. The director of construction and materials at the Utah DOT was interviewed. Utah has an aggressive program for accelerating construction and using rapid renewal con­ cepts. The state is actively using construction manager at risk, design–build, accelerated bridge construction, and other innovative methods. The director had not attended any risk analysis workshops, but he said that the state had conducted these workshops in the past. Utah does have stated policies for when to use innovative project delivery. A primary factor in deciding when to use innovative delivery relates to acceler­ ating projects through rapid renewal techniques. concLuSionS The AASHTO SOC 2008 annual meeting assisted the team in completing Task 1 of the research and identifying state highway agencies for Task 3 workshops. The AASHTO SOC was helpful in keeping the research team current with the state of the practice in rapid renewal techniques and risk management practices. However, on discussing the research topic in the general question­and­answer session at the con­ ference and further discussing the topics informally with conference attendees, the research team concluded that an additional follow­up survey would not yield significantly new results. In summary, a review of existing SOC documents, the current state­of­practice survey, and the team’s rich database of risks from previous analyses provided the information required for completion of Task 1. Summary of Industry Experience Several R09 research team members have extensive experi­ ence in conducting risk assessment and risk management for projects with rapid renewal elements (see the Appendix C section Summary of Industry Experience and Table C.9 for a summary of the rapid renewal projects in which R09 team members have been involved). Several members of the team worked with state DOTs including Florida, Iowa, Utah, and Washington on previous risk assessments. The team drew on this experience to identify categories of rapid renewal risks, as well as potential risk­mitigation strategies. Review of Existing Risk Management Guidelines As discussed in Chapter 1, the Guide to Risk Assessment and Allocation for Highway Construction Management (Risk Guide) developed in 2006 by FHWA provides only an over­ view and is not a how­to document (Molenaar et al. 2006). Thus, training materials and tools for implementing the Risk Guide were developed (Golder Associates 2008); these tools and materials were successfully applied in late 2007 and early 2008 to projects for four state highway agencies: Colorado, Florida, Texas, and Virginia. The development of implemen­ tation materials involved significant expansion of various parts of the Risk Guide, although actual revision of the Risk Guide was not included. The implementation materials can be viewed as products developed to support the implementation of the original FHWA guidelines. The newly developed implementation materials deal with cost and schedule risks associated with completing a highway construction project through traditional deliv­ ery methods (such as design–bid–build, with one construc­ tion contract package), although they can apply to various ways of completing a project (e.g., accelerated construction methods). However, rapid renewal projects should con­ sider more than just cost and schedule risks associated with completing a highway construction project (e.g., risks of disruption and durability consequences) and can involve

12 nontraditional methods of project delivery (e.g., design– build), as well as accelerated construction methods. More­ over, less experience and understanding—and often more risk—are generally associated with some of the accelerated and nontraditional methods being considered. If agencies fail to evaluate these methods appropriately, the con sequences might be poor decisions and poor project results. Thus, team members assessed the applicability of the Risk Guide and related materials to rapid renewal projects and recom­ mended ways to expand them (see Tables C.10 and C.11 in Appendix C). Subtask 1.3: Identify Additional Material Needed (Gaps) On review of the guidelines and related information, the proj­ ect team identified additional elements needed to develop the Guide for the Process of Managing Risk on Rapid Renewal Proj- ects. The team found that generally the necessary tools and methods already existed. However, a database of signi ficant risks and feasible risk mitigation for accelerated construction and innovative contracting methods and specifications did not yet exist, especially for some of the newer methods. Recommended expansions and revisions of the FHWA Risk Guide for both traditional and rapid renewal projects are sum­ marized in Appendix C, Table C.10. In essence, for traditional projects, the Risk Guide first needs to be updated to reflect the expansion associated with development of the implementa­ tion materials. Such updates include the following: • Add a new chapter on baselining the project (identifying and documenting key assumptions, scope, delivery strat­ egy, and baseline costs; developing the flow chart and base­ line schedule). This would be the new Chapter 2. • Add a new chapter on implementation (how to implement the guidance in the new Chapters 2, 3, 4, and 6). This chap­ ter would reference the training workshop slides and the risk­management spreadsheet template. This would be the new Chapter 9. • Modify/update the existing Chapters 1–8 as needed based on work done for the short­course development. These would be the renumbered Chapters 1, Chapters 3–8, and Chapter 10. • Modify/update the existing Appendices A–D, glossary, and references/bibliography as needed based on work done for the short­course development. This would include adding case studies and a more complete generic risk checklist. • Include the training workshop PowerPoint slides (printed with Notes pages) as the new Appendix E to the Risk Guide. • Include the risk­management spreadsheet template as an electronic attachment (Appendix F). Once the Risk Guide has been updated to cover traditional projects, the unique aspects of rapid renewal (especially expanded project performance objectives and different proj­ ect delivery methods) would be covered primarily in a new Appendix G. Similarly, the currently recommended changes to the asso­ ciated implementation materials are summarized in Appen­ dix C, Table C.11. In essence, for traditional projects, no changes are needed. For rapid renewal projects, the unique aspects of such an application (especially expanded project perfor­ mance objectives and different project delivery methods) would be covered in a new Module 8 and by modifications to the software training module. Subtask 1.4: Develop Report and Plan (with Input from State DOTs) The research and discussions with various experts made clear that the scope and objectives of rapid renewal projects had to be more broadly but more carefully defined for the R09 proj­ ect. The research scope included accelerated construction, planning, and maintenance methods, as well as accelerated project delivery methods (e.g., design–build and public­private partnerships). Furthermore, the project needed to address additional project performance objectives, including mini­ mal disruption (during construction) and maximum longevity (considering cost and disruption of operations, replacement, and design life), as well as minimal planning–construction cost and schedule. The team found that the necessary tools and methods for managing risk on accelerated projects already existed, although the ones needed for evaluating rapid renewal projects were not adequately described in the FHWA Risk Guide. The team also found that a database of significant risks and feasible risk mitigation for rapid renewal projects was the item most in need of development. In consultation with various DOTs that expressed interest in Subtask 1.2 (review of the guidelines and other background documents), the team developed a detailed plan for filling the gaps. Once the needed developments had been identified, the project team prepared a draft report that (a) documented the review of the guidelines, other documents about accelerated construction and innovative contracting methods and speci­ fications, and the survey of agencies; (b) identified gaps in the guidelines that needed to be filled to cover rapid renewal projects adequately; and (c) presented a plan for filling those gaps and thus developing a complete guide for rapid renewal projects. Specific recommendations for expanding the exist­ ing guidelines and implementation materials appear in Appendix C, Table C.10. On the basis of the gap analysis, the team determined that a new, more­detailed guide for risk management—specifically

13 addressing the unique features of rapid renewal projects (as well as traditional projects)—was required. Also needed were new tools and training materials for state DOTs to use inter­ nally, without external assistance, on relatively simple projects. The development of the guide and associated materials are discussed further in the next section. task 2: Develop a Draft Guide and Implementation Materials After SHRP 2 approved the detailed work plan, the focus shifted to Task 2: developing the guide and accompanying implemen­ tation materials. To understand the risks (i.e., potential prob­ lems, potential opportunities, or both) of rapid renewal, the research team first developed an inventory of rapid renewal strategies and methods. That inventory informed the risk man­ agement process with the aspects unique to rapid renewal proj­ ects and their associated risks—in contrast to projects that follow the more traditional linear project development process and methods. The intent was to further expand on the FHWA Risk Guide and related implementation materials to cover the unique aspects of rapid renewal projects. The research team addressed these rapid renewal features: • Considering additional project performance measures in evaluating a project (e.g., disruption and longevity, as well as construction cost and schedule); • Considering the various potential project delivery meth­ ods (e.g., design–build, contractor­financed), as well as accelerated construction methods; and • Understanding the risks typically associated with the vari­ ous potential project acceleration methods (e.g., acceler­ ated bridge construction, accelerated permitting) and how they might best be managed. Task 2 consisted of two basic subtasks: • Subtask 2.1: Research (on several parallel paths) to identify the various unique aspects of rapid renewal projects, the risks associated with those aspects, and feasible ways to manage those risks, considering the various project perfor­ mance objectives (e.g., minimum schedule, minimum capi­ tal cost, minimum disruption, and maximum longevity). • Subtask 2.2: Development of the draft guide and associated implementation materials that incorporate the above research. Research was completed and submitted to SHRP 2 on Sep­ tember 1, 2009. The draft guide and associated implementa­ tion materials were completed and submitted to SHRP 2 on October 29, 2009. Subtask 2.1: Research The key elements of this research included the following: • Establishing appropriate rapid renewal project perfor­ mance objectives and related measures; • Developing inventories of rapid renewal methods, risks, and feasible management actions; • Establishing an appropriate risk management process; • Developing a template for documenting assessments (also forms) and automatically calculating performance mea­ sures, consistent with that process; and • Developing the guide, training materials, and other work­ shop materials. The main effort of Task 2 involved the development of a checklist of risks (or categories of risks) and associated risk mitigation for rapid renewal projects, in particular, innova­ tive contract methods and specifications. In general, methods and tools for identifying and assessing risks, as well as for identifying and evaluating risk mitigation, already exist. But the specific nature of the risks associated with rapid renewal projects and how they are handled tends to be unique and less understood—or at least less well com­ municated within DOTs and the contracting community. This, in turn, required a thorough understanding of the con­ tractual relationships in rapid renewal projects and of the specifications appropriate to those relationships. Both expe­ rience with such contracting methods (which is generally limited) and with theoretical analysis (especially for newer methods) were required to identify risks and, even more so, risk mitigation. The first steps in this process were to define the various performance measures to be considered and to identify the various potential project delivery methods and project accel­ eration methods that might be proposed for any given proj­ ect. The next logical steps were to identify the risks that might be associated with each of the potential project acceleration methods, and how they should be assessed; and to identify possible ways to manage those risks, and how they should be managed. This research to identify the risks and their mitigation involved the following three parallel approaches: 1. An additional literature review to supplement the one con­ ducted under Subtask 1.2, including further development of an annotated bibliography focused on identi fication of risks and risk mitigation for rapid renewal projects. The analysis of the 25 ACTT project reports conducted in Sub­ task 1.2 yielded some common findings across the recom­ mendations for accelerating the project. Although it might be argued that not all of these recommendations are unique

14 to rapid renewal (e.g., brand the project, consider owner­ controlled insurance programs) or that some are actually risk management methods (e.g., require a pavement war­ ranty), the research team synthesized these recommenda­ tions into a preliminary rapid renewal inventory. 2. A theoretical approach based on the expert judgment of team members used brainstorming to identify all the poten­ tial problems that could conceivably arise with each innova­ tive contracting method. Then team members identified feasible ways to mitigate each of those problems using their judgment; this included identification of the factors that affect the severity of risk and the cost­effectiveness of risk mitigation. The theoretical approach involved an intensive 2­day workshop during which the team members identified contractual relationships and their effect on risk. 3. Interviews with DOT personnel helped refine the list of potential problems and mitigation approaches (see Appendix D) to validate the problem list and ensure iden­ tification of any additional mitigation approaches. (The interviews are discussed further in the next subsection, Interviews with DOT Personnel.) The results of these three approaches was a list of risks (or categories of risks) related to each dimension of rapid renewal and innovative contracting methods with feasible risk miti­ gation, as well as factors that affect risk severity and risk miti­ gation cost­effectiveness. An inventory of rapid renewal strategies and methods, as well as lists that present more­detailed descriptions of the recommendations, appear in the Guide for the Process of Man- aging Risk on Rapid Renewal Projects (http://www.trb.org/ Main/Blurbs/168369.aspx). Table 2.1 provides an overview of the rapid renewal inventory; the guide provides more­ detailed tables of rapid renewal dimensions, methods, risks, and mitigations. Interviews with DOT Personnel Additional surveys were conducted to support the develop­ ment of inventories of rapid renewal methods, risks, and fea­ sible management actions. A draft survey for DOTs was developed under Task 1 and revised under Task 2 to more effi­ ciently solicit information from DOTs on rapid renewal meth­ ods and their risks and possible mitigation (see questionnaire in Appendix D, Figures D.1 and D.2). The team completed interviews with five DOT personnel. The interviews solicited details on accelerated construction projects from the Utah DOT, Center for Transportation Research at the University of Texas at Austin, Texas DOT, California Department of Trans­ portation (Caltrans), and Colorado DOT (see Appendix A for agency contact information). Interviewees explained some of the main risks faced during rapid renewal projects and the risk management actions that their agencies employed. Table 2.2 presents the risk categories and risk management actions named; a summary of the interviews is also available in Appendix D. Subtask 2.2: Development of Draft Guide Once the general risks, risk mitigations, and their contribut­ ing factors were identified, the process of developing the final guide began. The team generated an outline for the guide and then proceeded to develop an annotated outline for the guide and associated implementation materials. (The investigators found this to be an important step in the research process.) Development of a new, more­detailed guide and related implementation materials to appropriately fill the needs identified in the gap analysis included development of the following major components: • Risk management process; • Rapid­renewal project performance objectives; • Rapid renewal methods, risks, and mitigation inventories; • Risk management planning methods, tools, and guidance; and • Risk management program guidance. Risk Management Process The research team developed a formal risk management pro­ cess to improve understanding of rapid renewal projects and to optimize project performance, especially by anticipating and planning for potential problems (risks). This process, which is a significant expansion of a previously developed risk management process for non–rapid renewal projects (and for which the expanded process is also applicable), con­ sists of a well­defined series of steps that are sequential and in some cases iterative (see Figure 2.2). Agencies must follow the steps in such a way to ensure compatibility and consistency of those steps and to ultimately ensure adequate accuracy and defensibility of results (“adequacy” depends on how the results will be used), as efficiently as possible. The steps in the risk management process include the following: 1. Structuring. Define the base project scenario (including the relevant project performance measures of cost, sched­ ule, and disruption through construction, postconstruction longevity, and trade­offs among them), against which risk and opportunity can subsequently be identified, assessed, and eventually managed. 2. Risk identification. Identify a comprehensive and non­ overlapping set of risks and opportunities relative to the

15 Table 2.1. Rapid Renewal Inventory Overview Construction Structures Traffic Engineering/ Safety/ITS Innovative Contracting/ Financing Geotechnical Materials/ Accelerated Testing Public Relations Environment Roadway/ Geometric Design Right-of-Way/ Utilities/ Railroad Coordination Long-Life Pavements/ Maintenance • Closures • Preliminary work/staging • Project administration streamlining • Construction operations • Prefabrication • Component reuse • High- performance materials • Integral designs • Standardized design • Construction placement • Temporary structures • Long-life structural design • Advance planning • Alternate routes • Alternate modes • Improved physical separation • Coordinated emergency response • Signage and signalization • Closures • Work zones • Alternative financing • Project delivery • Procurement • Contract payment • Warranties • Alternative insurance • Advance contract packaging • Bonding/ performance securities • Subsurface exploration • Walls • Pavements • Alternative materials • Intelligent compaction • Material testing • Team integration • Single-point communication • Additional investment • Project branding • Stakeholder awareness • Performance measurement • Master planning • Context- sensitive solutions • Comprehensive scoping • Advance permitting • Alternate access • Alternate geometrics • Advance roadwork • Advance right-of-way planning • Early utility location • Common utility crossings • Early railroad coordination • Life-cycle design • Performance indicators • Long-life materials • Maintenance involvement Note: ITS = intelligent transportation system.

16 Project Scope/Strategy/ Conditions Structuring Risk Identification Risk Assessment Risk Analysis Risk Management Planning Risk Management Implementation Figure 2.2. Risk management process. Table 2.2. Risk Categories and Risk Management Actions Risk Categories Risk Management Actions Potential failure of innovative equipment Coordination with utilities/stakeholders Accelerated bridge design Off-site prefabrication of bridge elements Delayed-start contract provisions to prepare for in-traffic work before starting Coordination risk of maintenance of traffic and utilities Public relations Management of traffic Failure of innovative pavement materials Agency unfamiliarity with the process ROW acquisition issues Innovative delivery methods (design–build, CMR) Facilitated partnering sessions with utility partners Use of performance specification for bridge design Lane rental provisions Incentives/disincentives Extensive public outreach Hiring a general engineering consultant to coordinate contracts Focus on maintenance of traffic and utilities Incentives/disincentives at contract coordination points Early and continuous stakeholder interaction and communication Extensive mix design research and off-site testing Agency training Augmentation of ROW staff base (i.e., scenarios that might occur to change the base project performance). In addition to first brainstorming and then performing project analysis to identify risks, use checklists of common risks (developed as part of this research) to ensure completeness. Document the set of risks and opportunities at the start of the project in a risk register (a record in which all project risks, including information such as risk probability, impact, and counter­ measures, are listed). 3. Risk assessment. Assess and prioritize each of the risks and opportunities in the risk register on the basis of severity. Generally this requires (a) subjectively assessing the rele­ vant risk factors (i.e., the probability of a scenario occurring and the impact if the scenario occurs), either qualitatively (e.g., high versus low, when these descriptors are quantita­ tively defined by ranges of values) or quantitatively (in rela­ tion to mean values or, for quantitative risk analysis, full probability distributions); and then (b) analytically com­ bining the risk factors to determine changes in project performance measures and thus severity. Document the risk factor assessments in the project risk register. 4. Risk analysis. Assess and analytically combine the uncer­ tain base and risk factors to determine the uncertain proj­ ect performance measures (e.g., ultimate escalated project cost), as well as changes in those measures (e.g., combined using trade­offs, as a measure of severity) associated with each risk. The quantification of the uncertainty in the performance measures is expressed as correlated proba­ bility distributions and calculated as a function of subjec­ tively assessed uncertainties in (and correlations among) the base and risk factors. To conduct this quantification appropriately requires that the analyst have specialized skills. 5. Risk management planning. Identify and evaluate possible ways to reduce risks proactively, focusing on those that are most severe. Evaluate each possible action in relation to its cost­effectiveness, considering changes in both base (e.g., additional cost) and risk (e.g., reduced probability) factors, and select those that are most cost­effective. Consider sub­ sequently reanalyzing the project performance measures for this risk reduction program, including quantification of uncertainty, on the basis of which appropriate budgets and milestones can be established (e.g., to achieve a specified level of confidence). As part of these budgets and milestones, establish contingencies (additional funds and schedule float, as well as recovery plans) and procedures to control their use. Document all in the risk management plan. 6. Risk management implementation. Implement the risk man­ agement plan as the project proceeds, including (a) moni­ toring the status of risk reduction activities and changes in risk (whether from risk reduction or simply changes in project development, conditions, and information) and (b) monitoring budget and milestones, especially with respect to contingencies. This monitoring might involve periodic updates (iterate Steps 1 to 5) at regular intervals or at major milestones or changes. For example, contingencies

17 might be reduced as engineering reports or designs are completed and risks are avoided or reduced. Rapid Renewal Project Performance Objectives As previously discussed, the performance objectives for rapid renewal projects had to be expanded beyond simply mini­ mizing construction costs and schedule. The expanded objec­ tives were to (1) minimize cost, time, and disruption of traffic (user costs) during construction; and (2) maximize longevity (i.e., minimize costs and disruption of traffic associated with operations and maintenance, as well as with ultimate replace­ ment or decommissioning; and maximize the time from the end of construction to replacement or decommissioning). Metrics, or performance measures, were defined by the research team for each of the objectives, as well as for the com­ bination of objectives, as follows: • Time through construction is expressed as the project oper­ ations date, which requires an analysis of the schedule through construction. • Cost through construction is expressed as the total inflated [year­of­expenditure (YOE)] construction cost, which requires a construction cost–loaded schedule (i.e., un­ inflated construction cost estimate, schedule, and alloca­ tion of cost items to schedule items) and inflation rates. • Disruption through construction is expressed as total equiva­ lent lost person­hours, which requires a traffic disruption analysis (i.e., duration of disruption, average number of users affected, and average delay per user) and a business disruption analysis (if needed, for example, because it is not mitigated or translated to equivalent lost person­hours). • Longevity is expressed as total equivalent postconstruc­ tion (i.e., operations and replacement) discounted cost [or net present value (NPV), to the end of construction], which requires uninflated cost for operations (i.e., sched­ ule for operations and average cost per year) and for replacement, disruption during operations (i.e., schedule for operations and average number of days disruption per year) and during replacement, value of disruption (as of the end of construction), and net discount rate after construction. • Combined project performance is expressed as equivalent total inflated cost [in year­of­expenditure dollars (YOE$)], which requires inclusion of these elements: 44 Cost through construction (in YOE$); 44 Time through construction (date) and the value of changing the operations date (in YOE$/month) from a specified milestone (date); 44 Disruption through construction (equivalent lost person­ hours) and the average value of lost person­hours (in YOE$/h); and 44 Longevity [net present value dollars (NPV$) at the end of construction] and the value of changing longevity (in YOE$/NPV$). Combined project performance appropriately merges all the various project performance objectives (on the basis of formal methods of decision analysis). Thus, changes in com­ bined project performance can be used to define the severity of the various risks; on the basis of the relative severities, risks should be prioritized for managing and for gaining the ben­ efits of such management. Rapid Renewal Methods, Risks, and Mitigation Inventories Comprehensive inventories of rapid renewal methods, risks associated with those methods, and potential ways to pro­ actively reduce each of those risks were developed by the research team. 1. A comprehensive hierarchy of rapid renewal methods was developed by reviewing FHWA ACTT workshop reports (including those for 25 workshops and other information such as training materials) and by interviewing knowledge­ able DOT personnel, supplemented by personal experience of the research team (see Appendix C and Appendix D). 2. A comprehensive set of risk categories for each rapid renewal method was developed, primarily on the basis of the personal experience of the research team and supple­ mented by interviewing knowledgeable DOT personnel (see Appendix D). The full risk checklists are available in Appendix B of the guide (Golder Associates et al. 2014). 3. A comprehensive set of management actions for each risk category for each rapid renewal method was developed, primarily on the basis of the personal experience of the research team and supplemented by interviewing knowl­ edgeable DOT personnel. See Appendix B of the guide for the full list of rapid renewal risk categories and potential management actions. In evaluating a project, these inventories should be used after brainstorming to help ensure that the risk register and the risk management plan have considered all conceivable options and potential problems. Risk Management Planning Methods, Tools, and Guidance In addition to rapid renewal performance objectives and inventories, successful and efficient implementation of the risk management process required development of various methods and tools, as well as guidance for their use.

18 For successful project structuring the research team devel­ oped the following: • A comprehensive but efficient format or outline for ade­ quately describing the relevant aspects of the subject project. • Standard simplified project flowcharts that graphically depict the sequence of major project phases, specifically for relatively simple traditional (design–bid–build) and nontra­ ditional (e.g., design–build) project delivery. See Figure 2.3 for nontraditional project delivery; the project flowchart for a traditional project is similar but more simple. • A generic project performance model to calculate (e.g., using an MS Excel template) the various project perfor­ mance measures (such as project schedule, escalated project cost and project disruption through construction, and postconstruction longevity, as well as a combined perfor­ mance measure), as a function of various inputs (e.g., cost, schedule, disruption factors, and trade­offs), which must be assessed separately for each project consistent with the rel­ evant simplified project flowchart. The base (exclusive of risk) project performance is determined by implementing the model with base input values; the mean (probability­ weighted average) project performance is determined approximately by implementing the model with mean input values. • Methods, tools (e.g., forms and an MS Excel template spe­ cifically for relatively simple rapid renewal projects), and guidance for assessing mean base inputs for a project. For successful risk identification, in addition to developing a risk checklist, the team developed methods, tools (e.g., forms and an MS Excel template specifically for relatively simple rapid renewal projects), and guidance for identifying a comprehensive and nonoverlapping set of project risks (including opportunities relative to the base scenario). The team also categorized risks by the project phase during which they are most likely to occur and developed the structure of an appropriate project risk register. For successful risk assessment, the team discussed available methods and tools for assessing the severity of identified risks. The team developed methods, tools (e.g., an MS Excel template), and guidance for calculating mean unmitigated (before additional risk management) project performance and its sensitivity (risk severity), using the same generic proj­ ect performance model as discussed above but considering all the identified risks and their severity, as well as the base. The team also developed methods, tools (e.g., forms and an MS Excel template), and guidance for assessing mean risk inputs (either values or ratings) for a project. For successful risk analysis, the team discussed available methods and tools, and developed guidance for conducting appropriate probabilistic performance and sensitivity analy­ sis, including assessment of probability distributions for (and correlations among) base and risk inputs for a project. This project focused on a qualitative assessment of risk; there are tools available for quantitative risk assessments. Special methods and tools required for this risk analysis were previ­ ously developed by the authors outside this research project. For successful risk management planning, the researchers developed the following: • Methods, tools (e.g., forms and an MS Excel template), and guidance for identifying and evaluating possible risk man­ agement actions, including cost–benefit analysis for evalu­ ating proactive individual risk reduction alternatives. In addition they developed a checklist for possible risk reduc­ tion actions (see the guide). • Methods, tools (e.g., an MS Excel template), and guid­ ance for calculating mean mitigated project performance and its sensitivity, using the same generic project per­ formance model as discussed above, but considering the cost­effectiveness of the selected set of risk management actions, as well as the risks and the base. A – lag (remaining) from finish of Environmental Permits to B – lag (remaining) to finish of Procurement C – lag (remaining) from finish of Environmental Permits to D – lag (remaining) to finish of ROW/Util/RR G – lag (non-overlap) after start of Final Design to start of Construction and H – lag (remaining) after finish of Final Design to finish of Construction I – lag (remaining) after finish of ROW/Util/RR to finish of Construction J – lag (remaining) from finish of ROW/Util/RR to K – lag (remaining) to finish of Procurement <D> Planning Scoping Enviro Proc, Prelim Design D/B Final Design Procure- ment D/B Con- struction Opera- tions Replace- ment Enviro Permits ROW, Util, RR 4 5 2 3 1 <E> <F> <K> <J> <I> <H><G><A><C> <B>Time Notes: 1,2,3 = funding 4 = project delivery 5 = replacement Enviro Proc = Environmental Process Util, RR = Utilities, Railroad Notes: <x> = lag E – lag (remaining) after finish of ROW Fund to finish of ROW/Utilities/RR F – lag (overlap) from finish of ROW/Util/RR to start of Construction Figure 2.3. Design–build (D/B) project phases.

19 • Contingency and recovery analysis on the basis of miti­ gated project performance analysis (either quantitative risk analysis or the approximate mean value approach for relatively simple rapid renewal projects) and desired levels of confidence (target percentiles). • Content requirements (i.e., an annotated outline) for an appropriate project risk management plan. For successful implementation of risk management, the team developed methods, tools (e.g., forms and an MS Excel tem­ plate), and guidance for monitoring and updating the risk man­ agement plan as the project proceeds, especially the following: • Base project performance (e.g., as contracts and change orders are established and actual costs and schedule are monitored); • Risk register (e.g., changes in severity of the risks, such as “retirement” of risks if they have not occurred by the end of a particular project phase); • Proactive risk reduction plans (e.g., their status, including actual implementation costs and effectiveness); and • Contingency and recovery plans (e.g., their status, includ­ ing remaining capacity and changes in requirements). In addition to development of a formal risk management process and set of performance objectives for rapid renewal projects, the researchers developed the following tools: • An appropriate generic overview of the risk management process for a risk facilitator to present at the beginning of a workshop, specifically for a relatively simple rapid renewal project. • A suitable set of paper forms to guide a risk facilitator through the various methods and tools, specifically for relatively simple rapid renewal projects. • An MS Excel workbook template (with a comprehensive user’s guide) to automatically document the inputs (con­ sistent with the paper forms) and to do the calculations, specifically for relatively simple rapid renewal projects. • Inventories of possible risks and their possible risk man­ agement actions for various rapid renewal methods to serve as checklists to help ensure comprehensiveness. • An annotated outline for a suitable risk management plan for most rapid renewal projects. • A complete hypothetical, relatively simple rapid renewal case study, including development of a complete risk man­ agement plan and quantitative risk analysis. The case study was evaluated by using all the other materials in this list. Risk Management Program Implementation of a risk management process, which can provide substantial benefits (e.g., in improved project perfor­ mance), requires a formal risk management program within a DOT. Such a program consists of the following elements: pol­ icy, procedures, organizational structure (roles, responsibili­ ties, authority, and resources), and an information network (for both gathering and distributing information). A key requirement of a risk management program is skilled staff who can organize workshops; lead workshops and sub­ sequently conduct analyses on relatively simple projects; write reports and plans, and monitor and update the risk reg­ ister as risks arise and are addressed; and on complex projects, supervise others who are leading workshops, conducting analyses, and the like. A 2­day course was developed to train DOT staff to be capable of accomplishing these tasks; the course was supplemented by, essentially, an apprenticeship program in which staff members become increasingly profi­ cient and ultimately independent. The training course, as well as the application of the methods and tools for risk manage­ ment planning, were successfully tested and finalized during two pilot workshops (as discussed in the next section). Once the Guide for the Process of Managing Risk on Rapid Renewal Projects and associated implementation materials were expanded and appropriately revised, they were submit­ ted to the SHRP 2 Expert Task Group (ETG) for review and approval. After receiving ETG approval, the team completed and submitted all the materials to SHRP 2 for review and approval. The subsequent pilot workshops (see Task 3) were conducted to test the guide and implementation materials, after which the research team refined the material based on feedback and lessons learned (under Tasks 3 and 4). task 3: Conduct pilot Workshops The scope of Task 3 of this research project included conduct­ ing two pilot workshops with state DOTs that used the draft guide and associated implementation materials. Feedback from these workshops would be the basis for finalizing the materials (under Task 4). The pilot workshops involved the following steps or subtasks (see Appendix E): 1. Plan pilot workshops. 2. Conduct first pilot workshop. 3. Evaluate first pilot workshop. 4. Conduct second (and final) pilot workshop. 5. Evaluate second (and final) pilot workshop. Initially, both workshops were structured to be 2 days long, with lectures the first day and application of the training materi­ als to an actual DOT rapid renewal project on the second day. Six state DOTs were initially identified as potential candi­ dates, being both interested and suitable for piloting a risk management workshop for rapid renewal. The six states were the following: Colorado Department of Transportation (CDOT), Florida Department of Transportation (FDOT),

20 Minnesota Department of Transportation (MnDOT), New York State Department of Transportation (NYSDOT), North Carolina Department of Transportation (NCDOT) and North Carolina Turnpike Authority (NCTA), and Virginia Department of Transportation (VDOT). A letter was sent to each of these DOTs inquiring into their interest and qualifications to pilot a risk management work­ shop for rapid renewal (see Appendix A for the contact infor­ mation of these DOTs). Five of the six DOTs expressed interest in piloting a risk management workshop for rapid renewal. The team developed a rigorous process for selecting two DOTs for the pilot workshops; this process was based on mul­ tiple characteristics of the project and the agency. To be fair in selecting from among the five remaining DOTs, a primary set of selection criteria was established. The team looked for the following: • High chance of success of that DOT’s workshop; • High contribution to the chance of success for that DOT’s future risk management program; and • High contribution of that DOT to the chance of success of a future national risk management program. The factors that contribute to meeting these primary crite­ ria were identified, and the specific information needed to assess those factors and evaluate the criteria was identified. Another letter was then sent to the five remaining DOTs requesting that information, which would allow the research team to evaluate their suitability for piloting a risk manage­ ment workshop for rapid renewal. Each of the DOTs provided a candidate project for the train­ ing. The responses of the DOTs to the request for information were then rolled up into each of the three categories and sub­ jectively graded, considering their suitability in that category (making some assumptions when information was missing). Scores were assigned to each grade (e.g., A = 4.0), and the cate­ gories were weighted to reflect their relative importance; these scores and weights were then combined to get a total weighted score for each DOT, with higher total weighted scores being preferred. The DOTs were then ranked on the basis of their total weighted scores. The sensitivity of these scores and rank­ ing was then evaluated. Based on this evaluation, NCDOT and FDOT came out clearly ahead and were initially selected for the pilot workshops. For a summary of the request for information and the scoring of the DOTs, see Appendix E. Subsequent to the selection of the two DOTs, the research team worked with both on logistics and, especially, on refin­ ing their projects. However, the primary purpose of the work­ shop was to train DOT staff, not to evaluate a project (a benefit that was merely a by­product). The first and second day targeted potential DOT risk management facilitators and subject matter experts who might be involved in future risk management assignments; the second day targeted relevant project staff for the example project. The training allowed a DOT to conduct simple risk management (develop a risk reg­ ister and a risk management plan) on relatively simple rapid renewal projects without external resources, or to supervise external resources in conducting more­detailed analyses and/or evaluating more complicated rapid renewal projects. A specific suitable (relatively simple) rapid renewal project was needed for the 1­day evaluation (on the second day of the workshop), to reinforce the training given on the first day. Such a project was expected to be in the $25 million to $100 million range and have a significant rapid renewal element, with one workshop evaluating an accelerated construction method and the other evaluating an innovative project deliv­ ery approach. The schedule did not allow for a risk analysis (i.e., quantification of uncertainty in project cost and sched­ ule) on the example project, which would generally take more than 1 day. However, the schedule did include development of a preliminary risk register and a risk management plan (among other things) that could subsequently be used and developed further in a full risk analysis. The first pilot workshop was held in Raleigh, North Carolina, for NCDOT/NCTA on October 29–30, 2009. The comments on the first workshop eventually led to a major change in the for­ mat of the training, eliminating evaluation of an actual project. After these changes had been made, the second pilot workshop was held in Redmond, Washington, on May 18–19, 2010, for representatives of various state and other DOTs. Subtask 3.1: First Pilot Workshop—NCDOT/NCTA The goal of the pilot workshop was to assist departments of transportation in understanding and applying risk manage­ ment techniques throughout the project development process, especially for rapid renewal projects, thus improving project performance. The approach was a synergy of theoretical prin­ ciples, practical tools for implementation, and guidance for using the results in decisions concerning construction­manage­ ment risk. The intended outcome of the workshop was a heightened awareness within the highway construction man­ agement community that risk can be understood and managed in a structured and cooperative way of doing business. Work­ shop organizers also hoped to spur development of an inde­ pendent capability within the department of transportation to accomplish this—either (a) actually doing the most important parts on relatively simple projects or (b) supervising others in doing the other parts (e.g., quantitative risk analysis) or in evaluating more complex projects. An MS Excel workbook template was provided to each participant and guided the user through the various steps of risk management, producing a risk register and parts of a risk management plan (RMP).

21 The workshop duration was 2 days. Day 1 consisted of lec­ tures and exercises from the guide to provide a fundamental understanding of the risk management process and how to do each of the important steps, including project “structuring” for risk management, risk identification, risk assessment, risk management planning, and subsequent implementation. (Note that only an overview of risk analysis was provided.) Day 2 involved a practical application of the tools and tech­ niques discussed on Day 1 for a preselected DOT project. In addition to providing education, the workshop resulted in a working risk register and parts of an RMP for that project. It was anticipated that NCDOT would subsequently implement those parts of the RMP (which includes the risk register) on the preselected project and might choose to use the risk reg­ ister in a subsequent quantitative risk analysis (e.g., to deter­ mine appropriate budgets and contingencies). This workshop, including the actual project evaluation, receives further coverage in Appendix E. About 25 NCDOT/NCTA staff, two SHRP 2 staff, and three workshop facilitators attended the first pilot workshop in Raleigh, North Carolina. A preselected NCDOT rapid renewal project was used on the second day of the workshop to dem­ onstrate the methods and tools taught on the first day. Appen­ dix E provides a description of the rapid renewal project in Topsail Island, North Carolina. It should be noted that only construction cost and schedule, not disruption during con­ struction or longevity, were of interest to the project team. A hypothetical project scenario used to document the assess­ ments and to automatically conduct the analyses is included in the guide (http://www.trb.org/Main/Blurbs/168369.aspx). Evaluations of First Pilot Workshop The research team requested feedback from both the NCDOT personnel and the SHRP 2 staff on the first pilot workshop. Although the feedback from the NCDOT participants was generally positive, comments from both them and SHRP 2 staff suggested a significant change to the format, which was to eliminate the application to a real DOT project on the second day and replace it with an expanded version of the illustrative example project from the guide. This change and several other revisions were made, and the team began planning for the sec­ ond pilot workshop. The first workshop is further evaluated in Appendix E. Course participants completed evaluations of the first pilot workshop (17 evaluations total), a summary of which is pre­ sented in Table E.3 in Appendix E. Average scores for each of the course evaluation questions range between 4 and 5 on a 1–5 scale (i.e., As and Bs). In summary, the comments reflected the assessment that it was a good course. However, participants found Day 1 to be overwhelming (too dense, too much theory especially in Module 5, too fast, too much lecture, and not enough interaction). In addition, the notebook needed better organization (e.g., the appropriate forms should have been with each module), and the forms were hard to read and understand. Participants indicated that the real project application on Day 2 clarified the process and material from Day 1 but should have been more integrated with that material. Under a separate contract (using no research project funds), the research team conducted another risk identification, assessment, and management workshop for NCDOT/NCTA on January 27–28, 2010. The focus was a confidential, large design–build project, using the methods and materials devel­ oped for this research project. Like the second day of the first pilot workshop, this workshop did not involve training; it pro­ vided only a short introductory overview of the risk man­ agement process. However, it did involve use of the process, methods, inventories, and template. This workshop was suc­ cessful and essentially validated the methods and materials (including the template and introductory overview presenta­ tion) developed for this research project. The project descrip­ tion, base cost and schedule, unmitigated risk register and performance, risk reduction plan development, and mitigated risk register and performance that were generated in the 1.5­day workshop proved to be very valuable to the project team; however, they are considered confidential and cannot be presented here. Nonetheless, feedback from this workshop was used to revise the template (and user’s guide and forms) and the introductory overview presentation before the second pilot training workshop. Revisions Based on the Evaluation of the First Pilot Workshop On the basis of the approved changes that resulted from feed­ back on the first pilot workshop, the guide and materials were extensively revised before the second pilot training workshop in the following manner: • The template (an MS Excel workbook) was revised and a user’s guide for the template was developed to improve the template’s functionality. • The hard­copy forms, which are intended to provide all the inputs needed by the template but are designed to be filled out quickly by hand, were revised to be consistent with the revised template and to improve their functionality. • A presentation (in MS PowerPoint) providing a relatively short overview of the simplified risk management process was developed, which would be appropriate for a facilitator to use as an introduction to a risk management workshop (possibly with some modification to customize it for each specific application). The slides contain significant anima­ tion (which can only be seen in presentation mode), as well

22 as significant notes (which can only be viewed completely in “notes page” view). • The hypothetical example for the guide and for the pilot training workshop was significantly expanded to include (among other things) a complete risk management plan (including an application of the template and an adden­ dum for quantitative risk analysis), to better illustrate con­ cepts and methods. • The syllabus (including an evaluation form) for the pilot training workshop was revised, as were the annotated slides (in MS PowerPoint) for the pilot training workshop, consistent with the revised guide and other materials. Further detail about revisions to the guide and materials is provided in Appendix E. Subtask 3.2: Second Workshop As noted earlier, the second (and final) pilot workshop was postponed pending resolution of comments on the first workshop and subsequent revision of the guide and materi­ als. The objectives of the second workshop were the same as the first workshop. The second workshop consisted of 2 days of lectures and exercises based on a hypothetical project. Time was allotted for discussion to provide a fundamental understanding of the risk management process and how to do each of the important steps, including project “structur­ ing” for risk management, risk identification, risk assessment, risk management planning, and subsequent implementation. Only an overview of quantitative risk analysis was provided. Because this workshop used a hypothetical project instead of a real project, members of a specific DOT project team were no longer needed in the workshop; only future risk managers (and, to a lesser extent, subject matter experts) who would apply the guide and materials to their DOT’s projects needed to attend. Because only a few such people from any DOT would attend, representatives from various DOTs were invited to attend the workshop to have a sufficient number of participants. Staff from selected state DOTs (Arizona, Cali­ fornia, Colorado, Florida, Hawaii, Iowa, Minnesota, Nevada, New York, North Carolina, Oregon, Utah, Virginia, and Washington) and Canadian provincial ministries of trans­ portation (Alberta, British Columbia, and Ontario), as well as FHWA and SHRP 2 staff, were invited to attend the workshop (see Appendix A for agency contact information). This pilot training workshop was conducted May 18–19, 2010, in Redmond, Washington, using the revised guide and materials. The course was ultimately attended by 13 staff from various organizations, including Washington, Minnesota, Nevada, and North Carolina DOTs, FHWA, SHRP 2, and a consultant for the Federal Transit Administration (FTA) (see Appendix E for list of organizational attendees). Evaluations of Second Pilot Workshop Feedback was requested from both the DOT participants and the SHRP 2 staff who attended the second pilot work­ shop. Only a few participants formally evaluated the course; in summary, the commenters said that the course was very good but needed some revisions. The standard flowcharts needed revision (to reflect that the environmental process is tied to preliminary design) and the practical exercise needed better implementation. They found the course still dense in places (e.g., regarding structuring exercise) and suggested that trainees would need “hand holding” for real applications. A summary of course participants’ evaluations is presented in Appendix E (only three evaluations total, because other attendees left before the end of the course and did not respond to subsequent requests). Average scores (which have limited value with such a small number of responses) for each of the course evaluation questions range between 3 and 5 on a 1–5 scale (i.e., As, Bs, and Cs). task 4: Finalize Guide and Implementation Materials Based on the feedback on the guide and associated implemen­ tation materials, the project team resolved the comments from pilot workshops and generally improved the materials. The team finalized and submitted the guide and the associated implementation materials to SHRP 2 on February 15, 2011 for approval. The final guide and implementation materials are described further in Appendix D. The materials are available at http://www.trb.org/Main/Blurbs/168369.aspx. task 5: Manage the Study The task of overall study management of the R09 research entailed coordinating research team members and the pro­ gram officer, as well as regularly communicating the project status. Managing the research team involved coordinating the technical Tasks 1–4 (including technical meetings) and sub­ mitting technical reports. The gap analysis and detailed plan report, as well as the final report, were developed under this task. The team also responded to a request for recommenda­ tions of additional future work. Throughout the duration of the project, regular commu­ nication was established through monthly and quarterly progress reports, interim meetings with staff, teleconferences, web meetings, and status reports and briefings at Technical Coordinating Committee meetings. The research team also completed miscellaneous SHRP 2–directed activities, such as presentations at conferences. More detail on study manage­ ment is found in Appendix B.