Guidebook on Alternative Quality Management Systems for Highway Construction (2015)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

68 Highway Project Quality Assurance Organization Selection Guide C.1 Objective This quality assurance organization (QAO) selection guide assists STAs in the assignment of project quality assurance (QA) roles through the selection of the most applicable project QAO. Ideally, selection will begin early in the project development process. At the latest, it should occur before the procurement of design or construction contracts begins. This guide provides the basic definitions for the project QAO selection tool, includes instructions on the use of the tool, and presents a demonstra- tion of the tool on a project for illustration. This appendix also includes the selection factor definitions, the factor appropri- ateness ratings, and all blank forms required to apply the tool. Electronic versions of these forms are available for download by searching for NCHRP Report 808 on the TRB website. C.2 Definitions An understanding of a few basic quality terms and the fundamental QAOs is necessary to ensure an accurate imple- mentation of the selection tool. Definitions for the basic quality terms and the fundamental QAOs follow. Please review the following definitions of quality terms prior to completing the selection of a project QAO: • Quality Management (QM). The totality of the system used to manage the ultimate quality of the design as well as the construction encompassing the quality functions described below as QA, QC, independent assurance, and verification (Gransberg, Datin, and Molenaar 2008). • Quality Assurance Organization (QAO). The assign- ment of the roles and responsibilities associated with the quality management of a project from concept through completion. • Quality Assurance (QA). All those planned and systematic actions necessary to provide confidence that a product or facility will perform satisfactorily in service (Transportation Research Circular E-C137 2009). • Quality Control (QC). Also called “process control,” those QA actions and considerations necessary to assess and adjust production and construction processes so as to con- trol the level of quality being produced in the end product (Transportation Research Circular E-C137 2009). • Acceptance. The process of deciding, through inspection, whether to accept or reject a product including what pay fac- tor to apply (Transportation Research Circular E-C137 2009). C.3 Fundamental QAOs The definition for each of the QAOs is provided below. Table C1 summarizes the quality roles and responsibilities in each QAO, and Figure C1 provides a spectrum of the QAOs and the associated level of control the agency has over the quality of the project. • Deterministic QAO. The traditional approach to quality within the highway industry. The agency retains respon- sibility over all project quality roles, responsibilities, and activities. • Assurance QAO. The agency is responsible for all aspects of quality except for design and construction QC. • Variable QAO. Design and construction take different approaches to quality. For example, the STA may assign both design phase QC and acceptance to an outside party, while the construction phase QC only may be assigned to an outside party. This approach was found on design-build (DB) projects. • Oversight QAO. The agency takes on an oversight role by assigning design QC, design acceptance, construction QA, and construction acceptance to outside parties. • Acceptance QAO. The agency is responsible only for veri- fication testing and final acceptance. All other quality roles and responsibilities are assigned to the concessionaire. This variation was found only in public-private partnership (PPP) arrangements. A P P E N D I X C

69 C.4 Project QAO Selection Tool The QAO selection tool can assist STAs in selecting the most appropriate QAO for a project by rating the appropriateness of the five fundamental QAOs according to the categories of project selection factors. The definitions for the 10 selection factors can be found in Section C.6. The tool uses a three-step process (see Figure C2): identifying barriers to QAO adoption, preparing a selection factor profile, and through analysis of the selection factor profile in conjunction with selection factor category/QAO appropriateness ratings, selecting the most appropriate QAO. The steps are discussed in more detail in the following sections. Step 1: Identifying Barriers to QAO Selection Barriers are regulations or policies that either prevent the use of an alternative QAO or dictate that a specific QAO be used on a project. Possible barriers include—but are not limited to—federal, local, or funding regulations; political issues; and agency policies. It is important to identify these barriers at the beginning of a QAO selection process because it is very likely that if barriers exist, the QAO selection process will begin and end at this step. For example, when a specific QAO is required, that QAO must be selected. Step 2: Preparing the Project QAO Selection Factor Profile The goal of the second step is to prepare a project QAO selection factor profile. The selection factor profile identifies which category of each selection factor applies to the project being analyzed. The information in the selection factor profile will be used in Step 3 to identify the appropriateness ratings for each selection factor that applies to the project. For some selection factors, such as project size or project delivery method, it is easy to identify which category applies. However, identifying the correct category for selection factors such as the amount of quality responsibility the agency wants to shift to other project participants requires that the project goals be established and understood so that the correct selection factor category can be determined. Establishing the project goals will also provide the user with further understanding of the motivation for the project and why the project might need to diverge from the agency’s default project QAO. A com- plete understanding of the project goals will ensure that the agency is making a fully educated decision. Once the goals are established, the user can complete the project QAO selection factor profile form included as Figure C5. Step 3: Using the Project QAO Analysis Form to Select an Appropriate QAO The final step of the QAO selection process is selecting the appropriate QAO based on a comprehensive analysis of the appropriateness ratings for each QAO that corresponds to the category of each of the project selection factors. In this step, the user transcribes the appropriateness ratings onto the project QAO analysis form from the project QAO selection factor profile form in Step 2. Appropriateness ratings for all categories of selection factors are included in Section C.7, and all forms are included in Section C.8. QAO Design Acceptance Design QC Construction Acceptance Construction QC Project Acceptance Deterministic Agency Agency Agency Contractor Agency Assurance Agency Designer Agency Contractor Agency Variable Designer Designer Agency Contractor Agency Oversight Designer Designer Contractor Contractor Agency Acceptance *Concess Concess Concess Concess Agency *Concess = Concessionaire Table C1. Summary of roles and responsibility assignments for each QAO. Deterministic Assurance Variable Oversight Acceptance High Low Level of Agency Control Over Quality Figure C1. Spectrum of QAOs and level of agency control over quality.

70 Local regulaons Federal regulaons Funding regulaons Use tradional agency QAO Yes No Pr oj ec t d el iv er y m et ho d Pr oj ec t co m pl ex ity QAO selected QAO selected Yes No QAO selectedYes No Is an excepon allowed No Yes Barrier idenficaon Selecon factor preparaon QAO selecon Q A sh i  a w ay fro m a ge nc y Pr oj ec t s ize Sc he du le ri sk Ag en cy cu ltu re In du st ry a bi lit y Ag en cy ’s tr us t i n in du st ry Ag en cy st aff ex pe rie nc e No Yes Ag en cy st aff av ai la bi lit y Any regulaons prohibing alternave QAO Does this determine the QAO Do any of these dictate the QAO Any policies prevenng alternave QAO Set project goals and define project selecon factors Figure C2. Project QAO selection flow chart.

71 The four ratings used to indicate the appropriateness of the factors are the following: • Fatal flaw (denoted with ×) • Least appropriate (-) • Appropriate (+) • Most appropriate (++) The fatal flaw rating (×) indicates that, for that particular category of selection factor, the QAO has the potential to harm the success of the project. The fatal flaw rating effec- tively eliminates that QAO from further consideration. A least appropriate rating (-) indicates that, for the particular selec- tion factor category, the QAO can work, but it is not the best option. If a QAO with a least appropriate rating is imple- mented, there may be extra measures needed to accommo- date this particular selection factor. An appropriate rating (+) indicates that the QAO can work for that particular selection factor category—it neither harms nor improves the success of the project. Finally, the most appropriate rating (++) indicates that a project can be improved by the implementation of the associated QAO. The project QAO selection analysis form is split into two separate sections: primary selection factors and secondary selection factors. The primary factors are all the selection factors that resulted in at least one fatal flaw rating during the NCHRP Project 10-83 research and testing. Secondary factors are the selection factors that did not result in a fatal flaw. Due to the diversity of the appropriateness ratings, primary factors have a more decisive role in project QAO selection. If the potential QAOs are not narrowed down to two options or fewer at the completion of the primary selection factors analysis, then the same process is used for the secondary selection factors. Also, if the potential QAOs are narrowed down to two or fewer in the primary factor analysis, the user can continue on to the second- ary selection factors to further understand the potential QAOs, investigate the details of each project selection factor rating, and/or make a final selection of the project QAO. This section has presented the three steps of the project QAO selection tool: barrier identification, selection factor preparation, and QAO selection. The following section dem- onstrates the use of the tool by presenting an implementation of the tool with an actual industry project. C.5 Demonstration Project A highway project in a state in the mid-section of the United States was selected to demonstrate and validate the project QAO selection process tool. The exact project location is not identified for reasons of anonymity in the research. The scope of the project was to construct a landmark river bridge(s); rehabilitate or replace approximately 4 miles of Interstate; improve traffic operations, geometrics, and safety; and add mainline capacity. The budget for the DB project was approximately $230 mil- lion. The STA selected the DB method to achieve the project goals: • Reduce/compress/accelerate project delivery period. • Get early construction contractor involvement. • Encourage innovation. • Complete different design solutions through the proposal process. • Address flexibility needs during the construction phase. This STA has been very open to trying new delivery meth- ods and does have a focus on shifting more quality respon- sibilities away from the agency. The agency has experienced staff reductions over the past decade and significant losses of expertise through retirements. The agency and the local contracting and engineering industry have built up high levels of mutual trust as a result of increasing use of alterna- tive delivery methods in the state. The first step of the project QAO selection tool is identify- ing barriers to the implementation of alternative project QAOs. The demonstration project is in a state that is leading the way in shifting quality responsibility away from the agency, and no state or local barriers preventing alternative QAO selection were identified. Also, no federal regulations pertain to this project that would prohibit the implementation of an alternative QAO. The agency itself is relatively progressive and encourages try- ing new processes and strategies that can improve projects and overall efficiencies. As such, no agency polices exist that prevent alternative QAO implementation. The result of the first step is that all project QAOs are still viable for this project. The second step of the project QAO selection tool is complet- ing the project QAO selection factor profile form. The project goals were already established, so the selection factor profile form was completed to show which category of each selection factor cor responded to the project (see Figure C3). The com- pleted selection factor form is used in Step 3. The third and final step in the project QAO selection tool is using the project QAO analysis form to select the project QAO. This form is completed by transcribing the appropriateness ratings for each QAO to the category of selection factor appli- cable to the project. The completed project QAO analysis form for the demonstration project is presented in Figure C4. In this case, the Deterministic, Assurance, and Variable QAOs had fatal flaw ratings in at least one of the primary selection factors and, as such, they were deemed inappropriate. This left two potential QAO options, Oversight and Acceptance. The secondary selection factors for these QAOs were analyzed to further understand the two possible QAOs so a final QAO decision could be made with confidence and the appropriate considerations.

Primary Selection Factors Deterministic Assurance Variable Oversight Acceptance Agency staff availability x – + ++ ++ Trust between agency and industry + ++ ++ ++ ++ Industry’s ability to manage its own quality + + + + + Project delivery method – – + ++ – Project size – + + ++ ++ Shift the quality responsibility away from the agency x x x ++ ++ Tally of primary selection factor results x x x ++ ++ Secondary Selection Factors Deterministic Assurance Variable Oversight Acceptance Project complexity n/a n/a n/a + + Project schedule sensitivity n/a n/a n/a ++ ++ Agency culture n/a n/a n/a + + Agency staff experience n/a n/a n/a ++ ++ Tally of secondary selection factor results n/a n/a n/a ++ ++ Rating key: x Fatal Flaw – Least Appropriate + Appropriate ++ Most Appropriate Figure C4. Completed project QAO analysis form for the demonstration project. Figure C3. Demonstration project QAO selection factor profile form. Primary factor categories Agency staff availability Minimal Moderate Full Trust between agency and industry Low Moderate High Industry’s ability to manage its own quality Low Medium High Project delivery method DBB DB CMGC PPP Project size <$10M $10M–$50M $50M–$500M $500M–$2B >$2B Shift the quality responsibility from the agency None Some QC Some QA Some QC and Some Acceptance All Secondary factor categories Project complexity Low Medium High Project schedule sensitivity Low Medium High Agency culture Traditional Moderate Progressive Agency staff experience <5 years 5–10 years 10–20 years >20 years

73 The project delivery method selection factor can provide an example of how an appropriateness rating is determined. The project delivery method for the demonstration project is DB. The appropriateness ratings for each QAO for the DB category were transcribed onto the project QAO analysis form (see Figure C4) in the project delivery method row. Both Deterministic and Assurance QAOs are rated as least appro- priate, but are not fatal flaws. Deterministic and Assurance QAOs can be used with DB on small, non-complex projects or when an agency has little experience with DB project delivery. This is not the case in this example. The Acceptance QAO can shift too much responsibility away from the agency in a DB project and, as a result, also has a least appropriate rating. The Oversight QAO is considered the best fit because the design builder is responsible for delivering both the design and con- struction of the project while the agency is responsible for clearly stating the requirements for the project and is not involved in the day-to-day management of project design or construction. The Oversight QAO allows the agency to ensure that the design builder is meeting the requirements of the project. The project QAO selection tool indicated that the Oversight and Acceptance QAOs are the most appropriate for the project. Because projects and agencies are unique, in cases like this it is up to the agency to decide whether the Oversight or Acceptance QAO would be the best fit for the project. At the time this tool was developed, the demonstration project was already well into construction, so the tool could not be used to select the QAO for the project. However, the Oversight QAO, which is one of the options indicated by the QAO selection tool, was implemented for the demonstration project and was proving to be successful. C.6 Factor Definitions This section provides the definitions of the 10 QAO project selection factors: • Agency staffing ability. The quantity of agency project staff available to be committed to the project as compared to traditional levels. • Trust between the agency and the industry. The level of agency confidence that project decisions will be based on achieving the best results for the project, rather than the individual or specific company. This requires the agency and industry to overcome the long-standing adversarial para- digm of the project participants (designer, engineer, con- tractor, consultant, and agency). • Industry’s ability to manage its own quality. The local industry’s level of competence in managing its own quality. The industry includes both the design and construction communities. Competence can be increased through expe- rience, training, education, industry culture, or a combina- tion of any of these. • Project delivery method. The comprehensive process by which designers, constructors, and various consultants pro- vide services for design and construction to deliver a com- plete project to the owner. While names can vary in the industry, and owners often create hybrid delivery methods, there are essentially three primary project delivery methods: DBB, CMGC, and DB. • Project size. The total dollar value of the project’s design and construction budgets. • Quality responsibility shifted away from the agency. The amount of liability for the management of the project’s qual- ity that the agency wants to shift to another project partner (contractor, designer, engineer, design builder, construction manager/general contractor [CMGC], or concessionaire). • Project complexity. The intricacy of a project’s scope as compared to a typical project in the same locale—stemming from programming requirements, design constraints, con- struction methods, site conditions, budget and funding constraints, quality requirements, and so forth. • Project schedule sensitivity. The vulnerability of the project schedule to changes due to delays, conflicts, and/or events outside of the designer’s and/or contractor’s control, such as coordination of observations, inspections, and/or testing performed by the agency. • Agency culture. In this context, agency culture refers to the agency’s attitude toward the implementation of change in project management techniques. • Agency staff experience. The average number of years of experience of the agency staff committed to the project. C.7 Factor Appropriateness Ratings Sheets This section presents the factor appropriateness ratings sheets, which are based on expert input to the research pro- cess. Please refer to the final research report for NCHRP Proj- ect 10-83, published as NCHRP Web-Only Document 212, for a full explanation of the data collection and analysis method used to produce these ratings sheets. There are 10 “sheets” (presented as Tables C2 through C11), one for each selection factor. Each sheet presents ratings of different combinations of selection factor categories and QAOs. Table C2 shows the appropriateness ratings for combina- tions of the five QAOs with the three categories (fully staffed, moderately staffed, and minimally staffed) of the availability of agency project staff selection factor: • If a full staff is available for a project, as compared to typical past projects, the Deterministic QAO is appropriate because it requires a large staff to manage the day-to-day quality needs of the project: inspection, observation, materials testing, and so forth. The Acceptance QAO is rated as a fatal flaw in

74 this case because these resources will be underutilized due to the fact that the Acceptance QAO shifts the bulk of the quality responsibilities away from the agency. • A moderately staffed project, as compared to typical proj- ects, is best suited to the Assurance, Variable, and Oversight QAOs. Selecting which of these three to implement in a moderately staffed project is dependent on the goals and other requirements of the project. • Deterministic and Acceptance QAOs are best suited for opposite extreme ends of the agency project staff availability spectrum. • Acceptance and Oversight QAOs are both most appropriate for a project that has minimal staff, while the Determinis- tic QAO is a fatal flaw. A minimally staffed project doesn’t allow agency project staff the time to manage the day-to-day quality needs of a project, which is exactly the use for both the Oversight and Acceptance QAOs. Table C3 shows the appropriateness ratings for combinations of the five QAOs with the three categories (low, moderate, and high) of the trust between agency and industry selection factor: • As the amount of quality responsibility shifts away from the agency, the amount of collaboration among all the project team members increases. This is directly reflected in the amount of trust that is needed between the agency and industry. • The Acceptance QAO shifts the largest amount of quality responsibility away from the agency. Without trust between the agency and the industry, it is difficult to implement the Acceptance QAO, which is why it is a fatal flaw. • All levels of trust are appropriate for the Deterministic QAO; however, it is the only one that is highly appropriate for a low level of trust because the agency is responsible for all elements of quality, requiring the rest of the project team to react to the agency. Table C4 shows the appropriateness ratings for combina- tions of the five QAOs with the three categories (low, medium, and high) of the industry ability to manage its own quality selection factor: • When shifting responsibility for quality away from the agency, it is critical that the party receiving the responsi- bility has the ability to successfully meet the responsibility. Ensuring this could require additional training, education, and/or resources on the part of the party receiving the new responsibility. • The “industry” in this selection factor is meant to be the local design, consulting, and/or contracting community. • Additional requirements or qualifications may need to be included in the RFP to ensure that the parties proposing on the project can manage the level of quality responsibility successfully. • As the quality responsibility shifts away from the agency, the importance of succinctly stating the quality require- ments in the RFP, specification, and contract documents increases. Selection factor category Deterministic Assurance Variable Oversight Acceptance Trust between agency and industry Low ++ + + − x Moderate + + + + + High + ++ ++ ++ ++ Table C3. Appropriateness ratings of QAOs for various categories of trust between agency and industry. Selection factor category Deterministic Assurance Variable Oversight Acceptance Availability of agency project staff Fully staffed ++ + + + x Moderately staffed − + + + − Minimally staffed x − + ++ ++ Table C2. Appropriateness ratings of QAOs for various categories of agency project staff availability.

75 Table C5 shows the appropriateness ratings for combinations of the five QAOs with the four categories (DBB, DB, CMGC, and PPP) of the project delivery method selection factor: • As the amount of project responsibility shifts away from the agency—i.e., DBB to PPP—the amount of project quality responsibility shifts away from the agency, from the Deter- ministic QAO to the Acceptance QAO, allowing both the project responsibilities and the quality responsibilities to remain in sync. • The fatal flaw rating corresponds to the implementation of the Deterministic QAO on a PPP project, because the Deterministic QAO requires the agency to retain all QA control. However, in PPP projects, almost all QA over the project shifts away from the agency to the concessionaire. • DB shifts much of the project responsibility to the design builder at an early stage of the project. In order for the design builder to most effectively manage the quality of the work, the majority of the quality responsibilities need to be shifted as well; this is why the Oversight QAO is most appropriate. • DB is least appropriate for the Deterministic and Assurance QAOs because the amount of project responsibility shifted to the design builder does not match the amount of quality responsibility that is shifted. However, the Assurance QAO has been used on DB projects because of the discomfort some agencies feel with transferring so much project and quality responsibility to one design builder. This can stem from an agency’s inexperience in DB and/or alternative project QAOs. Table C6 shows the appropriateness ratings for com- binations of the five QAOs with the five categories (<$10M, $10M–$50M, $50M–$500M, $500M–$2B, and >$2B) of the project size selection factor: • As project size increases, the appropriate QAOs shift from Deterministic toward Acceptance. As a project becomes larger in size the complexity increases, the need for agency resources increases, and the risk also increases. Increasing size most frequently requires the agency to shift some of the agency quality responsibility to other project participants. • The Deterministic QAO is rated as a fatal flaw for proj- ects over $500 million primarily because of the inherent complexity of such projects, the requirement for expertise outside of the agency, and the amount of risk on the proj- ect. Additionally, the Deterministic QAO is agency staff intensive. As a project grows in size, the demand for agency resources grows. Selection factor category Deterministic Assurance Variable Oversight Acceptance Industry ability to manage its own quality Low ++ + + − x Medium + + + + + High −* + + ++ ++ *Considered in conjunction with the other factors as the research did not reach statistical significance on this rating. Table C4. Appropriateness ratings of QAOs for various categories of industry ability to manage its own quality. Selection factor category Deterministic Assurance Variable Oversight Acceptance Project delivery method DBB ++ + + +* − DB − − + ++ − CMGC − + + ++ + PPP x − − + ++ *Considered in conjunction with the other factors as the research did not reach statistical significance on this rating. Table C5. Appropriateness ratings of QAOs for various categories of project delivery method.

76 • Acceptance is not appropriate for projects under $10 million. The primary reason is that these projects are “standard,” and it would not be worth creating the infrastructure to support a non-standard Acceptance QAO. However, if the agency already has ability to implement the Accep- tance model, has past experience with the Acceptance QAO, and has the infrastructure in place to manage the Acceptance QAO, there is nothing prohibiting the Accep- tance QAO from being implemented on projects under $10 million. • Assurance is not appropriate for projects over $500 million because it does not adequately meet the needs associated with the inherent complexity of such a project and the need to allocate risk to different parties on large projects. • The Variable QAO is flexible and can be appropriate for all project sizes. Table C7 shows the appropriateness ratings for combina- tions of the five QAOs with the five categories (all, some accep- tance and some QC, some acceptance, some QC, and none) of the shifting quality assurance risk away from the agency selection factor: • The categories of shifting the quality responsibility away from the agency essentially track exactly with the definitions of the fundamental QAOs. For example, by definition, the Deterministic QAO assigns all QA to the agency. It is the equivalent to shifting none of the quality risk away from the agency. • The Deterministic, Assurance, and Variable QAOs still have the agency managing aspects of the day-to-day qual- ity needs of the project. As a result, each of them is a fatal flaw if the agency desires to shift all quality responsibility. • The Oversight and Acceptance QAOs shift, at a minimum, the day-to-day management of quality away from the agency; therefore, if the agency desires to shift none of the quality responsibility to other project team members, then each of these QAOs is a fatal flaw. • Assurance and Variable QAOs shift at least some of the project quality responsibility away from the agency; Selection factor category Deterministic Assurance Variable Oversight Acceptance Shi quality responsibility away from the agency All x x x ++ ++ Some acceptance and some QC − − ++ ++ + Some acceptance −* − + ++ +* Some QC +* + + ++ x None ++ − − x x *Considered in conjunction with the other factors as the research did not reach statistical significance on this rating. Table C7. Appropriateness ratings of QAOs for various categories of shifting quality assurance risk away from the agency. Selection factor category Deterministic Assurance Variable Oversight Acceptance Project size <$10M ++ ++ + +* − $10M–$50M ++ ++ + + + $50M–$500M − + + ++ ++* $500M–$2B x − + ++* ++ >$2B x − + ++* ++ *Considered in conjunction with the other factors as the research did not reach statistical significance on this rating. Table C6. Appropriateness ratings of QAOs for various categories of project size.

77 therefore, if an agency goal is to retain all quality respon- sibility, then the Assurance and Variable QAOs are less appropriate. Table C8 shows the appropriateness ratings for combina- tions of the five QAOs with the three categories (low, medium, and high) of the project complexity selection factor: • As project complexity increases, the amount of expertise needed from outside the agency tends to increase. As a result, the agency no longer has the expertise required to ensure project quality. As the complexity of a project increases, the most appropriate QAO shifts from Deterministic toward Acceptance. • A low-complexity project is most appropriate for a Deter- ministic QAO because the expertise needed typically resides in the agency. • A highly complex project will require more and more expertise from outside of the agency, resulting in the agency needing to be able to communicate the quality requirements effectively. • If a project has only a few complex items, it may be that the QAO for those elements is different from QAO for the remainder of the project that is more along the lines of a typical project (such as special materials or a construction sequencing item). Table C9 shows the appropriateness ratings for combina- tions of the five QAOs with the three categories (low, medium, and high) of the schedule sensitivity selection factor: • Schedule sensitivity is not a decisive factor in the selection of a project QAO unless the schedule is highly sensitive to delays resulting from quality coordination issues among varying members of the project team. • Schedule sensitivity specifically comes into play when work is being conducted around the clock, and there is no float in the schedule. • Schedule sensitivity can be reduced if a good quality plan and communication plan has been agreed to among all parties involved in the day-to-day quality of the project (design and construction). Table C10 shows the appropriateness ratings for combinations of the five QAOs with the three categories (traditional, moder- ate, and progressive) of the agency culture selection factor: • Regardless of the project QAO, the agency provides the leadership for the project and ultimately dictates the cul- ture of the project. The agency culture has to be aligned with the project QAO. The more alternative a project QAO is (as compared to the traditional Deterministic QAO), the greater the need for a progressive agency culture. Selection factor category Deterministic Assurance Variable Oversight Acceptance Project complexity Low ++ + + + + Medium + + + +* + High − + ++ ++ ++ *Considered in conjunction with the other factors as the research did not reach statistical significance on this rating. Table C8. Appropriateness ratings of QAOs for various categories of project complexity. Selection factor category Deterministic Assurance Variable Oversight Acceptance Schedule sensivity Low + + + + + Medium − + + + + High − + + ++ ++ Table C9. Appropriateness ratings of QAOs for various categories of schedule sensitivity.

78 • The agency culture cannot be manifested by only a few of the project staff; it has to be instituted throughout the agency. If the project team is progressive but the executive level of the agency is traditional, it will be difficult for the project team to implement any alternative QAOs. • A moderate culture indicates that the agency is not conver- sant in alternative QAOs but is willing to try new ideas tested out by other agencies. Because there is some acceptance of new ideas, a moderate culture is appropriate for all QAOs. Table C11 shows the appropriateness ratings for combi- nations of the five QAOs with the four categories (<5 years, 5–10 years, 10–20 years, and >20 years) of the agency project staff experience selection factor: • The most appropriate level of experience for all QAOs is 10 to 20 years. However, the experience is used in different ways across the different QAOs. The Deterministic QAO applies the experience to do more effective inspections whereas the Acceptance QAO applies the experience to create the quality requirement details, identify flaws in the quality plans, and resolve any quality issues that may arise. • Fewer than 5 years of experience is not appropriate for the Oversight and Acceptance QAOs because both of these organizations require the agency staff to be well versed in quality for all elements of the project. This experience can only be achieved through time in the field. • The experience levels shown in Table C11 represent the average for all of the agency staff. In general, there needs to be a combination of more experienced staff with less experi- enced staff, which is why the 10-to-20-year experience level is the most appropriate for all QAOs. • The experience considered in this selection factor is pri- marily project or field experience. When shifting to a more alternative QAO, such as Oversight or Acceptance, this experience may need to be complemented with training on how to manage the quality process at a higher level, away from the day-to-day level of management. • As the amount of quality responsibility shifts to other project participants, the role of the agency shifts toward a role of managing requirements. This shift can be difficult for some agency staff and can require additional training, education, and/or resources for them to successfully take on the new role. Selection factor category Deterministic Assurance Variable Oversight Acceptance Agency project staff experience <5 years + + + − −* 5–10 years + + + + + 10–20 years ++* ++ ++ ++ ++ >20 years + + ++ ++ ++ *Considered in conjunction with the other factors as the research did not reach statistical significance on this rating. Table C11. Appropriateness ratings of QAOs for various categories of agency project staff experience. Selection factor category Deterministic Assurance Variable Oversight Acceptance Agency culture Tradional ++ + − − − Moderate + + + + + Progressive − + + ++ ++ Table C10. Appropriateness ratings of QAOs for various categories of agency culture.

79 Primary factor categories Agency staff availability Minimal Moderate Full Trust between the agency and the industry Low Moderate High Industry’s ability to manage its own quality Low Medium High Project delivery method DBB DB CMGC PPP Project size <$10M $10M–$50M $50M–$500M $500M–$2B >$2B Shift the quality responsibility away from the agency None Some QC Some Acceptance Some QC and Some Acceptance All Secondary factor categories Project complexity Low Medium High Project schedule sensitivity Low Medium High Agency culture Traditional Moderate Progressive Agency staff experience <5 years 5–10 years 10–20 years >20 years Figure C5. Project QAO selection factor profile form. Primary Selection Factors Deterministic Assurance Variable Oversight Acceptance Agency staff availability Trust between agency and industry Industry’s ability to manage its own quality Project delivery method Project size Shift the quality responsibility away from the agency Tally of primary selection factor results Secondary Selection Factors Deterministic Assurance Variable Oversight Acceptance Project complexity Project schedule sensitivity Agency culture Agency staff experience Tally of secondary selection factor results Rating key: x Fatal Flaw – Least Appropriate + Appropriate ++ Most Appropriate Figure C6. Project QAO analysis form.

80 C.8 Project QAO Selection Tool Forms Project QAO Selection Factor Profile Form Figure C5 presents the project QAO selection factor pro- file form (an electronic version of this form is available for download by searching on NCHRP Report 808 on the TRB website). Select the specific category for each selection factor that applies to your project. Factor definitions are included in Section C.6. Project QAO Analysis Form Using the category of each factor that applies to your proj- ect, look up the value for each factor in the factor appropri- ateness ratings sheets provided in Section C.7 (see Tables C2 through C11) and fill in the corresponding ratings in the project QAO analysis form shown in Figure C6 (an electronic version of this form is available for download by searching on NCHRP Report 808 on the TRB website). If you are not able to select a project QAO using only the primary selection factors, then continue with the same process for the secondary selection factors. C.9 Appendix C Bibliography Gransberg, D. D., J. Datin, and K. Molenaar, NCHRP Synthesis 376: Quality Assurance in Design-Build Projects, Transporta- tion Research Board of the National Academies, Washing- ton, D.C., 2008, 130 pp. Transportation Research Circular E-C137: Glossary of Highway Quality Assurance Terms (Fourth Update). Transportation Research Board of the National Academies, Washington, D.C., 2009. http://onlinepubs.trb.org/onlinepubs/circulars/ ec137.pdf