Alternative Quality Management Systems for Highway Construction (2015)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

CHAPTER 5: QAO SELECTION AND TOOLS 5.1 Introduction The selection of a QAO should occur as early in the project development process as possible. At a minimum, it should be completed before any procurement of design and/or construction begins. Inclusion of the QAO in the procurement process allows for the RFQ, RFP and/or IFB to define the project QAO so that the responding party can appropriately account for costs, risks or staffing requirement. In implementing any QMS, project and quality managers need tools and procedures to implement a quality management plan. This chapter presents a set of tools of various types, which are potentially valuable aids in a quality management plan. These include items for inclusion with procurement documents, a contract agreement, or quality management plans. This chapter provides a brief description of these tools and how to select a tool based on individual project characteristics. This chapter describes the background and development of the QAO and tool selection procedures that are described in the accompanying guidebook. This chapter summarizes the methodologies, data collection and validation that were performed in the research to arrive at the selection procedures in the guidebook. 5.2 QAO Selection Process Development Appropriate project QAO selection requires an analysis of the factors that influence the selection and their relationship factors to each QAO. Identifying the factors involved interviews with agency project staff from 23 projects in 13 states. The interview process had two goals, first to identify the factors and second, to confirm that there is not a process currently in place for an agency to select a project QAO when the default QAO is not appropriate for projects. This was confirmed, and the interviewees reported that most often when an alternative QAO is needed (i.e., the Deterministic QAO is not appropriate) for a project, the selection is left up to the project team without guidance or a standardized, transparent decision process. 5.2.1 Identifying QAO Selection Factors Ten factors that influence the selection of a project QAO were identified through interviews. The ten factors fell into three categories: project, agency, and industry. Any factors that were a condition of circumstances occurring after the RFP process, such as the experience of the contractor’s project management staff, were excluded because this information is unknown at the time QAO selection for a project takes place. 5.2.2 Project Factor Categories Project factors are characteristics of specific projects. The four factors that influence the selection of a QAO are: project size, project complexity, project schedule sensitivity, and project delivery method. Project size is determined by the budget of the project including both design and construction. Project complexity is related to how similar the project is to a typical project. 127

Complexity can result from characteristics including project scope, design requirements and constraints, construction methods, site conditions, budget and funding constraints, quality requirements, project delivery method, and specialty materials. Project schedule sensitivity refers to the vulnerability of the project schedule to changes due to delays, conflicts, and/or events outside of the designer and/or contractor’s control, such as coordination of observations, inspections and/or testing performed by the agency. Project delivery methods is “the process by which a construction project is comprehensively designed and constructed for an owner including project scope definition, organization of designers, constructors and various consultants, sequencing of design and construction operations, execution of design and construction, and closeout and start-up” (Touran et al. 2011). 5.2.2.1 Agency Factors Agency factors are characteristics and abilities of SHAs that are responsible for projects. The four agency factors are culture, staffing availability, staffing experience and the amount of quality responsibility the agency wants to shift to another project participant. The culture of the agency is the agency’s attitude toward the implementation of change in project management techniques. Agency staffing availability stems from the SHAs across the nation being downsized, and is determined by the quantity of agency staff available to commit to projects as compared to the traditional levels of agency staffing for comparable projects. Agency staffing experience is the average number of years of experience of the agency staff committed to the project. The amount of quality shift away from the agency has to do with shifting responsibility for quality to another project participant. These shifts refer to the amount of liability for the management of the project’s quality that an agency wants to shift to another project partner (e.g., contractor, designer, engineer, design builder, CMGC, concessionaire). 5.2.2.2 Industry Factors Industry factors are characteristics or abilities of local design, engineering, contracting and consulting communities. The two industry factors are the industry’s ability to manage their own quality and the level of trust established between the industry and the agency. The industry’s ability to manage their own quality refers to the local communities’ levels of competence in managing their own quality. This competence may result from either education, training, experience, industry culture or a combination of these. The level of trust between the industry and agency is important because as agency control over a project is reduced, increased levels of trust are required because the project becomes more collaborative. Effective collaboration depends on an agency’s level of confidence that project decisions made by industry partners will be based on achieving the best results for the project, rather than on the partners’ interests. The next step involved is establishing relationships between each of the selection factors and each fundamental QAO to understand how the selection factors influence project QAO selection. 5.2.3 Establishing Relationships Between Selection Factors and QAOs A Delphi study was conducted to establish the relationships between the ten selection factors, and the five fundamental QAOs. A panel of 12 experts rated the appropriateness of each QAO to each category of selection factor. Each expert was required to have a minimum of 15 years of 128

industry experience. Four appropriateness ratings were used for the Delphi study, fatal flaw (denoted with X), least appropriate (–), appropriate (+), and most appropriate (++). After three rounds of the Delphi study 93% of the ratings had reached consensus in the Delphi study; the remaining 7% either had an outlier or were torn between two ratings that included appropriate and a rating on either side of appropriate. The selection factors that did not reach consensus require consideration in conjunction with the other factors; these factors are marked with * in the summary table 5-1. Table 5-1 – Selection factor appropriateness rating sheet Selection factor category Determin. Assure. Var. Over. Accept. Project delivery method Design bid build ++ + + +* - Design build - - + ++ - CMGC/CMAR - + + ++ + P3/DBOM X - - + ++ Project size <$10M ++ ++ + +* - $10M - $50M ++ ++ + + + $50M-$500M - + + ++ ++ * $500M - $2B X - + ++* ++ >$2B X - + ++* ++ Availability of agency project staff fully staffed ++ + + + X moderately staffed - + + + - minimally staffed X - + ++ ++ Industry ability to manage their own quality Low ++ + + - X Medium + + + + + High -* + + ++ ++ Trust between agency and industry Low ++ + + - X Moderate + + + + + High + ++ ++ ++ ++ Shift the quality risk away from the agency All X X X ++ ++ Some QA and some QC - - ++ ++ + Some QA - * - + ++ +* Some QC + * + + ++ X None ++ - - X X Project complexity 129

Selection factor category Determin. Assure. Var. Over. Accept. Low ++ + + + + Medium + + + +* + High - + ++ ++ ++ Schedule sensitivity Low + + + + + Medium - + + + + High - + + ++ ++ Agency project staff experience <5 years + + + - - * 5 years - 10years + + + + + 10 years - 20 years ++* ++ ++ ++ ++ >20 years + + ++ ++ ++ Agency culture Traditional ++ + - - - Moderate + + + + + Progressive - + + ++ ++ *needs to be considered in conjunction with the other factors. The selection factors and the appropriateness ratings presented in this section form the basis for the development of the project QAO selection process tool with the intent of providing guidance, transparency, and understanding to the process. 5.2.4 The QAO Selection Process Tool The goal of the QAO selection process is to help SHAs identify the most appropriate QAO for projects by rating the appropriateness of the five fundamental QAOs according to the categories of selection factors that apply to the projects. This research suggests that project QAOs be selected before the RFQ, RFP or IFB process for design or construction begins so project quality roles and responsibilities can be accurately accounted for the procurement. The QAO selection process tool uses a three-step process for selecting the most appropriate QAOs for a particular project (Figure 5-1). The three steps are identifying barriers to QAO adoption, using the selection process profile form to prepare a selection factor profile, and using the QAO factor analysis form to select the most appropriate QAO. 130

Local regulations Federal regulations Funding regulations Any regulations prohibiting alternative QAO Use traditional agency QAO Yes No Pr oj ec t d el iv er y m et ho d Pr oj ec t co m pl ex ity Does this determine the QAO QAO selected QAO Selected Yes Do any of these dictate the QAO No QAO selectedYes No Any policies preventing alternative QAO Is an exception allowed No Yes Barrier identification Selection factor preparation QAO Selection Q A sh ift a w ay fro m a ge nc y Pr oj ec t s ize Sc he du le ri sk Ag en cy C ul tu re In du st ry a bi lit y Ag en cy ’s tr us t i n in du st ry Ag en cy st af f ex pe rie nc e Set project goals and define project selection factors No Yes Ag en cy st af f av ai la bi lit y Figure 5-1 - Project QAO selection process 131

The QAO selection steps are discussed in detail in Chapter 4 of the accompanying guidebook. The guidebook provides an example project application for illustration. Additionally, the guidebook contains an appendix with forms for applying the process. The forms include instructions which have been tested and validated for use. 5.2 Tools for Alternative QMS An extensive literature review and the case study evaluations assisted with discovering the tools for alternative QMSs. These tools encourage effective quality management across a variety of QMSs, including both the baseline and the alternative systems. The purpose is to provide a matrix of each tool and each tool provides information on what it is, why use it, what it does, suggestions on when to use it, how to use it, examples case studies that used the tool, and helpful references. Appendix B of the guidebook provides a more in-depth discussion of each tool. This guidebook subdivides the tools into two major categories, pre-award and post-award. Additional tool subcategories under these primary categories aid in their selection and ultimate application to the various phases and parties that make up a project. 5.2.1 Pre-Award tools Incorporating pre-award quality management tools occurs before selecting the design or construction team. These tools help owners better define requirements for the project, inform interested designers or contractors of warranties designed to encourage a quality-focused approach, allow contractors to suggest changes to project documents or concepts before having to bid on them, and other quality management opportunities. Pre-award quality management tools set the tone and expectations early on for the project in terms of quality. The pre-award tools contain two subcategories of tools, owner-led and contractor-led. Owner-led tools are those that a STA initiates. These types of tools may include specific procedures to select designers and contractors based on the quality of their work, project warranties to ensure project members guarantee a quality product, or broad project goals regarding quality and its implementation. Then, it may seem counterintuitive to have contractor-led quality management tools available for use before a awarding the construction contract. However, several of the tools focus primarily on receiving contractor feedback regarding RFP details, project specifications and/or project designs during the procurement phase. Through experience, state transportation agencies found that contractors who compete to build their projects have valuable insights to share with project planners that can lead to a higher quality product. Also, contractors, in the right settings, are willing to share insights before an agency awards the contract. 5.2.2 Post-Award tools Post-award tools are those procedures and tools implemented after the completion of procurement and through the end of the project. These include quality management tools for both the design and construction processes. The basis for the subcategories of the post-award tools is the specific project phase for which they apply – design or construction – as well as the nature of the tools. The subcategories are design review tools, teaming tools, process control tools, and training tools. Design review tools provide assistance to ensure the production of quality designs. Teaming tools focus on increasing the levels of communication and cooperation on a project 132

during the construction phase. Process control tools can increase quality or the efficiency of the quality management process as work progresses. These tools assist project managers by providing streamlined access to quality management reports and information, incentivizing or de-incentivizing contractors specifically for quality, and by offering innovations to some well- used existing processes. Finally, training tools assist the project-team to focus on quality issues specific to the project and to extend a broad-level quality focus from the upper management down to individual subcontractors. 5.3 Selection of Tools The QMS tools are not necessarily compatible with every project delivery method or QAO. When project managers develop a QMS, they must consider the goals of the project and the justification for selecting to use a particular set of tools and procedures. For example, if a project has no administrative or legal authority to alter its quality standards or overall design, then inviting contractor input into the design or quality procedures would not be worthwhile and could in fact be counter-productive. Table 5-1 presents a matrix that agencies can use to identify a set of tools compatible with both the project QAO and the specific project requirements. Using their knowledge of the project, project managers should approach the matrix by first identifying categories of tools they are interested in adding to their QMS. Then project managers will need to determine which tools are compatible with the selected QAO. The project manager then adds the selected tools to the set of procedures compatible with the agency building the project. From this final set, project managers can select a combination of quality management tools that best meet the needs and goals of their project. Not every tool is useful for every project and agencies should not expect to incorporate all or even most of the tools listed here on a particular project. 133

Table 5-2 – List of Alternative Pre-Award QMS Tools Tools QAOs Applicable With D A V O S Pre-award Tools O w ne r L ed B.1. Pre-bid meeting with specific focus on quality + + + B.2. Industry review of draft RFP w/focus on quality + + + B.3. Alternative quality management approaches in procurement + + + + + B.4. Quality based selection of contractors/ subcontractors (project specific prequalification) + + + + + B.5 Use of warranties (performance or materials) + + + + + B.6 Requirements Management - Verification + + + + + C on tr ac to r L ed B.7. One-on-one procurement meetings with a focus on quality + + + B.8 Contractor involvement in establishing and streamlining quality control standards + + B.9. Alternative Technical Concepts (ATCs) + + + B.10. External contractor panel input + + + 134

Table 5-3 – List of Alternative Post-Award QMS Tools Tools QAOs Applicable With D A V O S Post-award Tools D es ig n R ev ie w B.11. Independent party design review + + + + B.12. Over-the-shoulder agency reviews + + + B.13. In-progress design workshops + + B.14. Discipline task force (parallel entire project) + + + + + C on st ru ct io n – T ea m in g B.15. Formal partnering with regulatory agencies + + + + B.16. Formal team partnering/goal-setting process + + + + + B.17. Co-location of quality management personnel + + + + + B.18. No low bid requirement for subcontractors + + + + + B.19. Use of dual CEI/OCEI roles + C on st ru ct io n – Pr oc es s C on tr ol B.20. Innovation in witness and hold points + + + B.21. Continuous internal process audit + B.22. Real-time electronic quality management information + + + + + B.23. Financial incentive/disincentives for quality + + + + B.24. Contractor “controlled” QC testing + + + + + C on st ru ct io n – T ra in in g B.25. ISO 9000 training + B.26. Project-specific quality management team training + + + + + Key: D = Deterministic QAO A = Assurance QAO V = Variable QAO O = Oversight QAO S = Acceptance QAO 135

5.4 Summary Guidance for Assembling a QMS The guidebook accompanying this research report provides a process for designing a QMS that meets the agency and project needs. The guidebook provides suggestions for designing a QMS that meets the user’s needs. The framework of a full QMS is introduced in Chapter 1. QAOs and their selection are discussed in Chapters 3 and 4. Alternative tools to use and their selection are introduced in Chapter 5 of the Guidebook. The guidebook suggests a process for assembling a QMS – to combine the QAO/tool combinations selected into a broader QMS applicable to every aspect of their project. Development of a QMS begins with an understanding of the project conditions, especially considering the project delivery system, and whether the design and construction organizations vary from the “baseline” systems noted in this guidebook. The next step is to determine potential QAOs. Once the options are clear for QAOs, then it is appropriate to establish the “business case” for each QAO, carefully considering the pros and cons for each QAO, especially in relation to administrative and regulatory requirements and industry custom. Once the business cases are established, any QAO that varies from the baseline QAO for the organization should be presented to appropriate decision-makers in the agency. Selection of the QAO for the project should be made as a collaborative effort between project personnel and administrative decision- makers. The chosen QAO can then be matched with tools described in Chapter 5 to assemble a complete QMS that will become the guiding quality principle for the project. Clear documentation of the holistic QMS will provide the basis for contractor and consultant proposal and contract documents, aligning team efforts toward the same quality goal and establishing each party’s role in the process. As the project participants are brought on board, it is important to disseminate the principles and procedures outlined in the project QMS. It is also important to recognize that some QMSs differ from the traditional QMS, and without specific communication, any new party to the project could just “default” to the traditional system, causing misunderstandings and conflicts. 136