Practices for Developing Transparent Best Value Selection Procedures (2015)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

7 proposals, and may potentially become an element in the contract (CFR 2011). Request for qualifications (RFQ): The document issued by the owner in the first phase of a two-phase selection process (CFR 2011). Short listing: The narrowing of the field of offerors through the selection of the most qualified offerors who have responded to an RFQ (CFR 2011). Solicitation: A public notification of an owner’s need for information qualifications or proposals related to iden- tified services (CFR 2011). OVERVIEW OF BEST VALUE PROCUREMENT Public clients use best value procurement when they aim to achieve the maximum outcome for their projects as opposed to the lowest price (Zhang 2006). Best value procurements can include price, schedule, technical and managerial merit, financial health, and past performance (Scott et al. 2006). Because the system provides a balance between price and qualitative considerations, it can optimize the benefits of fixed- price sealed bidding and sole source selection. The inclusion of key factors in evaluation criteria that match the specific needs of a particular project can raise the likelihood of meet- ing project performance goals (Abdelrahman et al. 2008). Best value procurement is one of many procurement options. It is not ideal for every project, but it can provide benefits on appropriate projects. Project goals and project characteristics can determine if the use of best value will be advantageous. Goals that align well with best value procurement include the shortening of project duration, creating opportunities for innovation, and selecting the most qualified team. Appropriate project characteristics include opportunities for innovation, the amount of design required to develop a competitive industry proposal, agency experience with the process, and market capa- bility. The 1996 version of the Federal Acquisitions Regulations (FAR) stated that best value procurement should be selected when there is a need for innovation and new technology or when a specific type of experience is required to obtain the desired outcome (FAR 1996). In addition, this approach can support the selection of a contractor when external factors such as traffic disruption or innovative environmental protection are at issue for project success. The best value procurement evaluation plan is a key ele- ment in the process. It describes the specific project needs INTRODUCTION This chapter documents the important findings from the liter- ature review on best value procurement methods. Its purpose is to establish background and context for the findings in this synthesis. Key definitions set a baseline vocabulary for the report. A discussion of process transparency provides con- text for issues and concerns that have been cited by previous researchers and the industry at large. The chapter concludes with a detailed discussion of best value concepts including best value parameters, evaluation criteria, rating systems, and award algorithms. KEY DEFINITIONS State DOTs have used low-bid procurement for well over 100 years, and a low-bid vocabulary is understandably engrained in their procurement processes. The use of best value procurement requires additional procurement terms to support the process. This synthesis offers the following key definitions to support the best value process; note, however, that individual state definitions may vary (even in the context of the case examples presented in this synthesis). Best value procurement: A procurement process where price and other key factors are considered in the evalua- tion and selection process to enhance long-term perfor- mance and value of construction (Scott et al. 2006). Construction manager/general contractor (CM/GC): A project delivery method in which a construction manager is selected to provide input during project design and then becomes at risk for the final cost and time of construction (Gransberg and Shane 2010). Design-bid-build (D-B-B): The traditional project delivery method in which the agency designs the project, solicits bids, and awards the construction contract to the lowest responsive bidder to build the project (AASHTO 2008). Design-build (D-B): A project delivery method that com- bines the design and construction phases of project delivery under a single contract (AASHTO 2008). Prequalification process: The agency’s process for deter- mining whether a firm is fundamentally qualified to compete for a project. This should not be confused with short listing or best value procurement (CFR 2011). Request for proposals (RFP): The document that describes the procurement process, forms the basis for the final chapter two OVERVIEW OF BEST VALUE PROCUREMENT

8 and the method for rating proposals. It also describes the method for including price as one of several evaluation fac- tors. It provides a project-specific approach to describing the best value evaluation criteria, weighting systems, and award algorithms. The evaluation criteria require measurable added value to the project (Molenaar and Johnson 2003). The weighting systems and award algorithms can become complex and the complexity of the evaluation process can become a challenge in best value selection (Gransberg and Senadheera 1999). TRANSPARENCY IN BEST VALUE PROCUREMENT Although best value procurement is widely used by pub- lic clients across multiple sectors, there are concerns with transparency in the system. These concerns are amplified in the transportation sector because, for most of its history, the U.S. highway system was constructed by the lowest bidder. Low-bid procurement is inherently transparent, requiring only the evaluation of price. Bids are sealed and then opened in a public forum. Best value procurement requires the evalua- tion of both price and non-price factors. Price factors are still sealed and often opened in a public forum. Non-price fac- tors are scored by technical evaluators. In some cases, these evaluators must exercise engineering judgment to arrive at an evaluation rating, which can create issues with transparency. These ratings are then combined with prices to arrive at a best value award recommendation. It is the scoring of non- price factors and the process of trading them off with price that creates issues with transparency. Later in this report there is discussion on how agencies promote transparency in best value procurements. Transpar- ency in the overall selection process, evaluation of non-price factors, scoring of proposals, conducting cost-technical trade offs, composition of the technical committee, debriefing unsuccessful proposers, industry outreach, and other items are reviewed. BEST VALUE CONCEPTS Figure 1 presents best value procurement at an operational level. The overall process includes four distinct concepts: (1) best value parameters, (2) evaluation criteria, (3) evalua- tion rating systems, and (4) award algorithms. Transparency in best value selection can be achieved by rigorously analyzing these four concepts. The project goals drive the selection of best value parameters that form an eval- uation plan. The best value parameters provide for measur able evaluation criteria. These evaluation criteria, at some level, provide proposers with details on the agency goals. Best value award algorithms describe the process for select- ing the best proposal. Transparency can be lost or gained at any one of these four points in the process. Best Value Parameters Defining key parameters is one of the steps necessary for obtaining transparency in the best value selection process. NCHRP Report 651 indicates that best value parameters must be well-defined and defensible to the public and the industry. These parameters determine evaluation criteria that add value to a project and support transparent and fair procurement systems (Scott et al. 2006). Some projects require only a few parameters (e.g., cost and time), whereas others require more parameters to ensure the optimal selection. NCHRP Report 651 concludes that “the owner should customize the parameters for the needs of the given project rather than strive to find a one-size-fits-all standard system” (Scott et al. 2006). The fol- lowing paragraphs discuss three main best value parameters; cost, time, and qualifications and performance. Cost By definition, cost (i.e., price) is always a parameter in the best value award decision. Price is traded off with non-price parameters to determine the best value. Transparency in the cost parameter is necessary for transparency and objectivity in the overall best value selection. Time Best value time parameters encourage the proposing of a schedule that meets the project goals. When time-related proj- ect goals and best value parameters are transparent, proposers will tend to make choices in cost-time tradeoffs that optimize value to the owner. Similar to cost parameters, time parameters can enhance transparency and objectivity because they are a quantifiable measure (e.g., overall project schedule, time of a closure, and meeting a major milestone). Qualifications and Performance The use of best value qualification parameters assist in selecting the most appropriate team. State agencies often use experience and past performance criteria to determine which FIGURE 1 Best value concepts (Source: Scott et al. 2006).

9 team has the greatest likelihood of success. However, it is important that this process not be confused with basic project prequalification. The main advantage of qualifications param- eters is the ability to invite proposers who have a proven track record of successfully completing a specific type of work. One of the primary concerns with qualifications parameters is the possibility of creating favoritism (Scott et al. 2006). To minimize these concerns and ensure the best value selection is transparent to industry, Parvin (2000) states that the owner should carefully develop qualifications parameters based on the project’s specific requirements. Evaluation Criteria Establishing evaluation criteria is an important step in best value procurement. Evaluation criteria vary on a project-by- project basis. Depending on specific project goals, objectives, and characteristics, the owner will develop a clear evaluation plan to ensure the selection process is transparent and fair. Cost is typically the most important criteria used in all types of best value evaluation plans. Projects with high road user costs typically include schedule criteria. For qualifications param- eters, best value evaluation criteria often include past perfor- mance, qualifications of key personnel, and sub-contracting plans. Montana DOT points out that “it is highly important that selection criteria be well defined in order that the most objec- tive, fair, equitable, and competitive determination is made for award” (MDT 2005). Georgia DOT (GDOT) highlights the importance of evaluation criteria to obtain a transparent best value selection in stating that evaluation criteria should: • Be clear, defendable, and easy for the proposers and public to understand; • Not overlap scoring criteria in the statements of qualifi- cation (SOQ), especially with respect to key personnel that have already been evaluated in the SOQ; • Focus on items that bring measurable value to the project; • Be tailored to the individual project; and • Avoid or minimize recycling criteria from project to project (GDOT 2014). NCHRP Report 451 provides a list of examples of evalu- ation criteria categorized into four main areas: (1) technical solutions, (2) management solutions, (3) past performance, and (4) price (Anderson and Russell 2001). Table 1 summarizes the sub-criteria and content from these four main areas. Similarly, Table 2 provides a list of best value evaluation criteria from NCHRP Report 651 (Scott et al. 2006). It should be noted that these evaluation criteria were based on the state of practice in the early 2000s. This synthesis will update current practices regarding evaluation criteria. Best Value Evaluation Rating Systems Evaluation rating systems are a tool to support the transpar- ency and fairness of the best value selection process. Parvin (2000) indicates that the owner should clearly state the weight assigned to each item in the evaluation criteria and ensure that they are used by the evaluation team. There are a wide variety of best value evaluation rating sys- tems from which agencies can choose. Some systems are simple and easy to use, making them reasonably transparent. Others are more complex and subjective, making them less transparent, but more appropriate for qualitative criteria. NCHRP Report 561 defines four categories of evaluation rating systems, including airetirc-buS/stnetnoC airetirC noitaulavE Technical Solutions • Technical expertise - Construction methods related to the project - Expertise with using relevant materials - Past project examples • Personnel qualifications and availability • Compatibility of the construction contractor with the agency Management Solutions • Organization/management plan - Proposed project organization - Responsibilities and expertise of key personnel • Relevant experience/past performance - Example of past projects with similar size, type, scope, and complexity • Proposed schedule/work plan - Proposed plan for project completion - Proposed schedule for project completion Past Performance • History of customer satisfaction • History of timely delivery • History of conforming to specifications • Standards of workmanship • History of adherence to schedule • Safety record • Past management of subcontractors Source: Anderson and Russell (2001). TABLE 1 EXAMPLE BEST VALUE EVALUATION CRITERIA

10 (1) satisficing, (2) modified satisficing, (3) adjectival rating, and (4) direct point scoring (Scott et al. 2006). The following discus- sion is also based on NCHRP Report 561. Satisficing Evaluation Rating System The satisficing evaluation rating system is the simplest form of the four methods. Minimum standards are set in the evaluation plan for every evaluation criterion, and each proposal is rated against these minimum standards. Agencies must include defi- nitions for fatal deficiencies and their consequences to support transparency. Industry often refers to satisficing as “go/no-go.” Modified Satisficing Modified satisficing can represent the degree of responsiveness in a proposal. Modified satisficing includes criteria ratings to allow the evaluator an opportunity across a variety of degrees of assessments. This method of rating system allows for those proposals that are nearly responsive but with minor deficien- cies to stay in competition and also rewards the proposals that exceed the published criteria. Red-Amber-Green systems are the simplest forms of modified satisficing. Green can denote that the proposal is fully responsive to criteria, amber that the proposal has minor deficiencies, and red that the proposal is non-responsive because of the fatal deficiency. Adjectival Rating This rating system is an extension of the modified satisficing. Adjectival rating systems use a precisely defined set of terms to evaluate criteria in each proposal. The three elements of adjectival rating systems are (1) definitions, (2) performance indicators, and (3) differentiators. Clear definitions enhance transparency in the selection process. Performance indicators Evaluation Criteria skrameR sedulcnI Initial Capital Cost Construction and procurement costs (also includes design costs in a D-B project) Sometimes called the “Bid” price Schedule Time to build project (also includes design time in a D-B project) Sets contract performance period Prequalification Financial and corporate information as well as bonding requirements Typically a routine government form used for all contracting opportunities. Past Project Performance Project experience on past projects that are similar to the project at hand. Also might include past history of claims and litigation. Preference is given to offerors with the most relevant experience. Key Personnel Experience and Qualifications Qualifications of key personnel Licenses, registrations, and past project experience of individuals Subcontractor Information Subcontracting plan including small business utilization Often requires that goals for participation by certain types of firms be met. Project Management Plans Plans for logistics, material management, equipment, traffic control, etc. Often related to schedule constraints Safety Record and/or Plan Corporate safety record and plans for specific safety hazards Often uses the Workmen’s Compensation Insurance Modifier as a metric to measure safety record. Quality Management Plans Typical QA/QC program submitted prior to award May include design QC if bid alternates or D-B is used. Proposed Design Alternate Owner allows contractor to propose an alternate material or technology for a given feature of work. Bid is submitted with and without alternates. Owner makes decision of which alternates will be accepted prior to award. Technical Proposal Responsiveness Proposals are considered responsive if they receive a minimum technical score. Requires that a measurable standard be developed for each evaluation criteria. Environmental Considerations Plans to prevent and/or mitigate pollution during construction Many are required by law and/or regulation. Source: Scott et al. (2006). QA/QC = quality assurance/quality control. TABLE 2 EXAMPLE BEST VALUE EVALUATION CRITERIA

11 help evaluators determine the appropriate rating (e.g., high, moderate, or low). Differentiators further distinguish between the proposal ratings. Direct Point Scoring Direct point scoring systems can be the most complex system. Points are assigned to the evaluation criteria in the proposals. The direct point system is used by many DOTs. The flexibil- ity of the scale on which the proposal is scored or rated is the greatest advantage of this method. The weaknesses of this method are the “false sense of accuracy” that can be provided by numerical ratings of qualitative criteria and the effects of a lack of consistency when assigning scores across best value criteria categories. Best Value Award Algorithms Best value algorithms are the steps taken by the owners to combine best value parameters, evaluation rating systems, and evaluation criteria to make a final award recommendation. NCHRP Report 561 distills seven best value award algorithms from the research analysis (Scott et al. 2006). Table 3 provides a summary of these methods. Meets Technical Criteria—Low Bid Price is the most important criteria in this award algorithm. The technical proposals are evaluated before receiving any cost proposals. The technical proposals that met the minimum proposed requirements are then taken forward and the price proposals for those are opened. The contract is then awarded to the proposer with the lowest price. Adjusted Bid Numerical scoring of the proposals is required in this type of algorithm. Technical proposals are scored first and the price proposals are opened. The project price is adjusted by the technical score. This adjusted bid is used only for awarding the project. The project is awarded to the proposer with the lowest adjusted bid. The contract price is based on the amount listed in the price proposal. Adjusted Score The adjusted score algorithm is very similar to the adjusted bid algorithm. The price proposals are opened after the tech- nical proposals are scored. The technical score of the proposal and the estimated total project price were multiplied and the result divided by the price proposal submitted by the proposer to calculate the adjusted score of the proposal. The project is awarded to the proposer with the highest adjusted score. Weighted Criteria The technical proposal and the price proposal are evaluated indi- vidually in the weighted criteria award algorithm. The technical evaluation factors and the price are assigned weights depending on their importance. The total score of the proposal is calculated by the sum of these evaluations and the project is awarded to the proposal with the highest score. TABLE 3 BEST VALUE AWARD ALGORITHMS selbairaV mhtiroglA mhtiroglA drawA eulaV tseB Meets Technical Criteria— Low Bid If T > Tmin, Award to Pmin If T < Tmin, Non-Responsive T = Technical Score P = Project Price Adjusted Bid AB = P/T Award ABmin AB = Adjusted Bid Adjusted Score AS = (T x EE)/P Award AS max AS = Adjusted Score EE = Engineer’s Estimate Weighted Criteria TS = W1S1 + W2S2 + … + WiSi + W(i+1)PS Award TS max TS = Total Score Wi = Weight of Factor 1 Si = Score of Factor 1 PS = Price Score Quantitative Cost—Technical Tradeoff TIncrement = [(Tj/Tj) – 1] x 100% PIncrement = [(Pj/Pi) – 1] x 100% If TIncrement > PIncrement, Award Proposali If TIncrement < PIncrement, Do Not Award Proposalj, Repeat with Proposalj+1 Repeat Process until TIncrement > PIncrement T = Technical Score P = Project Price Qualitative Cost—Technical Tradeoff Similar to above, only no quantitative analysis of difference. Award to proposal that has best value in proposed scope. Evaluation panel reaches consensus as to which proposal is the best. Fixed Price—Best Proposal Award Tmax, Fixed P T = Technical Score P = Project Price Source: Scott et al. (2006).

12 Quantitative Cost—Technical Tradeoff The technical and price proposals are scored in increments and then the difference between these incremental advan- tages is observed. The score (technical or price) increment is calculated by dividing the highest score by the next high- est score minus one and then multiplied by 100. The project is awarded to the proposer with the lowest price, unless the higher priced proposal can properly be justified to add more technical value to the project. Qualitative Cost—Technical Tradeoff This algorithm relies on the judgment of the selection com- mittee to determine the relative advantages of the proposals after a review of evaluation ratings and prices. The selection committee must document its rational decision based on ana- lyzing the differences. To support transparency the decision must (1) be consistent with the solicitation evaluation factors and sub-factors, (2) be based on a comparative analysis of the proposals, and (3) represent the selection committee’s rational and independent judgment. Fixed Price—Best Proposal This award algorithm relies on a maximum price or a fixed price of the project that is defined by the owner. Each proposer must submit a technical proposal with an agreement to perform the work within the price constraints. The project is awarded to the best technical proposal. SUMMARY The literature review results presented in this chapter docu- ment industry practices and provides key definitions for use throughout the synthesis. The best value parameters, evalu- ation criteria, rating systems, and award algorithms describe the published work at the time of this synthesis. The concepts in this chapter set the basis for the survey and case example protocols applied in this report.