Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
PRACTICES FOR DEVELOPING TRANSPARENT BEST VALUE SELECTION PROCEDURES State departments of transportation (DOTs) have historically used a low-bid approach to procure construction services. Under the low-bid approach, price is the sole competitive factor. Non-price factors such as qualifications, experience, technical approaches, and innovative solutions are not considered. Although the low-bid approach is inherently transparent, it does not always offer the best performance during and after construction. To improve project quality and performance, a number of DOTs are increasingly using best value procurement to deliver their transportation projects. In essence, best value procurement is a process to select the most advantageous offer by evaluating factors in addition to price. Price is always a consideration in best value procurement and it is usually the most important factor (i.e., has the greatest weight). The best value method allows for the consideration of additional factors such as schedule, technical merit, management solutions, and/or past performance. The selection factors can vary by project in order to optimize the probability of achieving unique project goals. Although the inclusion of non-price selection factors can increase the probability of achiev- ing project goals, these factors can raise concerns of transparency. Low-bid procurement is inherently transparent; it requires only the evaluation of price. In best value procurement, evaluators rate non-price factors. In some cases, the process requires the evaluators to exercise engineering judgment in an evaluation rating. It is the rating of non-price factors and the process of evaluating them with price that creates issues with transparency. The goal of this synthesis is to document the state of practice for developing transparent best value selection procedures. The report identifies methods that enhance the objectivity, fairness, and transparency of the evaluation process. The study methodology consisted of four main steps: (1) literature review; (2) survey of DOTs; (3) content analyses of requests for qualifications (RFQs) and requests for proposals (RFPs), state guidelines, and manuals; and (4) case examples. Following a rigorous literature review, a web-based survey was developed and distributed to the members of the AASHTO Subcommittee on Construction and the AASHTO Subcommittee on Design in all 50 states, the District of Columbia, and Puerto Rico. Forty-six DOTs responded to the survey (an 88% response rate). Thirty-five of the 46 respon- dents reported that they are currently implementing or considering best value procurement. The content analysis included an analysis of RFQs and RFPs, and guidelines and manuals from the aforementioned 35 DOTs to find relevant information relating to fairness and transparency in best value selection. Finally, structured interviews with seven best value selection-experienced DOTs provide case examples of best value implementation and lessons learned. The four-step methodology provides for a comprehensive state-of-practice summary and a robust set of defen- sible conclusions. Synthesis results indicate that transparent best value selection procedures require clear, comprehensive, and well-documented practices in both pre- and post-proposal submission activities. Specifically, this synthesis documents the following. 1. Best value methods that promote transparency: Best value procurement procedures consist of (1) evaluation criteria, (2) evaluation rating systems, and (3) award algorithms. SUMMARY
2 A wide variety of evaluation criteria were found in the literature and in practice. Using the minimum number of criteria to achieve project goals promotes transparency. The literature also describes multiple categories of rating systems and award algorithms. The study found that the adjusted bid, adjusted score, and weighted criteria award algorithms used in combination with direct point evaluation rating methods provide for the most transparency. The transparency stems from the concept that these algorithms most closely resemble low-bid procurement. 2. Evaluation criteria clarity and transparency: Clear, easy to understand, and project- specific evaluation criteria yield the most transparency. Evaluation criteria change with project goals and constraints. Technical solutions, management solutions, past performance, and price provide categories for a variety of evaluation criteria. Evaluation criteria that are quantitative provide the most transparency. Although some agencies employ adjectival ratings (i.e., the use of words or colors as opposed to numbers on the ratings), these methods were primarily found on only complex design-build (D-B) projects where direct point scoring was not feasible. Conveying the weights of evaluation criteria directly in the RFP was found to increase transparency. Indeed, 83% of states responding to the survey questionnaire convey an evaluation point range or weighting in their RFPs. 3. Evaluation committee composition and conduct: To maintain transparency, eval- uation committees contain a balance of technical members with no personal interest, either actual or perceived, in the outcome of the evaluation process. Agencies can establish best value evaluation committees on a project-by-project basis, depending on project goals and evaluation requirements. Forty percent of states include non-agency members to support transparency of the technical evaluation process. The outside agency members can include contractor and/or engineering association representa- tives, private sector subject matter experts, and/or public sector partners. Some proj- ects require selection committee sequestration during the evaluation. All states require that committee members sign agreements of confidentiality. While the evaluations are not a public meeting, the practice of treating evaluation discussions and written comments as public meetings was found to promote transparency. This practice also prepares the committee for any potential protests or requests through freedom of information acts. 4. Completeness of evaluation comments and debriefings: Thorough documentation of evaluation comments provides for a fair and transparent best value selection process. These comments are specific, concise, and tied to scoring. Detailed evaluation comments substantiate ratings and assist in debriefings. Timely and detailed debriefings help to clarify the basis for award, the selection process, strengths and weaknesses of proposals, and the rationale behind the decision. Ninety-eight percent of the agencies studied in this synthesis conduct debriefings orally and/or in writing. Almost half (46%) allow unsuccessful proposers to view the proposal of the wining proposer. Such practices help proposers understand the best value process and provide the learning for continuous improvement and competitiveness on future offers. 5. Communications to promote transparency: Clear communications were found to support transparency. Pre-proposal conferences and debriefings align industry goals with agency goals and demonstrate transparency in the process. The pre-proposal conference helps to clarify any potential ambiguities in the solicitation documents including the technical aspects from an RFP. Appointment of a single point of agency contact for the evaluation process was found to increase transparency and clarity. 6. Collaboration with industry in best value programs: Industry participation in program development and performance management increases transparency. Two-thirds of the agencies using best value procurement work with industry to develop their programs and one-half meet regularly to evaluate their programs. Although this synthesis focuses on transparency in best value procurement, the synthesis statement and the subsequent fact finding provide more generally for common practices
3 and lessons learned that are included throughout the report. The most significant are sum- marized here. 1. Best value procurement and project delivery: This study found that best value pro- curement is used with a variety of project delivery methods. Of the agencies respond- ing to the synthesis questionnaire, 100% use or can use best value for D-B contractor selection. Several DOTs are also employing best value procurement concepts in tra- ditional design-bid-build (D-B-B) delivery, construction manager/general contractor delivery, and indefinite delivery/indefinite quantity contracts. Various forms of best value procurement are also in use on public-private-partnership (P3) projects; however, these projects were outside the scope of this synthesis. 2. Importance of best value evaluations plans: Clear and comprehensive evaluation plans are a key to best value procurement success. Fair and objective processes are defined prior to procurement and adhered to during selection. As previously stated, clear and comprehensive best value evaluation plans are required by the courts to withstand any potential protests. Evaluation plans with clear processes and a definable set of standards help enhance fairness and transparency. 3. Training in best value procurement processes: Training promotes transparency, consistency, and fairness in the best value process. Training is often required because technical experts may not be versed in best value procurement processes. When evalu- ators are subject to transparency questions, both internally and externally, it is best to understand their roles and responsibilities in the entire best value evaluation process. 4. Lessons learned from best value protests: Of the 35 agencies responding to the question on best value protests, 24 had not experienced a protest. Nine of the 11 agencies with best value protests provided brief descriptions. Most protests related to a perception of improper proposal evaluation (i.e., a dispute with the technical scoring of the proposal). In all cases where the agency followed its predetermined procurement procedures the outcome was favorable to the agency. This report also provides detailed descriptions of four protests in Appendix D. Again, the most consistent theme in all of the protests is that DOTs are required to clearly plan their evaluations and follow their plans during the process to avoid and/or withstand protests. The synthesis discovered a number of gaps in the research that provide opportunities for future study. The intent of the following questions is to draw attention to these gaps and pro- mote discussion on potential future research. The Conclusions (chapter five) provide more detail on these opportunities. 1. Are streamlined best value methods for D-B-B projects feasible and how might they impact overall project performance? 2. How can agencies minimize industry best value proposal costs and still meet their objectives for finding competitive solutions? 3. How does the sharing of competitorâs proposals, during debriefing or through open records requests, impact industry competitiveness and intellectual property rights? 4. What are the most effective practices for writing best value evaluation plans? 5. What is the current state of practice for best value on P3 projects and what are the characteristics of an optimal method?
