Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
47 Evaluation criteria change with project goals and con- straints. The following characteristics were found to promote transparency in evaluation criteria: ⢠Evaluation criteria that are clear, easy to understand, and project-specific; ⢠Evaluation criteria that define how the agency will score them; and ⢠Approaches that contain the minimal number of evalu- ation criteria to succinctly align the procurement with stated project goals. Clear communications were found to promote transpar- ency. Agencies hold pre-proposal meetings to clarify both the project goals and the best value selection process. They also define a single point of contact for the process to promote transparency and consistency. Agencies can achieve transparency with evaluation com- mittees that contain a balance of technical members with no personal interest, either actual or perceived, in the outcome of the evaluation process. The use of non-agency personnel as participating members on the committee was found in prac- tice; however, this approach varied on a state-by-state and project-by-project basis. Some states sequester their selection committees during the evaluation. Evaluator agreements of confidentiality during and after the selection process were found in all cases. All discussions and comments can be treated as public meetings. This practice promotes transparency and prepares for any potential protest. Writing detailed evaluation comments to substantiate ratings during the process and in debriefings was found to be an effective practice. These comments are specific, concise, and tied to scoring. Timely and detailed debriefings help to clarify the basis for an award, the selection process, strengths and weaknesses of proposals, and rationale behind the deci- sion. These meetings help proposers improve future offers. The practice of sharing competitorâs evaluations varied from state to state and was often tied to open records acts. The literature review, industry survey, and case examples demonstrated that industry participation in program develop- ment and performance management increases transparency. Two-thirds of the agencies using best value procurement work INTRODUCTION The goal of this synthesis is to document the state of practice in best value selection methodologies that support transpar- ency. The literature review, national survey, content analysis, and case studies provide conclusions and gaps in knowledge for best value procurement. The conclusions were validated through two or more methods (e.g., the literature review and survey). Findings that were discovered by one method but not validated by a second method were reported in the synthesis, but are not included as conclusions. Although many of the fol- lowing conclusions can be considered effective practices, they are not considered to be best practices or recommendations. The gaps in knowledge found in this study provide ideas for future research. CONCLUSIONS As discussed in chapter two and detailed throughout the synthesis, transparency issues primarily arise in best value procurement as a result of the evaluation of non-price factors; the evaluation of price is transparent. Price proposals are sealed prior to receipt by the agency and opened in a public forum; there is no subjectivity in their evaluation. However, individuals or committees evaluate non-price factors in best value selections. In some cases, these evaluators are required to exercise engineering judgment to arrive at an evaluation rating. Agencies combine these non-price ratings with prices to arrive at a best value award recommendation. It is the eval- uation of non-price factors and the process of trading them off with price that creates issues with transparency. States most frequently using only a few of the available award algorithms and rating methods promote transparency in the best value process. These procedures have the following characteristics: ⢠Most commonly adjusted bid, adjusted score, and weighted criteria award algorithms; ⢠Applying direct point scoring methods; ⢠Clearly defining the weights, or relative weights, of evaluation criteria; ⢠Sealing price proposals until technical scoring is com- plete; and ⢠Opening price proposals and technical scores in a public forum. chapter five CONCLUSIONS AND FUTURE RESEARCH
48 with industry to develop their programs and one-half meet regularly to evaluate their programs. Best value legislation can dictate much of the process and its transparency. Some best value legislation is specific to the highway sector, while other legislation is more general for all state construction projects. Much of the legislation reviewed in this study was tied to design-build (D-B) project delivery. FUTURE RESEARCH This report found several gaps in knowledge surrounding the use of best value procurement. To promote effective use, this synthesis offers the following topics for future research. There has been limited amount of investigation on the use of best value procurement for traditional design-bid-build (D-B-B) delivery. Several departments of transportation indi- cated that the use of best value for D-B-B projects brings significant benefits to their agencies. Non-complex projects, in particular, have the potential for using streamlined best value processes. The evaluation criteria and award algorithms need not be as complex as those found on large D-B projects. To encourage the use of effective best value procurement on D-B-B projects, research could determine how to stream- line best value procurement, allocate the risks equitably for the agency and contractors, and quantify the project perfor- mance between D-B-B low-bid and best value projects. The results could provide guidance, policies, and perhaps model legislation for streamlined D-B-B best value processes. Industry incurs cost when preparing best value proposals. On non-complex projects with few evaluation factors, these costs can be marginally more than the cost of preparing a D-B-B bid. On complex D-B projects, however, the costs of developing designs and proposals can be significant. This study found literature relating to stipends that offset best value preparation costs, but it did not find definitive research or direction on how agencies can minimize the cost to industry while still achieving competition and innovation on best value proposals. States took different approaches to sharing best value ratings from competing teams during debriefings. Some states will not share the evaluations of other teamsâ proposals. Some states make the entire proposal and/or complete evaluation available to all proposers. It is frequently the freedom of information acts and similar legislation that dictate this choice to share this information. The impacts of these processes on transparency and intellectual property are unknown. Study of this topic could help to promote a more competitive procure- ment process. The literature review, review of legislation, and review of best value protests provides strong evidence that best value evaluation plans are a key to successful best value procure- ments. However, research has not been conducted to test the effectiveness of different approaches to writing evaluation plans. Research into evaluation planning could help to improve the process. The study scope did not allow for an exploration of best value on public-private partnership (P3) projects. P3 projects have a unique set of goals and constraints. Proposing entities are often involved at the earliest stages of project development. A synthesis of the state of practices could help to develop optimal best value methods for P3 projects. The results could provide guidance, policies, and perhaps model legislation. At a minimum, this research could help to provide more national consistency in P3 procurements.
