Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
40 Conclusions The primary outcome of this research effort is a practical set of recommended guidelines for transportation professionals to use for assessing benefits of a CSS project. To achieve this goal, a set of principles was identified with associated benefits to which metrics can be applied for measuring the magnitude of the benefits. The guidelines developed provide a methodology for com- pleting a systematic quantification of benefits of using the CSS approach for project development. The guidelines present a comprehensive approach for implementing assessment and provide a set of instructions on how to conduct a benefit quan- tification effort. It is apparent that a systematic approach needs to be undertaken where data will be collected periodically in order to provide the basis for evaluating individual projects and identifying areas for agency CSS improvement. The bene- fit quantification is a process that any agency can undertake in order to first determine the effectiveness of their efforts on a specific project, conduct a program evaluation, and use the les- sons learned to improve specific actions for future projects. This allows for continuous improvement effort that could be undertaken to positively impact project development and delivery operations using agency resources more effectively. The guidelines have also been designed with the realities of project scope, size, and extent. A project team has the ability to identify and customize the principle intensity applied in the project based on the specific needs of the project. This provides the ability to vary principles applications in terms of magnitude and allows for a flexible project development process that provides a broadened ability to the agency and project team to achieve desired outcomes. Moreover, the proj- ect team can also select the anticipated benefits and determine those that are to be monitored and measured. This allows for an evaluation procedure that is customizable to a process which provides flexibility to the agency and project team to achieve desired outcomes. To determine whether a benefit accrued, the project team can develop benchmarks that would be specific for the project developed and customize data col- lection to determine them. The evaluation and comparison of the collected data to these benchmarks allows for identifying successful application of principles and improvement actions for future applications of principles that were not successfully employed. These efforts could be then used by the agency to improve the development process of other projects. The principle-benefit matrix provides an agency with a link- age of direct actions to improve both project and program per- formance as well as to determine future process improvement opportunities. Once the agency targets benefits to be measured by all projects, associated metrics could be determined and the agencywide target threshold values could be established. The collected data could then be used to identify the remedial actions required to meet or exceed the thresholds set through an identification of the appropriate actions for improving each principle by examining the corresponding application criteria. For example, if an agency is experiencing an extreme lack of trust it can identify several actions, using the associations estab- lished by the matrix, that it might take to remedy that situation. The use of the matrix in this fashion will allow agencies to improve their overall performance and project development and delivery process. The action principlesâ criteria provide a level of detail that operationally defines the necessary CSS actions. This level of detail needs to be expanded and tailored to the circumstances of the agency. This represents the working level of the project development and delivery processes. Agencies wishing to pur- sue CSS must wrestle with the requirements of these criteria. This is the level of knowledge that is important to the develop- ment of project managers/leaders and to the various members of the core project team. Additional project team specialists that are brought on board will require some understanding of CSS at this level to perform their functions adequately. The matrix can also be very important in educating new transportation professionals or the project team specialist on C H A P T E R 6 Conclusions and Suggested Research
the project development and delivery process and the spe- cific actions (and expertise) needed to accomplish a success- ful project. Conducting targeted assessment of specific project devel- opment processes is difficult, but can be accomplished if con- ducted in a real-time proactive manner. Using standardized surveys to acquire expert opinions and assess stakeholder/ community satisfaction is possible and very informative. Project leaders can use this information during a project to determine effectiveness and program managers can use the information from multiple projects to make procedural adjust- ments and determine possible improvement actions that may include, for example, improving process handoffs or training. The use of the case studies identified a number of issues that would currently limit the widespread application of the proposed method for benefit quantification and these are dis- cussed here. Issues include the inefficiency, if not impossibil- ity, of a forensic approach for data collection, the need for systematic data collection, and the commitment of the agency to conduct such evaluations. The most important aspect encountered is that a systematic and well organized data collection effort should be undertaken from the outset of the project. The forensic approach imple- mented in this research showed that it is almost impossible to identify and collect data after the completion of the project. Such data is likely to be incomplete, not adequately cover the required metrics, and not have the necessary statewide com- parisons data available. The identification of the benefits to be monitored along with their metrics from the outset of the project is essential. This will allow for identifying the specific metrics to be monitored and allow for timely data collection and the building of comparison data. It is important for an agency to identify data collection needs from the outset of the project and include it in the project development process to ensure that critical windows of opportunity are not missed. Critical assessment data must be collected appropriately, maintained adequately, and be readily available. Some benefits cannot be easily quantified and not all met- rics can be converted to dollar values in order to determine the level of benefit accrual. The research concluded that most case studies benefits cannot be easily quantified, but need to be compared to the goals set forth in the purpose and need statement of the project. For example, benefit 13 âImproved mobility for all usersâ is measured by identifying the extent of new or improved facilities included in the project. For some case studies, this benefit accrued once a bike lane was added for the entire length of the project or a new bus line was incor- porated with bus stops along the route. The presence of the facility is a positive benefit of the project, since it was part of the purpose and need statement, but does not allow for con- verting this to a value to be added to time and cost savings and determine the extent of benefit accrual for the project. Another issue regarding difficulties with benefit quantifica- tion is the fact that benefits are relative to the project scope, size and goals. It is reasonable to assume that the magnitude of the benefits will be smaller for projects with limited scope or small size; this does not lessen the importance of achieving these benefits for a project. Also, such data should be collected to allow for inclusion of the project in the agency aggregation and estimation of benefits for potential agency continuous improvement efforts. The fact that goals vary by project cre- ate an obstacle for comparison of benefits across projects. The results from the study indicate that agencies do not sys- tematically collect data to evaluate project performance and develop benefit-cost information for projects. It is apparent that data is collected only when individual project teams con- sider it appropriate for documenting their actions to possibly avoid future difficulties with either stakeholders or the com- munity. This approach does not allow for the development of a systematic data collection effort across all projects for an agency nor for the establishment of a database that could be accessible to any interested party. The data requested in the case studies was not made available even when individuals noted that they had access to it. Therefore, there is a need for commitment by agencies to systematically collect such data and maintain a database to allow for benefit assessment and/or establishing a continuous quality improvement effort. Data from the case studies collected indicted that very few agencies and teams routinely document project decisions and actions and maintain a project file that includes data to be used for quantification of the potential project-related bene- fits once the project is completed. Several projects are typically completed over a long period of time and during their lifetime there may be a turnover of project personnel. Even though transportation agencies may bring new personnel up to speed internally, the need for developing project documents is essen- tial for retaining the knowledge developed and commitments made in the project. An issue that the research team faced was that team members noted the potential for benefit accrual but were not capable of presenting the appropriate documenta- tion to validate and quantify these benefits. Future Research The work completed here also identified areas where addi- tional research is needed to provide answers to the questions posed but were not addressed due to data limitations. The fol- lowing areas of future research are recommended as a result of the issues raised here: 1. A longitudinal case study could be undertaken where the implementation of the proposed approach for benefit quantification could be applied. This will require the iden- tification of projects in their initial stages to allow for the 41
42 development of the process and establishment of benefits to be monitored from the outset of the project and allow for a systematic evaluation of the proposed benefit quan- tification approach. 2. Another effort could be the identification of a select set of case studies among those used here for a detailed evaluation and documentation of potential benefits. This approach will allow site visits and data collection in order to com- plete data for all metrics. 3. The linkage between CSS and Practical Design/Solutions should be explored and determined. The new initiatives by Missouri and Kentucky in the area of Practical Design/ Solutions are considered as a new approach for project development. However, these efforts could be viewed as a subset of the CSS approach, since they still follow many of the same principles but the decisions are guided more by financial and budgetary constraints. Therefore, the effects of this effort on CSS should be investigated.
