Measuring and Improving Infrastructure Performance (1996)

If performance is to be a useful concept of infrastructure decision making, and if its assessment depends so critically on the decision making context, then the assessment process itself should encourage appropriate recognition of both the types of decisions to be made and the particular objectives, vision, and character of the areas the infrastructure serves. This chapter describes the committee's conceptual framework for that assessment process. This is a general process; detailed procedures and the individuals and entities involved will vary from one region to another.

The primary motivation for assessing infrastructure performance lies within the context of a larger system of decision making aimed at allocating resources and taking action to pursue the public purpose of infrastructure—that is, to produce desired outcomes. For example, an immediate outcome from building a wastewater treatment facility should be cleaner water within a river where the effluent is released. Over the longer term, the outcomes may extend beyond clean water itself to include improved health of the population using or living around the water system, which in turn represents enhanced quality of life for residents in the area. Outcomes may also extend to the downstream ecology, as seen in increased diversity of aquatic species of plants and animals and other wildlife up the food chain. In addition, the effects of cleaner water may flow downstream to neighboring communities that may be able to use a simpler and less

expensive process in providing clean, safe drinking water for their own residents. Seeking such favorable outcomes, avoiding unfavorable ones, and determining what outcomes have been achieved are the principal aims of the assessment process, but these aims should be explicit.

Assessment for its own sake, conducted in isolation from the people served by the systems being assessed and devoid of the values of the local community, is an empty exercise. The assessment process must begin with identification of a problem, demand, or need for assessment, that is, with the question "Why are we doing this?" There are many possible answers, ranging from "We have to" (e.g., to meet federal requirements) to "We want a clearer understanding of how to make our public assets work harder for us" to ''We have a strategic vision for our community and want to use infrastructure to help us achieve it."

Regardless of the particular motivation, the performance assessment process is a primary mechanism for the expression of community values and subsequent decision making about infrastructure development and management. It is through this process that objectives for infrastructure are defined, specific measures of performance selected, and judgments made about performance. The process must both encourage communication and facilitate resolution of the conflicts that often arise among the diverse objectives infrastructure is meant to achieve.

Figure 3-1 illustrates the generic process that assessment should follow. The process effectively begins with a question of whether infrastructure performance is adequate, which implies a problem, demand, or need for something different from the existing system. Formulating the question and beginning to search for answers involves identifying who is involved and how and what their ranges of interests may be. The process then proceeds through describing the infrastructure system and its setting in a way that enables performance to be measured and then making the measurement. A judgment is then made as to whether performance is adequate or might be improved by taking specific actions identified during assessment. The process seldom really ends but rather starts anew with different perspectives developed in the assessment. Data about the system and the values underlying a community's judgments inform all stages of the process, which are described further in the following paragraphs.

Figure 3-2 shows the context of questions that initiate the assessment and the types of answers likely to be given. The questioners may include community and other special-interest groups, elected officials, businesses, individual citizens, and others who use, own, operate, abut, or otherwise have an interest in the outcomes of infrastructure-related actions. These

people are the "stakeholders," and their interests typically extend beyond infrastructure. For example, infrastructure decision makers may consider employment opportunities, land use, community political interests, and a variety of other factors unrelated to infrastructure's performance. As the shaded area of Figure 3-2 indicates, the committee focused its attention primarily on certain elements of infrastructure and within the jurisdictional levels between state and local government, but it inevitably considered a much broader scope of interests.

The first step after the need for performance assessment is established is to dearly identify who the stakeholders are in the decision-making situation that motivates the assessment. The level at which the decision is made (e.g., local or state) and the type of derision (e.g., planning new facilities, determining how to implement a new regulation) will have much to do with who these stakeholders are. However, stakeholders may be found at many levels other than where the decisions are to be made.

In particular, an array of public and private institutions are responsible for building, operating, maintaining, and using the infrastructure of a region to provide services to that region (see Figure 3-3). These provider

FIGURE 3-3 Stakeholders in performance assessment as seen from the perspective of  infrastructure providers.

institutions (formal and informal, groups and individuals, large and small) are key stakeholders in the region's infrastructure. They prepare and execute master plans, capital budgets, architectural and engineering designs, approvals, purchases, construction, sales, and a range of other activities through which infrastructure evolves. This provider perspective may in principle be mapped out within any of the cells of the matrix shown in Figure 3-2 for a particular region and decision making situation.

Because infrastructure is essentially a local matter, performance assessment should always include the local perspective, even when the decisions of concern are essentially broader. Officials in cities the committee visited pointed out that some federal regulatory and funding programs fail to recognize this need for local perspective in achieving higher infrastructure performance and impose standards uniformly on all areas, regardless of the causes of performance deficiencies or the consequences of inappropriate standards. For example, in Minneapolis advanced treatment of municipal sewage was mandated to relieve a problem of high levels of nitrogen in a local lake that was the receiving body for treatment plant effluent. Studies undertaken by local agency officials demonstrated the source of nitrogen to be agricultural fertilizers.

The next steps in performance assessment deal with identifying the infrastructure system of interest, the boundaries and character of the system and area served, the objectives and vision the community ¹ sets for the system, and constraints (e.g., budgets, interagency relationships, jurisdictional) and regulations that may limit the feasibility of actions.

Identifying the system, boundaries, and context often involves use of maps and databases of the types typically maintained by municipal and regional planning agencies, departments of transportation, water utilities, and sewer authorities. These databases increasingly are being stored in computer-based geographical information systems (GISs). Rapidly evolving and sophisticated GIS technology is enhancing the currency of and access to these data, making these technologies valuable resources for performance assessment. This is discussed further in Chapter 4.

The vision and objectives may come gradually and from several sources. A consensus-building community-wide discussion may generate a collective vision of how infrastructure can and should serve the community. Objectives may also come from legislation or other mandates imposed by higher levels of governments. The constraints and regulations may be formally stated in laws, budgets, or official regulations. They may also be informal, as is typically the case with political constraints or the physical constraint that poor soils or steep slopes represent when new construction is being considered.²

Selecting and using specific performance measures are crucial tasks that are discussed in depth in Chapter 4. Depending on the problem or

demand motivating performance assessment, the selection may involve a substantial amount of public discussion or can be a relatively straightforward adoption of standard planning or engineering practice. Some experts in public decision making practice assert that this process of structuring objectives and measures is more an art than a science (e.g., Keeney, 1988).

While some objectives set for infrastructure may change from one situation to the next, many will remain constant for considerable periods of time. The measures selected for one decision may therefore be useful for subsequent decisions as well, but attention should always be given to confirming that the measures being used suit the situation, as defined by stakeholders, their current objectives, and decisions being made. The types of decisions and their influence on the process are the subjects of later sections of this chapter.

Local values have overarching influence on all steps of the assessment process. Because infrastructure is essentially a local concern, local values should be reflected in the assessment even when the motivation for assessment comes from other levels. Asking stakeholders why they judge particular objectives to be important helps to separate those that are a means to other ends (e.g., reduce automobile travel as a way to reduce automotive air pollution) from those that are related to basic values underlying the decision making situation (e.g., reduce air pollution because air pollution is simply undesirable).

Underlying the entire assessment process is a need for data. The committee found that in many areas the lack of data is one of the principal obstacles to implementing an effective performance measurement process. A continuing regional data collection system is needed to support performance assessment and enable longer-term performance monitoring. Many metropolitan areas have in place some basic elements—databases and modeling tools—but have not brought together the information on separate infrastructure modes into a uniform and rapidly accessible system.

As Figure 3-1 illustrates, the assessment process includes feedback to earlier steps. Sometimes work will proceed concurrently on several of the distinctly described steps. At each step, as participants increase their understanding of the problem and their options for solving the problem, reconsideration of earlier steps may be necessary. The process is in a sense progressive, helping people to explore their options, work toward resolving conflicting objectives, and seek a consensus on their preferred course of action, even if that course means no action.

The assessment process generally will yield a multidimensional set of performance measurements. The specific types of measures are discussed in detail in Chapter 4. Because some aspects of performance are difficult

or perhaps impossible to quantify, some of these measures may not be numerical. The committee recommends, however, that attempts be made to devise quantitative indicators wherever possible. Such measures help stakeholders and decision makers focus on the relative severity of problems and on the means to address these problems or to realize the vision that motivated assessment.

Arriving at decisions in the face of multiple, conflicting criteria (i.e., resolving what to do when several options have relative advantages over one another) can be a major challenge. The committee members were familiar with methodologies that have been developed to structure decision making with multiple stakeholders and criteria. The committee found, however, that these methodologies have not been widely used for infrastructure decisions. Some of the issues related to these methodologies are discussed in Chapter 5. Experience has shown that it is important that the technique not come to dominate the process.

A principal value of the committee's assessment process, described here, is that it structures and promotes interactions among stakeholders leading to a better understanding of community objectives and of the role of infrastructure in realizing those objectives. Building such an understanding is the best way to create performance measurement schemes that are responsive to the needs of decision makers. Even when a consensus is not reached, a structured approach yields the benefits of an orderly framework for debate.³

On the other hand, there are costs associated with structured processes and there are limits to how far a process Can take a community in realizing its objectives. The assessment process takes time and money, both of which may be constrained by budgets or the need for rapid action. If the process becomes overly formalized and bureaucratic, it will be of little value for responding to crises or short-term problems, and there is the danger that assessment will be delayed—or intentionally drawn out—and decisions and actions never realized.

If it is successfully carried through to completion, the performance assessment process provides a basis for making decisions and taking action regarding infrastructure development or management. The types of infrastructure decisions that are made fall broadly into three categories (see Figure 3-4): planning (including both early concept development and facility design), implementation (including construction and enterprise formation), and evaluation (as both a prelude to problem solving and a review of what has been accomplished after planning and implementation).

Planning is generally the earliest stage of infrastructure's evolution. Planning may include such elements as design and priority programming.

Decisions to be made may be structural or policy-oriented, involving construction of new facilities, refurbishment of old ones, pricing and operating policies, establishing or altering land use plans, and other actions typically having the potential for broad systemwide impact. As Figure 3-4 shows, the key question typically shaping the decision concerns the relationship of cost and performance that can be achieved. Planning is often the stage when the community's overall vision is explicitly considered and objectives for infrastructure's role are set. These objectives may include meeting federal mandates or regulations.

Identifying various alternative actions that could achieve objectives or realize the vision is a key step in planning. Stakeholders play an important role in identifying both the alternatives to be considered and their likely impacts. For instance, if the building of a new wastewater treatment plant is one possibility, then one group of stakeholders may raise the issue of nuisance from odor, while others may be more concerned about recycling of waste materials. The articulation of such concerns becomes a basis for establishing objectives and consequently selecting performance measures.

After the various stakeholders are heard and the costs and benefits are weighed, a preferred course of action is chosen. Performance may be only one of several aspects of ''benefits" upon which the decision is based. Infrastructure actions may have other benefits indirectly related to the services it is expected to provide, such as giving a temporary boost to the local economy. The decision on a preferred course is then typically reflected in an adopted plan, a capital budget, or some other document that guides subsequent action and decision making.

Decisions in the implementation stage are concerned with applying resources efficiently and effectively to realize a previously adopted plan. This stage brings a different set of objectives, now typically having to do with meeting budgets and time schedules and minimizing disruptions from construction or policy change. Stakeholders are likely to include operating entities, construction companies, and the residents of areas where actions will be taken but will typically be a smaller group of institutions and individuals than those involved in the planning stage. However, unresolved planning issues may reappear during implementation as the public reacts to specific policies or projects, shifting performance from what was expected on the basis of previous assessments.

Alternatives to be considered at the implementation stage include detailed aspects of projects selected in planning, such as the staging of construction, selection of process technology (e.g., for waste treatment), and specific hours during which peak-period pricing is to apply. Decisions

then represent strategies for plan execution and the start of operational management. Despite the more detailed aspect of implementation decisions, performance measures may reflect broad concerns such as the use of ethnic-minority contractors or employment of workers from a nearby economically disadvantaged neighborhood.

The third stage of decision is evaluation, which concerns the performance of existing infrastructure, that is, whether it is fulfilling the expectations of the community and whether improvements might be made. This third stage may in fact be the first opportunity for an area to undertake performance assessment.

If the process has been applied from planning to this stage, then the objectives set forth in planning, with refinements in the implementation stage, would be the basis for selecting performance measures. However, if the first performance assessment is to be made at the evaluation stage, for example, as an audit of an existing operating entity, performance measures must be developed or adopted from elsewhere. As in other stages, stakeholder input is instrumental in identifying objectives and adopting measures. Decision makers may be aided by benchmarks derived from infrastructure systems in other regions or standards from other levels of government.

In contrast to planning and implementation, there are no clearly defined alternative courses of action to be considered in evaluation. Decision making focuses instead on whether performance is acceptable or needs improvement. A decision that improvement is warranted should initiate a new round of planning, implementation and, eventually, future evaluation. A decision that performance is acceptable should be reconsidered at a later time (e.g., in annual or biennial performance reviews).

Many institutional units interact in the provision of infrastructure services, for example, agencies at several levels of government, private businesses, public interest groups, and private citizens. The impacts of infrastructure and, consequently, the responsibilities for those impacts cross infrastructure modes; for example, wastewater is generated by stormwater runoff from highways and affect the natural and built environment—air, groundwater and surface water, soils, wildlife, neighborhoods, and cultural and historic monuments. Existing institutions and institutional relationships generally do not match well the scope and structure of these prob-

lems. Agencies are typically mode-specific, making cross-cutting analysis and management difficult. Dealing with the full range of infrastructure impact, both positive and negative, requires a degree of interagency cooperation within and across levels of government that has rarely been achieved.

Each agency involved in infrastructure management typically has its own objectives and concerns. Some of these concerns may be nested within others (such as the need to control air quality at the national level translating into concerns for traffic congestion at local levels) but may be contradictory (e.g., improving the network connectivity of the interstate highway system versus providing public transit services at the local level). In addition, the impact from the construction or utilization of infrastructure may be felt locally, have regional or national consequences, affect other infrastructure subsystems, and be felt across generations.

For all of these reasons, it can be difficult to determine who makes the important decisions that influence infrastructure performance. The federal government may be involved in providing local infrastructure for a variety of reasons. Water and air pollution resulting from the use of infrastructure have a broad impact that may require federal action to control. National concerns such as security, international competitiveness, and productivity may not match well with local interests. Moreover, there may be insufficient technical, institutional, and financial capacity at lower levels of government to effectively manage all infrastructure.

Nevertheless, the subsystems of infrastructure exhibit many important functional interactions and relationships in budgeting and management . When budgets are allocated and used to make substantial improvements in transportation in an area, for example, levels of economic and social activity in that area often rise and increase demands for spending on water supply, waste removal, and other infrastructure services. In such a case, tax revenues often rise as well, although not necessarily enough to pay for increased infrastructure. On a more limited scale, financially constrained governments must allocate tax revenues among competing modes, and cannot increase spending on one element of infrastructure without reducing spending elsewhere. The committee found that effective performance management generally requires a broad systems perspective encompassing these interactions, despite their often poor match of agency responsibilities. This broader perspective will generally extend beyond the traditional limits of the public works budget.

There are a number of ways in which different institutional entities can interact to influence decisions and infrastructure performance. Government bodies may enact laws and impose regulatory standards or planning and coordination requirements. Private entities may impose such requirements and standards as well, for example, when banks or insurance com-

panies insist that borrowers meet certain conditions before financing is provided for infrastructure. Negotiation among stakeholders often is the decisive final basis for decision. Such methods are discussed further in Chapter 5.

In all these cases, performance assessment is a useful tool. As an orderly process yielding a debatable, defensible, outcome-based set of measures, performance assessment supports decision making and subsequent action. The committee recommends that responsible agencies adopt infrastructure performance measurement and assessment as an ongoing process essential to effective decision making. Adequate budgets should be maintained to support the continuing performance assessment process.