California Pilot Test of the Ecological Approaches to Environmental Protection Developed in Capacity Research Projects C06A and C06B (2013)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

71 APPENDIX B Credit and Valuation Approach Summary Describing credits for different transportation and ecological actions was an intent of C06 and the C21 test of C06. No specific methods were described in C06, so we limited our investigations to conceptual discussions of crediting and valuation with transportation and other stakeholders. This introductory and conceptual approach is appropriate at this point because the field of valuation and economic valuation of environmental attributes (including benefit/cost analysis) is relatively new in transportation planning and decision-making. The approach described here formed the basis for presentations and discussions with transportation and other stakeholders in this C21 project. It also formed the basis for how two valuation approaches were conducted: 1) preference surveys to quantitatively describe stakeholder value systems and 2) impacts analysis conducted with impacted-area as the currency of valuation. Because there were no specific projects defined in this study, there was no credits system developed or tested. However, we felt that the concepts were sufficiently well-introduced and supported by both Caltrans’ previous research into valuation and guidance from TRB/FHWA that it is possible that corridor management could adopt this form of decision-support in the future. What Are Credits? In order to plan for infrastructure development in complex social-ecological systems, it may be necessary to create devices that draw equivalencies among non-like objects. Credits are one type of device that use units of measure that are native to part of the system (e.g., Ha of land), or derived from financial calculations (e.g., $-equivalents), or that are normalized on a preference scale of some kind (usually from least to most preferred). In the current study, credits are units of value whereby dissimilar attributes of the Highway 37 Corridor Context can be compared in planning, impacts analysis, programming, and mitigation budgeting. Credits in this study are proposed as scores on a scale from 0 to 100 given to alternatives for five themes: Transportation, Environment, Cost, Community, and Reversibility. Each theme is accompanied by indicators of impact within each theme, which allows the development of stewardship-oriented scenarios, as well as evaluation of the actual impacts that accompany each scenario. The normalization of impacts to a 0 to 100 credit scale can serve as an intermediate step for subsequent conversion to fiscal equivalents for system attributes for which fiscal equivalents are known. Because these equivalents are approximate at best, the unit-less credit scale permits valuation without the inexactness of monetizing benefits and dis-benefits (including costs) of various project choices. For the environmental theme for this corridor, the nearby tidal and freshwater wetlands provide both constraints and opportunities for stewardship

72 planning. Because of the unique potential for wetland restoration in the State Route (SR) 37 Corridor Context, there may be few possibilities for mitigation bank strategies or payment of ecosystem services. However, even if mitigation banking might not be appropriate in this corridor, if wetlands around SR 37 are restored, these activities could confer credit benefits to other project areas. What is Valuation? Choice of the Valuation Method Two Valuation Methods Considered: Crediting Strategy and Monetary Valuation In its application to pilot test the tools from C06A and B, UC Davis proposed to use two approaches for the sixth step, develop crediting strategy. The first approach is to use one of the products of the C06B project: a credit system, as one accounting system for ecological, economic, and equity effects of decisions. To be functional in this system, the accounting or credit system would provide a way to both indicate relative or absolute effects or impact and to measure potential performance of credits, usually in the context of mitigation. In our case, this valuation will be based on a value given by normalization and aggregation of indicators on a defined scale (0 to 100, for instance). An alternative framework based on Caltrans’ existing valuation approach for impacts, developed in collaboration with the UC Davis Road Ecology Center and Sustainable Transportation Center is also proposed, and this method is based on monetary values to evaluate impacts. The aim with the combined approach is to contribute to a more complete accounting of environmental, economic, and equity impacts of transportation early in decision-making, including describing a crediting strategy. We describe in the next sections how each of these two methods works. However, for our study, we will use only the valuation approach based on a crediting strategy, for reasons developed below. Use of Monetary Values for SR37 The use of monetary value gives a common scale for the valuation of impacts. Such dollar values for some impacts (emissions, for instance) are already used for Cost Benefit Analysis (CBA) by Caltrans, more precisely in life-cycle benefit/cost analysis. Such analysis is performed using a model called Cal-B/C1 and impacts such as accidents or vehicles emissions are monetized in this type of analysis. But others impacts such as noise or water pollution should be monetized as well. Many wetland functions, for instance, result in goods and services that are not traded in markets and therefore remain un-priced. It is then necessary to value these goods or services using a non-market valuation technique. For SR 37, monetization of wetlands would be a key step and we will discuss its implementation as an example of how we could give a dollar value to impacts for our project. The first step for monetary valuation is to understand what characteristics of the wetland can be valued, so we will first summarize the functioning, uses, 1www.dot.ca.gov/hq/tpp/offices/ote/LCBC_Analysis_Model.html

73 and values of wetlands. Then we will present valuation methods and why they cannot be implemented in our project. Monetization Process for Wetlands FUNCTIONING, USES, AND VALUES OF WETLANDS The functioning of the wetland comes from different ecological processes (e.g., photosynthesis), characteristics (e.g., water depth), and structure (e.g., fauna and flora). Then, wetland uses result from the functioning of the wetland (Figure B.1). Wetlands uses contain both wetlands services (e.g., flood control) and goods (e.g., fisheries). It is at this stage that connection is made between ecology and economy since wetland uses can be monetized because links can be made between wetland uses and human activity. Yet monetization of wetland uses is not direct and it depends on what type of use is considered. In addition, decision-making regarding wetlands does not have to rely upon monetization as the only way to include wetlands’ value in decision-making. • Goods provided by wetlands have a direct use value, so they can be monetarized with market analysis, contingent analysis, and mitigation costs • Indirect use value can be found for some wetlands services. For instance, flood control can be monetarized by the costs of maintenance of levees. Contingent analysis or hedonic prices can also be used to monetize these services. • Some services, like knowing that the wetland exists, don’t have a use value. Therefore, contingent valuation must be used for these services. Main economic valuation techniques are described in the next section.

74 Source: Turner et al. 2000. Figure B.1. Connections among wetland functions, uses, and values.

75 Valuation Methods Three valuations methods that we could use to value wetlands are revealed and stated preference methods, contingent analysis, or benefit transfers. REVEALED AND STATED PREFERENCE APPROACHES The two main types of valuation for non-market goods (wetland services, in our case) are the revealed preference methods and the stated preference methods. Revealed preference approaches depend on a connection between the non-market good of interest (for instance, noise) and a market good (for instance, housing). The method uses data revealed by behavior related to actual decisions (for instance, changes in prices of housing). The major problem of this method is that it is based on existing conditions and so the possibilities of alternatives are limited. In contrast, stated preference techniques are based on hypothetical situations and surveys that are used to determine people’s willingness to pay for a situation. Contrary to revealed preference methods, stated preference methods can be used for environmental goods like a wetland where we have both use and non-use values. THE CONTINGENT VALUATION METHOD Contingent valuation method is a stated preference method and it is usually used to estimate the value of an environmental change scenario. The method implies the use of a survey which begins with a statement describing the change in environmental goods or services. Then it asks individuals to reveal how much they are willing to pay for the change. For example, we could ask people how much they are willing to pay to restore wetlands surrounding Highway 37. In theory this method can be used to estimate values for environmental resources and ecosystem services, including those supporting both use and non-use values, which is what we need to get a valuation of wetlands. However, respondents must understand the nature of what is being valued as well as be able to know how they would be willing to trade off between changes in the environmental attribute and their income. This is a major difficulty for the contingent valuation method which can vary a lot between places and respondents. It especially depends on people’s income. Another issue with contingent valuation is that this method is time- and resource- consuming, since it is based on surveys. In our case, the time limit does not allow for a contingent valuation of wetlands. BENEFIT TRANSFERS FOR WETLANDS Benefit transfer is defined as the transfer of existing estimates of non-market values to a new study which is different from the study for which the values were originally estimated. It is a secondary approach for valuation. This method is often used mainly because it saves time and resources. Usually, benefit transfer is best suited for tasks where the need for accuracy is low and it is generally considered a “second best” valuation method because benefit transfers involve reusing existing data, and a benefit transfer does not provide an error bound for the value in the new application after the transfer.

76 Since contingent valuation method would be time and resource consuming, benefit transfers were considered for our study. However, “A recent review by Heimlich et al. (1998) lists 33 studies over the last 26 years with per acre values ranging from US$0.06 to US$22050. Even within the same study looking at a single ecosystem function, Batie S.S., and Wilson (1978) find values per acre that differ by two orders of magnitude from one site to another.”(Woodward et al., 2001). This study shows that variability comes from the methodology used for the evaluation and insists that in-site studies should still be used, knowing the potential biases of valuation methods. Therefore, it is less desirable to use benefits transfer to estimate wetland’s value and we will only use the valuation method based on credits proposed in C06B. Crediting Strategy in C06B Approach Although C06 proposed a list of steps to be followed (Table B.1), the choice of the methodology was broad because these steps were not detailed and they can be seen as a list of what the evaluation should include rather than a precise guideline. Therefore it leaves a lot of possibilities as to how the evaluation will be conducted. Table B.1. Purpose and Implementation of Step 6 from C06B Project (TRB, 2011). Step 6: Develop Crediting Strategy Purpose: Develop a consistent strategy and metrics to measure ecological impacts, restoration benefits, and long term performance – with the goal of having the analyses throughout the life of the project be in the same language. Implementation Steps: 6a. Diagnose the measurement need. Examine the ecological setting (including regulated resources and frameworks, non-regulated resources, and ecosystem services); examine the regulatory and social setting, and identify additional opportunities. 6b. Evaluate ecosystem and landscape needs and context to identify measurement options. 6c. Select or develop units and rules for crediting (e.g., rules for field measurement of ecological functions, approved mitigation/conservation banking, outcome-based performance standards using credit system). 6d. Test applicability of units and rules in local conditions.

77 6e. Evaluate local market opportunities for ecosystem services. 6f. Negotiate regulatory assurance The crediting strategy can be seen as a multi-criteria analysis, as we can see in our implementation of Step 6 to SR 37 study described below. Choice of the Crediting Strategy Approach If we use a valuation approach, we look at economic values of environmental impacts and we have to use contingent valuation for different types of values, which demands time and resources. The difference with a crediting strategy (multi-criteria analysis) is that the crediting strategy uses indicators from the wetland structure and characteristics and not only its uses. This approach can be linked to strong sustainability as opposed to the valuation approach which is closer to weak sustainability: when we use economic valuation, we do not take the irreversibility factor into account. Giving an economic value to a wetland might also mean that the benefits from this wetland are the same as benefits from another ecosystem, which we can buy through mitigation banking. But what is not considered here is the irreversibility of the damage caused to the wetland and factors like the uniqueness of the considered wetland. Therefore, economic value as it is used here only considers part of the total value of the wetland. Definitions (Joumard et al. 2010): Weak sustainability : “According to the weak approach of sustainable development, the natural capital is a component of the total capital composed by all the productive goods, so-called productive capital, the human capital and the stock of knowledge and know-how of the people, so-called social capital, and the resources and natural goods, renewable or not, so-called natural capital. These different types of capital are supposed measurable and equivalent. The annuities due to the use of the natural capital by the present generation can be reinvested in the form of a reproducible economic capital, to be transmitted to the future generations. […] In these conditions, the sustainable development of an economic sector is not limited by an ecological constraint.” Strong sustainability: “The second variant of sustainable development is the strong approach, which claims the irreducible character of the natural capital. It means that the sustainable development should comply with the ecological constraints due to the preservation of the quantity and the quality of the natural capital, i.e. the nature.”

78 Also, the aim of our study is to help decision making by stakeholders through a better knowledge of impacts, and indicators might be a better approach as they are more transparent and can be easily understood, unlike economic valuation. Economic valuation is made through methods like contingent valuation and then uses concepts such as discounting which are not as easy to understand as a range of indicators. Thus, it might be easier for stakeholders to discuss a rather simple evaluation in which they can discuss different objectives described by indicators and weights of indicators. This would help decision making more than an economic valuation because stakeholders can easily discuss every point and by that process get a better understanding of potential impacts and concerns. Relationship to C06 Approaches The general methodology chosen for this study is described below. We divided it into the substeps developed by C06 team and adapted to our project (Figure B.2). Substeps For each substep, we will give a summary of C06B recommendations and describe how we will implement them in the SR 37 study. Step 6a: Diagnose the measurement need: examining the ecological setting, the regulatory and social setting, identifying additional opportunities. C06B RECOMMENDATIONS This first substep is targeted at diagnosing the resource measurement needs. It is divided into three parts. The first part is the ecological setting: examining natural environment and resources in the area. The second part is the regulatory and social setting, which can be examined through a historical review of stakeholder’s experiences and a forward-looking review that evaluates potential regulations or social expectations from projects. The third part, additional opportunities, can be evaluated by examining ongoing efforts and conservation programs. APPLICATION FOR THE SR 37 STUDY The draft Caltrans Corridor Plan for SR 37 was used as a basis for this step as it already provided a description of the corridor including its transportation characteristics (current and forecasted), environmental constraints, and previous and potential future projects. The regulatory setting is already described in the Corridor plan.

79 Figure B.2. Application of C06B framework for Step 6 to SR 37 study. Review by stakeholders

80 In order to get a global image of ecological and social setting, we developed a matrix divided into themes, objectives (or criteria), and indicators (Figure B.3). Figure B.3. Hierarchy of themes, criteria, and indicators. Definition of Themes and Objectives The matrix was first organized into different themes and objectives listed by TCAPP: environment, transportation, cost, economy, and community. It was then adapted to the SR 37 study. Other themes and objectives were added to match the SR37 study, like objectives concerning wetlands. Objectives will be used as criteria for each theme. For this step, meetings and feedback from stakeholders can help with getting to know different impacts that were not previously listed, as agricultural impacts linked to sea level rise and wetland restoration. Also, this matrix can be compared to and, if needed, completed by the lists of impacts listed in Caltrans’ existing valuation approach for environmental impacts developed in collaboration with UC Davis Road Ecology Center and Sustainable Transportation Center. Definition of Indicators for Each Criterion With help from the TCAPP website and depending on available data, each criterion (impact/measure listed) will then be linked to one or several indicators that can be provided for which data can be provided. TCAPP web tool also includes lists of existing studies for several indicators. That can eventually be carefully transferred to SR 37 study if we have a lack of data and if a transfer is consistent with both the existing study and SR 37 characteristics. Step 6b: Identify measurement options. This step mainly aims at defining units for managing the resources. C06B RECOMMENDATIONS In this section, various existing measures used in environmental management settings are presented: condition-based measurements, model-based measurements, and function-based measurements.

81 Condition-Based Measurements Condition based measurements focus on quantifying changes in the status of the regulated resource. For instance, species of concern would be measured through population surveys. These systems also include pollutant load measurements, which are normally defined by quantifying specific amounts of criteria pollutants added or removed from the system. Condition-based examples include water quality measurements, and indices of biological integrity. Two forms of condition-based measures are indices of environmental quality and observation-based systems. Model-Based Measurements This type of measures relies on data to estimate species or ecosystem response and on a set of rules and conditions that are expected to result in an environmental outcome. Model-based systems are similar to condition-based measurements systems, but are usually employed for planning purposes because they focus not only on sample-based data but also on the elements of the ecosystem that can be affected by human action. Function-Based Measurements These measures focus on habitats, structures, and processes as the basis for measuring the environment. Function-based systems are not species-specific, and are used when rare or unique resources need measures, but are not easily measured with one species. Model-based measurements can start to combine elements of a function-based measure and a condition-based system where the model relies on habitat or field data to estimate habitat use and densities. To truly get at a measurement for use in transportation projects, the results need to tie the natural impacts back to specific actions at a site. This is needed for the full suite of mitigation decisions: avoidance, minimization, and compensation. These concerns need to guide the selection or development of a measure. APPLICATION FOR THE SR 37 STUDY Given the sensitivity of resources, SHRP C06 research recommends that functional measures are used in the study. This approach should provide “a common unit of measurement for biological, chemical, and physical processes”. The dollar could have been this common unit, as recommended by Caltrans’ existing valuation approach for environmental impacts developed in collaboration with UC Davis Road Ecology Center and Sustainable Transportation Center. The reasons why we won’t use this approach were explained earlier. Using models of wetland condition and processes could be an appropriate basis for valuation in the SR 37 study. Caltrans provides forecasts for transportation data as well as ecological models on impacts, such as noise. But a major challenge for the ecological impact concerns wetlands, and wetlands are a very complicated ecological system. Not all of its functioning is well understood, especially when hydrology is concerned. Therefore, it remains difficult to conduct model-based measures for wetlands because it is difficult to find out exactly how indicators of wetlands well-being will react to alternatives. Therefore, we will use

82 condition-based measures in our study: for instance, species of concern would be measured through population surveys. Step 6c: Select or develop units and rules for crediting. C06B RECOMMENDATIONS In this section, C06B provides recommendations to develop a custom measurement system for multi-resource crediting: define the spatial unit, develop a conceptual diagram, generate attributes (criteria) and scores, check attributes, and check that measures can work at any point of time. All rules developed during this process must be agreed upon. APPLICATION FOR THE SR 37 STUDY The method proposed here and summarized in Figure B.4 is rather simple and transparent. These qualities were needed here because they will help the discussion between stakeholders who can discuss every indicator. Figure B.4. Implementation of Step 6c. In the matrix of impacts constructed earlier, each theme will be equally-weighted. Each criterion inside each theme will then be weighted. A conceptual diagram was developed with the matrix of impacts and can eventually be used to help with the choice of weights or to show how weights are distributed, following C06 recommendations. The issue with these diagrams that we

83 have is that some impacts like water quality appear at different points of the diagram (for instance, as a consequence of runoff water or as a consequence of vehicles emissions). Therefore, weighting cannot be based on the conceptual diagram because we would have too much double counting, but it will help stakeholders to have a more comprehensive understanding of the system. Each criterion will then get a score depending on the performance of the alternative considered for this criterion through a normalization process of its indicators explained later. When we have the scores, we can calculate a score for each theme. No global score will be given for each alternative because the idea of the evaluation is to help dialogue among stakeholders and between Caltrans and regulators. Thus, the most important part of this work is to give stakeholders the best possible knowledge of potential impacts of different alternatives in order for them to make the best decision. That’s why it may be more important to discuss different weighting options and aggregation options than to have a final score. The final result of the evaluation study should give a good idea of both positive and negative impacts of alternatives, which should ultimately help stakeholders weight their own concerns regarding the other stakeholders’ concerns and then hopefully come to an agreement on the alternatives. One issue about this approach is that weights of themes and criteria are decided before the valuation and therefore the results of the valuation depend on how each theme was previously weighted. However, different weight options can be used in order to represent different approaches or points of view on the project. The normalization process and the weighting will be reviewed for uncertainty and sensitivity. Uncertainty tests look how the uncertainty of an indicator can spread and affect the global evaluation and sensitivity tests focuses on how much a single indicator affects the evaluation. Sensitivity tests for these two parts of the evaluation will be made to check the consistency of the indicators. Then, stakeholders should review the results to agree on results and values used in normalization and weighting. The selections of criteria and indicators will be discussed with stakeholders as well as how each weight and determination of benefits and dis-benefits during the evaluation process could be useful as this can be a basis to determine credits. Finally, ways to deal with the benefits and dis-benefits will be developed. Step 6d: Test applicability of units and rules in local conditions. C06B RECOMMENDATIONS The application is recommended in three steps: determine baseline condition using on-site data, generate alternative scenarios, and evaluate future conditions. APPLICATION FOR THE SR 37 STUDY A current and future condition evaluation is generated for each alternative considered. This means that a precise definition of the alternatives should be produced at this point or earlier by

84 stakeholders. Defining alternatives earlier could be useful because it can help determine objectives and needed data. Step 6e: Evaluate local market opportunities for ecosystem services. C06B RECOMMENDATIONS Market opportunities include existing conservation/mitigation banking systems or payment for ecosystem service (PES). PES programs are negotiated contracts with landowners to maintain a certain level of environmental performance to maintain or enhance ecosystem services. Examples of PES can be found in Forest Trends and Ecosystem Marketplace, 2008. Developing ecosystem metrics and tracking project impacts using those measures can make it easier to access any operating regional ecosystem markets. If ecosystem markets are available and if metrics were developed from previous step, then the ecosystem measurement system should be well-suited to use within the ecosystem market. Ecosystem markets present various benefits for departments of transportation: • First, they remove the risk of uncertainty of the project linked to the needed approval by environmental agencies. Projects are often slowed or stopped by deficient environmental analysis like the Environmental Impact Report (EIR) required by federal and state laws: National Environmental Policy Act (NEPA), California Environmental Quality Act (CEQA), or Clean Water Act, for instance. • Second, ecosystem markets include a transfer of liability: the liability for the restoration or conservation success is placed on the banker and not on the department of transportation. • Third, this system produces a better alignment of mission since instead of road constructors, restoration professionals build mitigation sites. • Fourth, ecosystem markets can produce improved ecosystem outcomes because bankers can have more comprehensive and meaningful projects to address ecosystem priorities. But although PES systems have great potential power for ecosystem preservation, according to Redford and Adams, seven major criticisms can be listed (Redford and Adams, 2009), including the risk that economic arguments about services valued by humans will overwrite and outweigh noneconomic justifications for conservation and the concern that there is no clear way to track the performance of the system. Therefore, ecosystem markets must be only one of several tools aiming at preserving ecosystems. APPLICATION FOR THE SR 37 STUDY The unique potential for wetland restoration in the SR 37 setting might not make bank strategies or PES sufficient mitigation strategies in this case. Indeed, in the geographic setting of SR 37 there is a low housing density and development (cf. urban areas in Figure B.5). This makes this

85 place a unique opportunity of wetland restoration for the Bay Area and this nationally-important estuary. Therefore, since the ecosystem is unique, banking systems or PES might not be a satisfying approach for this project because they imply that mitigation or restoration projects can be equivalent to the impacts, which is not the case for unique systems. Indeed, widening Highway 37 would have irreversible impacts that cannot be compensated by another wetland project because no other wetland project has the same potential benefits in the Bay Area. However, if mitigation banking might not be sufficient for this project, if the removal alternative or the causeway alternative is considered, these wetlands could become a mitigation bank themselves and receive money from crediting strategies from other projects. It would also be a way to pay for the extensive wetland restoration. Source: Caltrans draft Corridor Plan, 2010. Figure B.5. Geographical setting of Highway 37. Stakeholders Role in the Evaluation Process Stakeholders are supposed to participate at every step of the evaluation. The crediting strategy described here is designed to get a maximum involvement of stakeholders. Discussion about objectives will help identifying and discussing concerns; discussion about criteria will help sharing data; and discussion about weighting will help getting a shared agreement on priorities.

86 The construction of alternatives will also necessitate the involvement of stakeholders, because of the various issues that have to be addressed. Stakeholders at this point will help in constructing better alternatives because they can share their own expertise that other stakeholders may not have. Construction of alternatives is an important step and it will also lead to better knowledge of concerns and potential impacts. Therefore, this step can help in defining objectives and criteria and thus it should take place early in the process. The evaluation should help in understanding the benefits and impacts associated with each alternative scenario for the highway. Benefits and impacts can be used to optimize alternatives, where optimization limits imapcts and maximizes benefits. Implementation of the Method Choice of Themes and Objectives Themes and objectives are the two levels under which the project will be evaluated. Therefore, this choice is a very important step. The question here is: what do we want to measure? Stakeholders have various goals. For instance, Caltrans wants to reduce congestion, Sonoma Land Trust wants to restore wetlands, the Bay Trail wants bikes paths, US Fish and Wildlife Service want to preserve biodiversity and protect listed species, and land-owners want levees to protect their land from flooding. Themes and objectives must reflect all these different intents in order to inform agreements. TCAPP proposes a list of themes and objectives on its web tool. It can be used as a basis and adapted to SR 37 after feedbacks from stakeholders. Themes for SR 37 could be Transportation, Environment, Cost, Community, and Reversibility: • Transportation could include objectives like reducing congestion and delays, reducing the risk of injury-causing accidents, and improving accessibility. • Environment could include wetland conservation, tidal wetland adaptation to sea level rise, air and water quality, preservation of habitat and biodiversity, and noise pollution. • Cost could include infrastructure and mitigation costs, cost effectiveness and economic impact, and avoidance of future catastrophic costs from sea level rise. • Community could include objectives like land preservation, historic preservation, equitable distribution of transportation costs and benefits, access to recreation, community cohesion, and public health. • Reversibility would measure the “possibility of re-orienting, or cancelling the project once finished, according to future choices” (Joumard et al. 2010). Criteria for the Choice of Indicators Quality of basic data affects the quality of the overall evaluation. Several dimensions, listed in OECD Handbook (Nardo et al. 2005), need to be considered while selecting data:

87 • Relevance The relevance of data is a qualitative assessment of the value contributed by these data. Value is characterized by the degree to which statistics meet current and potential needs of the users. • Accuracy The accuracy of data is the degree to which they correctly estimate or describe the quantities or characteristics that they are designed to measure. Accuracy is usually measured in terms of the error, or the potential significance of error. • Timeliness The timeliness of data products reflects the length of time between their availability and what they describe. The punctuality of data is also important: it depends upon the existence of a publication schedule and reflects the degree to which data are released in accordance with it. • Accessibility The accessibility of data products reflects how readily the data can be located and accessed. It implies, for instance, distribution channels, pricing policy, affordability, copyright, and suitability of the form in which the data are available. • Interpretability The interpretability of data products reflects the clarity with which the user may understand and analyze the data. It reflects how well the indicator varies with what it represents and how it is influenced by uncertainties. It should move in an analogue fashion to the phenomenon. The choice of basic data mainly depends on its availability in the area of concern and the quality of the overall evaluation depends on the coherence between indicators and not only on the quality of each single indicator. However, these criteria will be used as guidance when a choice between indicators can be made. Double Counting The problem if the evaluation is based on objectives is the potential double counting because the same criteria can be used to assess different objectives. For instance, water quality can be an indicator for the preservation of wetland or wildlife, but it can also be used for the objective of human health. But even though evaluation based on objectives can lead to double counting, it

88 might be more important to see how far each objective is reached with different alternatives than how many times an indicator was used. Also we can consider that double counting is useful to some extent because if a resource is a valid indicator for different objectives, then maybe it should be counted twice because it serves two different objectives. Normalization of Indicators Once we have our criteria and indicators, the next step is to normalize them to a common value scale. Several normalization processes are described. Choice of the Type of Normalization Different methods exist for normalization, although none of them is totally satisfying. Table B.2 below summarizes the main methods and their advantages and disadvantages. Table B.2. Comparison of Methods for Normalization Method Advantages Disadvantages Empirical normalization Min max method gives the 0 value (Min) to the most unfavorable observed value and 1 or 10 (Max) to the best recorded value. All intermediary values are calculated based on the formula: Y = X – Min/(Max – Min). Simple and efficient to compare alternatives with an initial state. Variability of Min and Max values that depend on observed values, new observation outside the previous limits will lead to new normalization. Extreme values/or outliers could distort the transformed indicator. Axiological normalization Close to the empirical approach with min and max limits. The limits are not statistically identified, being chosen based on the undesirable situation, which receives the “0” value, and on the ideal situation, which can or cannot correspond to a strategic objective and which receives the value “1”. Alternatives to min and max here are : • Distance to a reference method that takes the ratios of the indicator to a value of mean reference for this indicator: Y=X/Xexpected

89 • Indicators above or below the mean: this transformation considers the indicators which are above and below an arbitrarily defined threshold, p, around the mean Xexpected: Simple and efficient to compare alternatives. Reduced impact of extreme values. Might be less realistic than the empirical approach because limits depend on objectives, not on observations. Mathematical normalization Transformation of data by means of a mathematic function in order for the values to range between an upper and a lower limit. Lack of transparence for the user and possible change of initial distribution of values. Statistical normalization All values are expressed in standard deviation, so that the variables average is equal to zero. Does not depend on min and max values determined by strategic objectives or statistics. Does not depend on min and max values determined by strategic objectives or statistics. Since the aim of the study is to get stakeholders involved in a more comprehensive process, transparency is important. Therefore, an axiological or empirical normalization would be better here because stakeholders can easily understand and discuss indicators since they understand the normalization process. An empirical normalization is preferred for our study because we aim at having few alternatives (three to five), and therefore an axiological normalization could distort the reality of the impacts of each alternative by comparing them on a reduced scale.

90 Positive versus Negative Count of Impacts Another question here is to choose how we want the indicator to be read: more is better or less is better? The appreciation by stakeholders might be different for some indicators. For instance, congestion can seen as a “less is better” indicator because drivers earn time when the road is less congested and this is counted as positive impact with a positive value of time which derived from the observation that people are willing to pay to save time. However, we can look at congestion from another point of view: congestion might be an indicator for which more is better because if the road is congested people might want to avoid congestion by using other modes or by car sharing, if these alternatives are available. Or they might also want to live closer to their work, which would limit urban sprawl. In that case loss of time consequent to a transportation project can be seen as a positive impact from a transportation and accessibility point of view. Therefore, the direction of each indicator (more is better or less is better) must be derived from the objectives. Aggregation of indicators and criteria Choice of the Aggregation Method Aggregation is the process through which several indicators are summarized into a single index. The questions related to aggregation are: Do all indicators have the same weight? If not, how should weights be determined? What mathematical function will we use to aggregate indicators? In our study, a simple and transparent method is preferred since it is necessary to get stakeholder’s involvement. Therefore we will calculate the mean of aggregated indicators. Choice of the Weighting Method Weighting methods have various advantages and disadvantages (Table B.3). The weighting method that seems the most accurate for our study is the budget allocation method, because of its transparency and easiness to implement. This method could include some stakeholders as experts. The idea is to ask stakeholders how they would weigh the criteria in their field: transportation criteria should be weighted by transportation stakeholders, environmental criteria by environmental agencies, and then average weight could be used as weights. However, we cannot to have too much difference between weights for the same criteria, so the given weights might need to be transformed in order to get an arbitrarily-defined standard deviation. If a standard deviation is too high it might be an indicator that the criteria is not accurate, in which case the criteria can be changed..

91 Table B.3. Advantages and Disadvantages of Some Weighting Methods Weighting method Advantages Disadvantages Hierarchical Weighted Total (aggregation on tree) When criteria can be organized in a tree, weights are attributed to each single indicator and to all combinations of indicators belonging to the same node, at all different levels of the aggregation tree. • Simple to use and transparent. • Difficulty in constructing the tree. • The creation of a tree is not always possible. Public Opinion Similar to budget allocation, people are asked to express their degree of concern (e.g. great or small) about issues, as measured by indicators. • Allows all stakeholders to express their preference and creates a consensus for policy action. • Implies the measurement of “concern” (see discussion on the Budget Allocation). • Could produce inconsistencies when dealing with a high number of indicators (see discussion on the Budget Allocation). Budget Allocation Experts on a given criteria are asked to allocate a “budget” of 100 points to the indicator set, based on their experience and subjective judgment of the relative importance of the respective indicators. Weights are calculated as average budgets. • Weighting is based on expert opinion and not on technical manipulations. •Transparent, relatively straightforward nature and short duration. • Expert opinion can increase the legitimacy of the evaluation. • Weighting reliability : Weights could reflect specific local conditions (e.g., in environmental problems), so expert weighting may not be transferable from one area to another • Allocating a certain budget over a too large number of indicators may produce inconsistencies ( for a number of indicators higher than 10). • Weighting may not measure the importance of each individual indicator but rather the urgency or need for political intervention for the individual indicator concerned.

92 Analytic Hierarchy Process Pairwise comparisons of indicators are made and then the relative weights of the individual criteria are calculated using an eigenvector. • Can be used both for qualitative and quantitative data. • Weighting is based on expert opinion and not on technical manipulations. • Expert opinion is likely to increase the legitimacy of the composite and to create a forum of discussion in which to form a consensus for policy action. • Requires a high number of pairwise comparisons and thus can be computationally costly. • Results depend on the set of evaluators chosen, therefore not reproducible. Conjoint Analysis Surveys are conducted asking for an evaluation (a preference) of a set of alternative scenarios. A scenario might be a given set of values for the individual indicators. The preference is then decomposed by relating the single components (the known values of individual indicators of that scenario) to the evaluation. • Weights represent trade-offs across indicators. • Takes into account the socio-political context and the values of respondents. • Time and resource consuming. • Depends on the sample of respondents chosen and on how questions are framed. • Could produce inconsistencies when dealing with a high number of alternatives (see previous discussion on the Budget Allocation). • Requires a large sample of respondents and each respondent may be required to express a large number of preferences. • Estimation process is rather complex Sensitivity and uncertainty tests Several subjective choices have to be made during the evaluation process including: • Choice of indicators, • Definition of criteria, • Choice of aggregation process, and • Choice of weighting model.

93 We cannot suppress the subjective factor of our evaluation and the message given by the final evaluation results is determined by these choices. The uncertainty test aims to quantify the overall uncertainty in themes rankings as a result of the uncertainties in the model input. The aim of sensitivity analysis is to assess the evaluation impacts associated with the subjective choices taken. Sensitivity analysis studies how the variation in the outcome can be caused, qualitatively or quantitatively, by different sources of variation in the indicators. Sensitivity analysis and uncertainty analysis are thus closely related. Using both uncertainty and sensitivity analysis can help: • To assess the robustness of the final ranking, • To increase its transparency, • To identify which themes or objectives are favored or weakened under certain choices, and • To help frame a debate around the index. These tests also help to identify benefits and dis-benefits in the evaluation process: if we change one weight, how does that affect each stakeholder? Graphic Representation of Evaluation Outcomes Alternatives are evaluated under several themes, which are themselves divided into several objectives, and these objectives are evaluated through a range of indicators (cf. Figure B.4: Implementation of Step 6c). Therefore, we can disaggregate the evaluation into three levels. The outcomes of our study should be the two main levels: • Evaluation of alternatives’ general performance related to each theme (Figure B.6), and • Evaluation of performance of alternatives for the objectives within the theme (Figure B.7).

94 • Figure B.6. Example of spider diagram for themes. Figure B.7. Example of spider diagram for environmental objectives.

95 The combination of these two levels will give a good sense of how each alternative is performing and it will also make it easier to discern benefits and dis-benefits. The use of spider diagrams like these is clear and it permits one to see how different alternatives are performing on multiple criteria and clearly distinguish strong and weak points of each alternative. Comparison of Alternatives As shown in figures B.6 and B.7, we can compare alternatives with a spider diagram. Within the framework of sustainable development, the most circular alternative will be preferred as it shows equilibrium among criteria. A weak point of spider diagrams is that a change in the order of criteria can affect the perception of the performance. In order to mitigate this weakness, a number of criteria shown on the diagram must remain small (less than eight) and a matrix summarizing the evaluation will be presented in addition to spider diagrams. At this point of the evaluation process, an identification of benefits and dis-benefits in each alternative could be made by identifying each stakeholder with his/her objectives. However, this might also not be useful in the decision process as some stakeholders will be characterized as beneficiaries which might lead to opposition to them by non-beneficiaries. In the crediting strategy proposed by C06B however, designation of who benefits and does not is useful as this can be a basis to determine credits and then develop negotiation among relative beneficiaries.

96 References San Francisco Bay Conservation and Development Commission (BCDC), Draft Staff Report. Living With A Rising Bay: Vulnerability And Adaptation In San Francisco Bay And On the Shoreline, 2009. 175 pages. Boulanger P.-M., Les indicateurs de développement durable : un défi scientifique, un enjeu démocratique. Les séminaires de l'Iddri,2004, n°12, Paris, avril 2004, p. 24. Carraro C. , F, Ciampalini, C. Cruciani, S. Giove, E. Lanzi, Aggregation and Projection of Sustainability Indicators: a New Approach, Paper prepared for the OECD 3rd World Forum, 27- 30 October 2009, Busan, Korea. Department for Communities and Local Government: London. Multi-criteria analysis: a manual. 2009. 168 pages. Forest Trends and Ecosystem Marketplace. Payments for Ecosystem Services: Market Profiles. PROFOR. 2008. 35 pages. Groot, S. D. Environmental functions as a unifying concept for ecology and economics. Environmentalist, 1987. 7(2):105-109 Joumard R. and Gudmundsson H. (eds), Indicators of environmental sustainability in transport: an interdisciplinary approach to methods, INRETS report, Recherches R282, Bron, France, 2010. 422 p Joumard R., Mancebo Quintana, S., Arapis, G., Zacharz, T., Chains of causalities of environmental impacts. Seminar COST 356 EST ‘‘Towards the definition of a measurable environmentally sustainable transport’’, 20 February 2008, Oslo, Norway. Proceedings, Institute of Transport Economics/TOI ed., Oslo, 2008, pp.9–21 (http://cost356.inrets.fr). Joumard, Nicolas. Transport project assessment methodology within the framework of sustainable development. Ecological indicators, 2010, Volume 10, pages 136–142 Kerry Turner, Jeroen C. J. M. van den Bergh, et al., Ecological-economic analysis of wetlands: scientific integration for management and policy, Ecological Economics, 2000, Volume 35, Issue 1, October 2000, Pages 7-23 Lee, J.F.J., M. Springborn, S.L. Handy, J.F. Quinn and F.M. Shilling. Approach for economic valuation of environmental conditions ad impacts. Prepared for Caltrans. 2010. 124 pages

97 Mogridge, M. J. H., The self-defeating nature of road capacity policy. 1997. Transport Policy, Vol. 4, No. I, pp. 5-23 Nardo M., Saisana M., Saltelli A., Tarantola S., Hoffman A. and Giovannini E., Handbook on constructing composite indicators: methodology and users guide, 2005, OECD-JRC joint publication, OECD Statistics Working Paper, STD/DOC(2005)3, JT00188147, 108 p. Navrud, S., and G. J. Pruckner. Environmental Valuation - to Use or Not to Use? A Comparative Study of the United States and Europe. Environmental and Resource Economics, 1997, 10:1-26. NRC (National Research Council). Valuing Ecosystem Service. Washington, D.C.: The National Academies Press. 2005. Redford, K. H. and W. M. Adams., Payment for ecosystem services and the challenge of saving nature. Conservation Biology, 2009. 23(4):785-787. Singh, R. K., Murty, H. R., Gupta, S. K., & Dikshit, A. K. An overview of sustainability assessment methodologies. Ecological Indicators,2009, volume 9, pages 189-212. Transportation Research Board, PROJECT C06(B): Development of an Ecological Assessment Process and Credits System for Enhancements to Highway Capacity, 2010, 88 pages Transportation Research Board, Integration of Conservation, PROJECT C06(A): Highway Planning, and Environmental Permitting Using an Outcome-Based Ecosystem Approach, 2011, 189 pages Woodward, Yong-Suhk Wui, The economic value of wetland services: a meta-analysis, Ecological Economics, 2001. Volume 37, Issue 2, May 2001, Pages 257-270