C

Prioritization Framework: Research and Rationale

Eleanore Douglas, Ph.D.

This Appendix provides an overview of the research and rationale that drives the development of a prioritization framework for infrastructure resilience projects in the Gulf of Mexico region, advanced through this workshop. It starts with the literature review overview. Then it introduces three basic approaches to framing and orienting a framework for this purpose of project or investment prioritization, three options for supporting project prioritization, the six-step process proposed for this framework, and a closer look at Step 3 of the six-step process. Step 3 was the focus of the workshop because it was best positioned to offer the most value to other decision-making agencies, as this is an immediate and concrete challenge that many agencies are likely to face in the short term. The appendix concludes with potential next steps to continue building out this prioritization framework.

First, workshop designers surveyed general approaches to, and mechanisms that support, making investment decisions specifically in the public policy space. This literature review included previous National Academies of Sciences, Engineering, and Medicine studies and a number of formalized frameworks for resilient infrastructure decision-making published by other institutions over the past decade (see Box C-1).

The literature review revealed three general approaches to framing and orienting the authority and legitimacy of such decision-making: (1) top-down, (2) bottom-up, and (3) technocratic. Top-down approaches tend to align the legitimacy of resource decisions with strategic vision and values, planning and operational guidance, or other national or executive-level policy. They analyze proposed projects or investments in light of their explicit alignment with and support for these guidance and policy documents. The bottom-up approach draws its authority and legitimacy from the populations affected by the resource decisions being made. As a result, this approach tends to focus on representative participation or sampling of the relevant populations, as well as the accurate elicitation and reflection of the populations’ preferences to help shape decisions on investments. The technocratic approach tends to source the authority and legitimacy for its decision-making in explicit scientific, expert, and technocratic analysis and formalized understanding of the space in which investment decisions are being considered.

While there are currently government entities and other organizations that adhere to each of these approaches (e.g., Department of Defense as “top-down,” local school-boards and city councils as “bottom-up,” and the Environmental Protection Agency as “technocratic”), most public policy decision-making constructs tend to reflect a mixture of all three approaches. Nevertheless, decision makers must understand the different sources of authority and legitimacy underpinning their decisions, in order to convey their decisions effectively across a population, whose perceptions of authority and legitimacy may differ significantly.

Several mechanisms exist for framing and providing analytical support to public investment decision-making that blend the three approaches. Given the mission of the National Academies and the Gulf Research Program (GRP), and the focus of this effort being infrastructure resilience during disasters, three options for supporting project prioritization emerged as most viable—Scenario-based Systems Analysis Construct, Modular Table-Top Game for Local Engagement, and Risk-based Portfolio Management Approach. Portfolio management was best suited to the task because of its flexibility on specificity and ability to address trade-offs, including across sectors. The other two work best with a narrow focus and narrow criteria, or are difficult to scale up to a regional level. As a result, we decided to use the risk-based portfolio management approach, drawing on elements of the others, using scenarios and serious games in support of that approach.

RISK-BASED PORTFOLIO MANAGEMENT APPROACH

The main objective of a portfolio management approach is to make the best trade-off of risk against return across a number of different investments or assets within the context of a

particular goal or criteria. Portfolio management tools and methods are powerful due to their explicit engagement with both risk and uncertainty and their inherent flexibility with respect to almost every other aspect of the analytical context. Portfolio approaches are neutral with respect to the organizational context and the particular selection of governing criteria. This has made it a useful framework for supporting investment decision-making across a broad range of different organizations and criteria, from private-sector financial investment management to the National Institutes of Health priority setting and grant management to the Department of Homeland Security R&D/Science and Technology (S&T) portfolios. A weakness of portfolio management, particularly in the context of infrastructure investment decision-making, is the tendency of traditional portfolio analysis tools to treat each project or proposed investment as a fully independent item, balancing them collectively only across the whole group. It takes additional layers of analysis to acknowledge and/or track interdependencies between particular projects or to evaluate sub-clusters of projects within a larger portfolio. Still, the risk-based Portfolio Management Approach lends itself most readily to the objectives of this National Academies effort and therefore serves as the general driver for the process developed and implemented during the workshop.

ELICITING STAKEHOLDER INPUTS

Considering existing frameworks for portfolio management in a public policy and infrastructure context and then adapting them to support the prioritization of projects likely to increase infrastructure resilience in the Gulf region, workshop designers proposed a six-step process for a useful framework (see Figure C-1).

While the six steps provide a comprehensive approach to risk-based portfolio management, the National Academies determined that, to use the limited time of participants

most effectively and develop actionable, practical outputs, this workshop would focus on refining, validating, and potentially ranking the criteria by which infrastructure project proposals can be effectively analyzed, or Step 3 of the process.

The literature review also revealed a number of key criteria useful in analyzing and evaluating the impacts of particular actions, projects, and investments with respect to resilience. Workshop designers leveraged the resilience and investment frameworks referenced above as well as current Federal Emergency Management Agency (FEMA), Environmental Protection Agency (EPA), and U.S. Army Corps of Engineers (USACE) grant evaluation guidelines, to capture a variety of potentially salient criteria. First, workshop designers grouped these criteria into five key macro-criteria categories: Environment, Economy, Society, Resilience, and Project Governance (see Figure C-2).

The five macro-criteria served to help structure the literature- and participant-identified sub-criteria and prevent information overload during the workshop. Second, workshop designers reviewed the key criteria suggested by the literature and identified the subsets that would likely be relevant for this type of prioritization framework. Workshop facilitators reviewed this information to help drive discussion when ideas were not organically volunteered from workshop participants and for comparison purposes after initial brainstorming. The approximately six sub-criteria per macro-criterion that were common across all or most domains in the final set of master criteria developed during this workshop were also largely identified in this initial research. Several of the domain-specific sub-criteria suggested by participants were also identified in the initial research. While almost all of these sub-criteria emerged through small group discussion naturally and very limited leading on the part of the facilitators was necessary, there was still a great deal of consistency between sub-criteria suggested by participants and by the literature.

The National Academies and GRP knew that the workshop would not authoritatively validate a prioritized short list of sub-criteria; however, they did expect that useful and meaningful discussions would occur and some preliminary insights would be drawn, which was the case.

Building out the Framework Further

The macro-criteria and sub-criteria provide useful inputs into how Step 3 (and 4) of this portfolio management approach to a prioritization framework might be implemented. Step 3 focuses on evaluation of individual infrastructure project proposals and was the focus of this workshop. Step 4 focuses on evaluating infrastructure proposals as a group or a portfolio. Useful next steps apply to each of these steps.

Step 3

Lists of evaluation sub-criteria can be quite extensive. In fact, the set of criteria that was developed through this workshop includes between 12 and 14 sub-criteria for each macro-criterion. While it is important to be comprehensive in the consideration of each macro-category, for realistic implementation it is also vital to think about the two or three most indicative sub-criteria for each category. Further refinement is an appropriate next step.

As described in other sections of this proceedings, some participants felt that a couple of the criteria needed to permeate the entire decision-space and process more thoroughly than in just a single macro-criterion category. Examples include the macro-criterion of Resilience or the sub-criteria of Equity or Equity and Inclusion. Another next step would be to further explore if these criteria are useful at the project level, should be incorporated into portfolio-level considerations, or should be applied to both, and how.

Step 4

Another useful action to build out the framework further is to focus on the particular requirements of the portfolio evaluation step and explore what criteria are most important to balancing a portfolio of infrastructure investments, as opposed to making funding decisions at the individual project level. This will require a more explicit definition of infrastructure risk and what it might mean to buy down infrastructure risk. It would be useful for stakeholders to consider the most important sources of uncertainty in infrastructure investments. Important insights could be drawn from how existing federal and private-sector portfolio managers build and adapt their portfolios. Finally, the portfolio evaluation step is also the one at which systems-level analytical approaches might effectively be applied.

This page intentionally left blank.