Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
77 The primary purpose of collecting and analyzing data on the benefits and cost of airport diver- sity programs is to understand the effectiveness of the programs and make informed decisions about how to increase workforce and contracting diversity. Airports also can make the most of these analyses by using them to communicate the airportâs diversity participation achievements, support requests for funding or other resources to make improvements, and share the airportâs successes with the community. A benefit-cost analysis requires significant time and resources, and putting the results to work for the airport will maximize the value of that investment. This chapter provides guidance on how to gather the information on benefits and costs developed under Chapters 3 and 6, respectively, evaluate the results of the analysis, and identify strategies for communicating this information to internal audiences (e.g., employees, executive staff, and other airport decision makers) and external audiences (e.g., the larger community). This section also demonstrates how the Benefit-Cost Tool can be used achieve the airportâs communication goals. Planning the Analysis When planning an analysis of benefits and costs, it is ideal to anticipate the desired outcomes before beginning data collection and analysis. This will help the airport collect the right data at the appropriate level of detail to conduct the desired analysis. Once an analysis is under way, it may be difficult or impossible to go back and fill in missing data that was not captured at the start of the analysis. The airport staff should begin this process by framing the analysis. This will include high-level decisions such as: ⢠The timeframe of the analysis; ⢠The scope of the analysis in terms of the types of costs and benefits that will be considered; ⢠Which specific programs will be examined (e.g., workforce diversity, DBE program, ACDBE program); ⢠The measures of value that will be used in the analysis based on which benefits and costs can be expressed in quantitative versus qualitative terms; and ⢠The desired results in terms of level of detail and metrics. These decisions will help the airport plan the data collection process in a way that will support its desired outcomes. They will also help the airport set realistic goals for the scope and detail of the analysis. When planning the analysis, the airport should involve staff who offer complemen- tary skills and perspectives to ensure that the analysis is robust and representative of the airportâs needs. Figure 9 illustrates the steps needed to complete the analysis. Collecting Data After the airport has planned the analysis and chosen the measures of value, it can begin collecting data. These data likely will be both quantitative and qualitative. Examples of C H A P T E R 7 Analysis and Communication Strategies for Airports
78 Guidance for Diversity in Airport Business Contracting and Workforce Programs quantitative data include monetary costs associated with diversity program implementation, diversity participation in different areas of the airport workforce, the number or value of contracts awarded to DBEs, and the number of individual DBEs contracting with the airport. Many of the quantitative data elements for the airportâs contracting program can be tracked in the Benefit-Cost Tool accompanying this guide. The Benefit-Cost Tool can accommodate information relating to the airportâs DBE, ACDBE, or local diversity programs. Examples of qualitative data include employee attitude provided in response to surveys and exit interviews and measures of goodwill such as awards, accolades, or improved commu- nity relations. While much of the information collected through surveys and interviews will be qualitative and subjective in nature (e.g., specific complaints an employee expresses about the workplace), these measures can often be designed or analyzed to produce quantitative but non-monetary results, as described in Chapter 6. For example, survey and interview results are often collected and summarized to communicate quantitative results, such as the number or percentage of employees who provided a specific response. To the extent possible, airports should develop quantitative metrics to facilitate the comparison of benefits and costs. Chapter 6 provides guidance on how to measure benefits. Interpreting Results Following the data collection phase, the airport is ready to develop metrics and interpret the results. As discussed above, the results of the analysis will most likely be expressed in monetary, quantitative, and/or qualitative metrics, depending on how the airport has designed its analysis Figure 9. Flowchart process of comparing benefits and costs.
Analysis and Communication Strategies for Airports 79 and the types of data it was able to collect. Developing these metrics is a key step in evaluating a diversity program as well as effectively communicating the results to stakeholders internal and external to the airport. Many quantitative and monetary metrics have been described in detail in FAA and ACRP reports, including ACRPâs recent publication Benefit-Cost Analyses Guidebook for Airport Stormwater (Krop et al. 2020). Some airports evaluating diversity programs may have fewer monetary metrics available for their analyses than traditional financial benefit-cost analyses because of the difficulty in monetizing the benefits of diversity. The sections below sum- marize some of the measures of value that may be most useful to an airport when evaluating a diversity program. The sections also identify to what uses and communication strategies these metrics are best suited. Quantitative and Monetary Metrics Once an airport has identified which benefits and costs can be expressed in monetary or quantitative terms, the airport is ready to develop quantitative metrics that are most effective in helping the airport evaluate its diversity program. Table 2 summarizes the types of metrics that can represent value in an analysis. Many of these metrics are described in Chapter 6, with the exception of cost-effectiveness, which is described in more detail below. The term cost-effectiveness can be used broadly to describe a number of metrics. Cost- effectiveness is intended to measure the cost of achieving specific outcomes. In the context of Economic Impact The direct, indirect, and induced economic effects of airport expenditures To estimate the distribution of economic benefits to diverse groups in an airportâs contracting or concessions program Represents how airport expenditures are distributed through the economy, but does not capture the ways the program may change the market through increased competition Cost-Effectiveness Cost per unit of measured outcomes To understand how efficiently the project achieves goals, targets, or priorities defined by the airport Does not communicate the overall effectiveness of the program in terms of meeting the goal or target Does not demonstrate the overall value of the project Diversity Participation A suite of metrics that represents the dollar value or percentage of contracts awarded to diverse businesses or the number or percentage of diverse employees working for the airport To understand the degree to which diverse individuals and businesses are employed by or contract with the airport Does not represent an actual value of measure but rather a proxy of value Metric Description Use Limitations Contractor Analysis Metrics such as the number of contractors working with the airport and the number of DBE contractors or ACDBE concessionaires working with the airport To understand the general pool of contractors, how it changes over time, and where the airport has gaps in diverse participation of contractors and concessionaires Is not an actual measure of value but rather a proxy of value Table 2. Summary of quantitative metrics to represent value.
80 Guidance for Diversity in Airport Business Contracting and Workforce Programs evaluating diversity programs, measures of cost-effectiveness can be a good way to determine how efficiently a project addresses a specific concern or outcome that is important to the airport. In this case, cost-effectiveness would be measured as the cost per unit of the outcome, as defined by the airport. For example, an airport implementing a contracting diversity program may want to understand how effectively the program: ⢠Attracts new DBE contractors to the airport (e.g., number of contractors per $100,000 of investment into the program); ⢠Results in economic benefits to diverse groups (e.g., number of jobs created for DBE firms per $100,000 of investment into the program); and ⢠Results in contracting expenditures to diverse groups (e.g., value of contracts awarded to diverse groups per dollar invested in the program). These metrics can help the airport understand or communicate how effective a project is at achieving a goal or target established by the airport. This metric has limitations and should not be used in isolation when evaluating project alternatives. For example, cost-effectiveness does not communicate the magnitude of the programâs benefits and offers no insight into the overall value of a diversity program. Cost-effectiveness measures should always be examined alongside the programâs overall ability to meet a target or goal. Airports must consider the value of their programs through other means when interpreting cost-effectiveness measures. Qualitative Measures Quantitative measures are typically the most influential when evaluating one or more projects because they are based on monetary benefits and costs, which are easy to understand and inter- pret and usually are highly important to the airport. However, diversity programs often result in moderate to substantial social benefits. Often, these benefits are qualitative and cannot easily be converted to dollars without cost- or time-prohibitive research and analysis. Despite this difficulty, major social benefits and costs cannot be ignored when trying to determine the outcomes of diversity programs. In these cases, the airport can use qualitative assessments to evaluate their programs. Results of a qualitative or non-monetary quantitative analysis can be used to supplement the results of the primary quantitative analysis. Qualitative measures can take the form of descriptive studies that describe the nuances of progress or improvements that cannot otherwise be communicated through numbers or graphics. Airports can also use scaled evaluations of outcomes that relate the airportâs progress against a goal or target. Scaled evaluations often use professional experience to estimate the degree to which the airport is achieving a desired outcome. For example, an airport may have identified five training classes that are effective at increasing skills of a diverse workforce. If the airport currently offers two of these classes, it would assign a â4â on a scale of 1 to 10. This scaled approach does not offer monetary values, but it provides a way for airports to measure and track improvement over time and can allow for comparisons to other airports. This approach is described in more detail in ACRPâs publication Benefit-Cost Analyses Guidebook for Airport Stormwater (Krop et al. 2020). Airports can also use visual methods to communicate results of qualitative measures, such as the scaled approach. The stoplight chart is a good example of how visuals can be effective in communicating progress. Table 3 provides an example of how a stoplight chart can be used to communicate progress on several different qualitative measures. In these examples, professional judgment may be required to evaluate how well the airport is performing on each measure of value or outcome. In the table, the results are demonstrated using a scale of negative (black), neutral (gray), or positive (white). This approach provides an intuitive visual representation of the airportâs progress. While the assessment may be highly subjective, the airport should ensure
Analysis and Communication Strategies for Airports 81 that factors influencing the outcome and any assumptions used in the evaluation are well docu- mented. This will help ensure that the outcomes are defendable and that assumptions can be easily updated as better information is available to the airport. A stoplight or similar approach will be most useful to an airport when: ⢠There are not enough data or information about the qualitative elements to support a more detailed analysis. ⢠The airport staff are conducting the analysis early in the BCA process, and potential projects are still being developed or selected. ⢠The qualitative elements in the analysis are of relatively low priority to the airport in the decision-making process, and a high-level assessment is sufficient. Communicating Results Identifying the Audience An effective and important strategy when building a case for diversity program efforts is tailoring communication strategies to the target audience. The airport should consider what type of information the intended audience will respond to best. Executive staff responsible for approving budgets may prefer simple, to-the-point tables or charts that help them understand the results of the analysis, especially as they pertain to their top priorities (e.g., whether or not the airport is meeting DBE targets, overall program costs over time, or employee satisfaction). In contrast, staff or managers heavily involved in diversity programs may prefer to view more detailed tables or charts to understand where there are gaps or shortfalls to evaluate where addi- tional efforts are needed. Other types of stakeholders, while not responsible for the implementation or approval of diversity program activities, may play a role in the airportâs ability to support diversity efforts. Ideally, the airport has already identified these stakeholders and addressed their concerns to the Measure of Value/ Outcome Year 1 Year 2 Year 3 Year 4 Year 5 Has the airport met its diversity training goals? Is the airport conducting the necessary outreach to attract new contractors? Has the airport fully implemented its contractor mentoring program? Is the airport meeting the diversity/inclusion goals identified in the airportâs strategic plan? What is the status of the airportâs relationship with the community as a result of diversity and inclusion efforts? Table 3. Stoplight evaluation for qualitative measures of success.
82 Guidance for Diversity in Airport Business Contracting and Workforce Programs extent possible in the development of the analysis. These stakeholders may include staff respon- sible for day-to-day management of airport activities that are affected by diversity programs, city or state staff responsible for ensuring that the airport meets local or regional diversity goals, or community members with an interest in diversity and inclusion. Building the Business Case Diversity program initiatives at airports may face critical review from within and outside the airport community. Results of the airportâs analysis evaluating their programs can be used stra- tegically to gain project approval for new initiatives, garner buy-in from airport staff involved in diversity efforts, and communicate the efforts to the larger community. By describing the net benefit (in both monetary and qualitative terms) of the program and its cost-effectiveness, the analysis can be used to build a business case for the airportâs diversity program. Below are common questions an airport may receive as it promotes or defends its program. Sample answers are provided, but each airport will need to tailor its response to its specific program and activities. Question 1: How do you measure the costs and benefits of your diversity programs? We engage in numerous initiatives in our procurement process to promote the award of contracts to disadvantaged, minority-owned, woman-owned, and small businesses. We monitor and measure the progress of participants through periodic reviews and regularly evaluate the progress of our programs using accumulated availability and utilization data. We conduct systematic evaluation of the costs and benefits of our diversity programs by itemizing the costs of developing specific business diversity initiatives and estimating the return on investment from implementation of these activities. Understanding these costs helps us further develop and/or modify our business diversity activities against our goals and objectives to create an environment for maximum participation by diverse businesses in our contracting opportunities. Question 2: What are the economic benefits of your airportâs workforce and business diversity programs? The operations of firms participating in business diversity programs generate economic activity across a spectrum of industries, including construction, professional services, food service, retail sales, and many other types of goods and services. The economic benefits of this activity associated with contracts and expenditures create personal income and jobs and may help reduce socio- economic gaps. Question 3: Where can I find information about the economic activity associated with contracts and expenditures arising from business diversity programs at your airport? Our website contains a wealth of information, including our strategic plan and goals for DBE and ACDBE participation in our contracting and concessions opportunities. Periodically we may also produce and post on our website a stand-alone report that provides in-depth informa- tion on the economic activity associated with our business diversity programs. Using the Airport Diversity Contracting Benefit-Cost Tool to Communicate Results The Benefit-Cost Tool has been designed to seamlessly translate information about airport contracts into transferable tables and graphics. These visual and numerical representations can be used by airports to communicate their progress toward their diversity goals to interested stakeholders. The Benefit-Cost Tool does this by aggregating user inputs about the types of businesses that participate on airport contracts along with information about the contracts themselves to create high-level and detailed summary tables. These summary tables are used to actively update a multitude of prefabricated graphs. After users have input all of the required information, they can export the summary charts or graphics to be used in external reports and communications materials.
Analysis and Communication Strategies for Airports 83 In addition to consolidating data in organized results tabs, the Benefit-Cost Tool allows users to analyze data in the inputs tabs by using the filter feature. Each inputs tab includes filters in each header which allow users to more closely examine subsets of their inputs. For instance, Figure 10 shows an example in which the user has filtered inputs by the company name and date. In this example, Concessionaire D as of 2019 was used as the filter criteria and as a result, all that companyâs contract history has been consolidated in the tab for ease of viewing. By doing this, the user can see that Concessionaire D as of 2019 has consistently contracted with the airport since 2019 and has won multiple concessions and contract types. The filter criteria can also be used on concession type, contract type, contract history, contract/opportunity, and all other headers in the inputâs tabs. These analyses will allow users to better understand contractor reoccurrence. The Benefit-Cost Tool interprets inputs in seven different results tabs that have a high-level and detailed numerical breakdown of the airportâs contract history. The results tabs are grouped by the type of result, which includes DBE participation; local business results, which include both local and small business participation on non-federally assisted contracts; ACDBE results, which include information about non-car rental and car rental concessions; and benefits, which includes a benefit-cost analysis of the airportâs diversity programs. The high-level summary on each page breaks down the airportâs progress toward its goals by year in terms of the types of contracts that are being analyzed (Figure 11). These tables can be copied and pasted from the Benefit-Cost Tool into a report. Figure 10. Example of using filter functionality on Inputs tab 6-Concessions Contracts tab. Figure 11. High-level summary of ACDBE car rental concessions results.
84 Guidance for Diversity in Airport Business Contracting and Workforce Programs Each results tab has a supplemental results graphics tab that produces graphs and charts that allow users to visualize the airportâs progress toward its goals. Each supplemental results graphics tab has between two and seven bar and line graphs that are set up to give users deeper insights into diverse business participation on airport contracts and concessions. For example, Figure 12 shows how these graphs can be used to look more closely at ACDBE participation by showing the percentage of participation that was achieved by concession type. In this example, ACDBE participation starts off evenly represented in each concession contract type; how- ever, the graph shows that over time, ACDBE participation begins to shift heavily toward food and beverage and retail concessions. In this example, these results might prompt airport diver- sity program managers to take a closer look at why they have achieved less ACDBE participation on Other Goods & Services contracts over time. By breaking the results into similar sub-analyses, the graphical representations tab empowers users to identify and communicate the nuanced trends in the aggregate data. The graphs are dynamic and do require the users to set them up. They are pre-populated based on data provided by the user in previous tabs. These graphs also can be copied and pasted into reports. Ultimately, the inputs tabs are rolled up into economic outputs tabs, which provide users with a high-level overview of the economic impacts (benefits) of their diversity programs. Like other results tabs, the R7-Benefits tab provides both high-level and detailed breakdowns of the data in the form of charts and is supplemented by a graphics tab (R7a-Benefits Graphics). These results are designed to communicate high-level concepts such as labor impacts to external stakeholders, in hopes that this will help them more broadly understand the impact of the airportâs diversity programs. For example, Figure 13, illustrates the economic impact of the airportâs diversity program by showing the number of times a contract is awarded to a diverse firm per $1M spent. This analysis enables users to gauge how effective their programs are at increasing the diverse business participation on contracts. Users can use these tabs to understand the cost-effectiveness of their programs and to communicate this to external stakeholders. Figure 12. Example of graphical representation of ACDBE non-car rental concessions results.
Analysis and Communication Strategies for Airports 85 # Times a Contract is Awarded to a Diverse Firm per $1 M Spent Figure 13. Example of graphical representation of economic benefits results.
