ACRP Product Update: Prioritization of Policy and Planning Products (2023)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2 Chapter 1 – Review and Prioritization Compile Products Spreadsheet & Screen Products for Update Products were organized by project number, product number, and title in a spreadsheet for data collection, product screening, and scoring. Data was collected in October and November 2022. Analysis is based on these dates. There were 65 completed projects (research projects and synthesis studies) in the research field as of this date that had produced 81 separate products. Analysis of Product Downloads The number of downloads for each research product shown in column P of the Product Update Screening Spreadsheet was obtained from the National Academies Publications (NAP) website page for that product. The cumulative distribution of the number of downloads for the research project products (Research Reports and Web-Only Documents) and Synthesis studies are shown in Figure 1. No download data was obtained for ACRP WebResources, since they are not unique documents that can be downloaded in the same way. The figure shows the percentage of the Policy and Planning products considered in the study (except for WebResources) that had the number of downloads shown on the x- axis or fewer. It answers the question “what percentage of the products had no more than a given number of downloads?” Figure 1. Cumulative Distribution of Product Downloads The number of reported downloads for a given product varies widely, as could be expected. The product with the most reported downloads (ACRP Research Report 143: Guidebook for Air Cargo Facility Planning and Development) had almost 60% more downloads than the second most downloaded

3 product (ACRP Research Report 150 NextGen for Airports, Volume 5: Airport Planning and Development). The synthesis study reports were generally downloaded somewhat less often than the research project products, but as can be seen from Figure 1, the number of downloads for a given synthesis varies widely. The synthesis with the most reported downloads (ACRP Synthesis 84: Transportation Network Companies: Challenges and Opportunities for Airport Operators) had over 50% more reported downloads than the second most downloaded synthesis (ACRP Synthesis 83: Preparing Airports for Communicable Diseases on Arriving Flights). However, there are two issues that should be borne in mind in interpreting the product download statistics. The first is that the NAP download statistics do not always extend back to the original publication date of the older reports, as many early ACRP reports weren’t posted to the NAP site until after 2012. The second issue to be taken into account in interpreting the product download statistics is that on average older products will have been downloaded more times than newer products, simply because there has been more time for interested users to download the older products. The profile over time of downloads for a given product is displayed on the detailed NAP website page for that product. Figure 2 shows the download profile for ACRP Research Report 23: Airport Passenger-Related Processing Rates Guidebook. Figure 2. Download Profile for ACRP Report 23 Source: https://nap.nationalacademies.org/catalog/22990/airport-passenger-related-processing- rates-guidebook. Downloaded March 20, 2023.

4 It can be seen that the rate of downloads has continued fairly consistently over time, with a somewhat higher rate in 2020 and 2021, returning to the earlier rate in 2022. Figure 2 also shows the missing download data prior to 2016. In order to show how the number of product downloads may have been influenced by the time since the product was first published, Figure 3 shows the number of downloads for each product by the number of years since the product was first published. Figure 3. Product Downloads vs. Years Since Publication Source: NAP website, ACRP product downloads as of 10/18/22. (See Appendix A, Product Screening worksheet, Column P.) It should be noted that products with more than 7 years since publication would have been published prior to 2016 and therefore are most likely missing download data. The amount of missing data would increase with the years since publication. Therefore, there are two different regimes on the figure. The data for products with 6 or fewer years since publication may be more complete and appear to show an increasing trend in the number of downloads with years since publication. The data for products with more than 7 years since publication most likely understate the number of downloads, with the extent of the missing data increasing with the years since publication. However, accounting for this understatement, this too appears to show an increasing trend with the years since publication. Whether the number of downloads for products with 7 years since publication is missing any data and therefore lies in one or other regime depends on both when the product was published in the year and when the NAP began recording the number of downloads, and would therefore require further analysis to resolve.

5 As can also be clearly seen from Figure 3, the number of downloads for products with the same number of years since publication varies widely across the different products, as would be expected, although this variation is much less for the Synthesis studies than for the research project products. Decision Tree Implementation To screen each product for consideration for updating according to the decision tree defined in the ACRP Review Process on assessing the need for updating research field products, the research team collected data on each product. Descriptions of the data collected and recorded for each product are included in Appendix B. The screening spreadsheet was presented to the panel at a web meeting on December 1, 2022. Following the panel meeting the research team compiled additional data from relevant industry newsletters to further document if the topic is considered a hot topic. The decision tree was applied to each product according to the logic in the Review Process as shown in Figure 4. Results were noted in the spreadsheet. Applying the decision tree logic resulted in only 11 projects being recommended for consideration for update, comprising 14 unique products, with the remaining 67 products (from 54 projects) not being recommended for an update. Figure 4. Decision Tree from Review Process

6 Obtain Stakeholder Input The process of obtaining industry stakeholder input on the need for ACRP product updates was undertaken using the following steps: 1. A list of potential members of an ISRG was developed and reviewed with the project panel. The list included representatives of all large hub and medium hub airports, a selection of small hub, non-hub, and general aviation airports, airport planning consultants, and state and regional aviation agencies. In addition, a number of industry associations, professional society committees, and TRB Aviation Group committees were contacted and asked to share the invitation to participate in the ISRG with their members. In total, 136 individual representatives from airports and other organizations were contacted, as well as nine industry associations, and all TRB Aviation Group committees. In total 25 large hub airports, 33 medium hub airports, seven small or non-hub airports, and 10 general aviation airports were contacted. In some instances, multiple staff were contacted at an airport. Figure 5 shows the breakdown by type of organization for each individual representative invited to participate by the research team. The list of ISRG members were shared with the panel on December 1, 2022. Figure 5. Potential ISRG Participants by Category Source: Airport Cooperative Research Program’s Research Field Product Update Review Process 2. Each potential member in the ISRG was contacted by e-mail or telephone, or both, and asked if they would be willing to participate in the review. Sixty-one respondents agreed to participate, in many cases after several follow-up contacts. Those who agreed were then sent a list of the ACRP Policy and Planning products (see Appendix C), grouped into nine categories. These categories were chosen to help respondents identify products that they were familiar with. Since there were 80 products listed in total (Synthesis 03-06 was inadvertently omitted and 0 5 10 15 20 25 30 35 40 University Staff Regional Transportation Agencies State Aviation Departments Federal Government Consultants General Aviation Airport Small and Non Hub Airports Medium Hub Airports Large Hub Airports Individuals St ak eh ol de r T yp e Potential ISRG Participants by Category

7 subsequently assessed by members of the research team), it was felt that finding a particular product in a single list could lead to products being overlooked. The categories are as follows: • Air service development • Airport capacity and delay • Airport planning • Airport system performance • Capital investment planning • Demand analysis and forecasting • Emerging technologies • Small airports and airport preservation • Terminal planning and air cargo Respondents were asked to identify the products they were familiar by checking a box in the spreadsheet and return the spreadsheet or PDF by e-mail. 3. By January 17, 2023 fifty-two responses had been received, resulting in more than 85 percent participation. Each respondent was assigned to evaluate no more than five of the products they had indicated familiarity with to keep the workload involved to a reasonable level. The products assigned to each respondent were chosen so as to evaluate as many different ACRP products as possible. 4. An electronic form was prepared on which the ISRG respondents could provide input on the products they had been assigned. The form was intended to avoid the burden on the ISRG member of downloading, editing, and attaching a file to an e-mail and facilitated compiling the responses. This form had a page for each assigned product that asked: • Select the product being assessed using a pull-down menu • Whether the respondent felt that the product needed to be updated • If so, what aspects needed to be updated (free-form comments) • If so, how urgent or critical the need for an update is (three options: very critical, moderately critical, not particularly critical) 5. The responses were compiled in an Excel spreadsheet. 6. The team reviewed the ISRG comments. Twelve products received feedback that the ISRG did not recommend the product be updated. Others received some feedback that recommended an update and some that did not. This resulted in the research team using judgment based on the feedback received and personal expertise to determine if the product should be recommended for update. Scoring ACRP Products for Update The scoring process followed the table provided in the Review Process methodology. See Table 1 below to see each ranking question. Each ranking question was equally weighted with a maximum of 17 points.

8 Data collected in the initial screening may have been adjusted based on ISRG comments and the research teams’ expertise. The team also evaluated which portion of each product need updating. A standardized average hourly rate of $195 was applied to the estimated level of effort required for updating. One of two different levels of effort – either a relatively minor update or an update requiring significantly more work –was assigned for six different aspects of a product that would need to be updated. The assumptions for the hours required for each of these generic product update tasks and the associated costs are documented in Appendix A tab Level of Effort and Cost Estimate. The cost estimates were then used to answer ranking Question 7 with a 0, 1 ($100k or greater), 2 ($50k or more but less than $100k), or 3 (less than $50k). If an ACRP research project was underway and not yet completed as of October 2022 a score was not generated. If a product was not recommended for update the score was set to zero. Table 1. Prioritization Scoring Methodology Ranking Question 3 Points 2 Points 1 Point 0 Points 1 Does the topic contribute to the Sponsored Research Suggestions in the most recent Policy and Planning Research Roadmap? — — Yes No 2 How integral is the obsolete or missing content to the integrity/usefulness of the product? Very integral / important Moderately important Marginally important Not important 3 Does this research topic relate to subjects in ACRP Problem Statements or upcoming ACRP Projects? — — Yes No 4 Is there interest in the original publication (demonstrated by number of downloads compared to other products in the same research field)? In the top third of downloads In the middle tercile of downloads In the bottom tercile of downloads No downloads 5 How urgent is the need for an updated product to the industry (i.e., is this a hot topic? Is information needed for compliance with new regulations, to respond to a crisis, etc.)? Updated product addresses an urgent need for industry Updated product addresses a moderately important need Updated product addresses a marginally important need No urgent need exists 6 How broadly applicable will the updated product be (compared to the other products recommended for update)? Applicable to a significant proportion of airports Applicable to a moderate number / type of airports Narrowly applicable — 7 How complex or costly are the required updates? Relatively limited / targeted updates to the product / tools required Moderate updates to the product / tools required Extensive updates to the product / tools required —

9 Prioritizing and Finalizing ACRP Products for Update The research team reviewed the results of the scoring and prioritization process. Scores varied from one to 14. Products not recommended for update received a score of zero. Given the available funds for priority product updates, the research team chose products above a score of 13 as the recommended threshold. There were eight projects, which included 12 products as shown below by ranking order, that scored 13 or higher for an estimated total update cost of $530,400. • ACRP Synthesis 2 Airport Aviation Activity Forecasting • ACRP Report 104 Defining and Measuring Aircraft Delay and Airport Capacity Thresholds • ACRP Research Report 161 Improving Airport Services for International Customers • ACRP Report 142 Effects of the Airlines Industry Changes on Small- and Non-Hub Airports • ACRP Report 132 The Role of U.S. Airports in the National Economy • ACRP Research Report 212: Airports and Unmanned Aircraft Systems, Vols 1- 3 and ACRP Web- Only Document 42 Toolkits and Resource Library for Airports and Unmanned Aircraft Systems • ACRP Synthesis 5 Airport Access Mode Choice Models • ACRP Report 150 NextGen for Airports, Volume 5: Airport Planning and Development Of these projects, the decision tree only identified ACRP Report 142 and ACRP Research Report 161, to be considered for update. The total estimated cost for updating just these two products is $152,100. Project Panel Meeting Summary An in-person 2-day project panel meeting was held March 8-9, 2023 to review the results of the prioritization. On the first day of the meeting Matt Griffin, the Project Officer, reviewed the goals and objectives of the project. Leah Whitfield reviewed the research team’s steps to prioritize products and walked the panel through the recommended products for update. The panel and research team held a discussion regarding the steps of the methodology, the recommended products for updates, and recommendations for changes to the Update Review Process. A number of recommendations were accepted by the panel and passed along to ACRP staff. Most importantly, the panel felt that the seven ranking questions used to prioritize the products for update were not equally important and should be weighted to reflect their relative importance. Since it had been agreed that the presence of the topic in the ACRP Research Roadmap should be removed from the scoring, this was given a weight of zero. After some discussion the panel agreed that the ranking questions should be weighted as shown in the following table: Table 2. Weighting of Ranking Questions Ranking Question Weight 1. Presence in the ACRP Research Roadmap 0 2. How integral is the obsolete or missing content? 20 3. Has a research idea to update this product been submitted? 10 4. Is there interest in the original publication? 10 5. How urgent is the need for an updated product? 30 6. How broadly applicable will the updated product be? 20 7. How complex or costly are the required updates? 10

10 The prioritization criteria weights were added in a row above the column headings in the “Product Update Screening Spreadsheet” (Appendix A) and the prioritization score formula for each product was calculated using these weights. This allowed the panel to change the weights and see the effect on the relative prioritization scores for each product. Resulting scores ranged from three to 26. The results did not significantly shift the top scoring products. The spreadsheet was shared with the panel for their discussions on the second day of the meeting, which the research team did not attend, so they could make any necessary revisions. The following table shows the top ranked products, which scored 23 or greater, proposed for updates as a result of the analysis and discussion during the project panel meeting during day one. Table 3. Top Ranked Products Proposed for Updates after Weighting Product Title Ranking ACRP Synthesis 2: Airport Aviation Activity Forecasting 26 ACRP Report 142: Effects of Airline Industry Changes on Small- and Non-Hub Airports 25 ACRP Report 150: NextGen for Airports, Volume 5: Airport Planning and Development 25 ACRP Report 27: Enhancing Airport Land Use Compatibility, Volume 1: Land Use Fundamentals and Implementation Resources 24 ACRP Report 38: Understanding Airspace, Objects, and Their Effects on Airports 24 ACRP Report 104: Defining and Measuring Aircraft Delay and Airport Capacity Thresholds 24 ACRP Research Results Digest 10: Light Detection and Ranging (LIDAR) Deployment for Airport Obstruction Surveys 23 ACRP Report 132: The Role of U.S. Airports in the National Economy 23 ACRP Synthesis 5: Airport Ground Access Mode Choice Models 23