Conducting Airport Peer Reviews (2013)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

5 This chapter provides an overview of peer review formats that surfaced during the literature review and interviews. The primary focus is on the range of peer review approaches being used by airports, but examples from other industries are included to demonstrate the history and depth of the process and to provide additional options for airport managers. The seven peer review formats described in this chapter are: • Conference-style exchanges • Host-focused reviews • Performance audits • Executive peer advisory teams • Site visits • Technical assistances • Online forums. This wide range of formats illustrates the flexibility with which airport managers initiate peer reviews and their will- ingness to embrace peer feedback in any form. These formats are fluid, not rigid, and can be customized and combined to meet an airport’s needs. CONFERENCE-STYLE EXCHANGES Examples Within the Airport Industry The conference-style exchange in the airport industry dates from the mid-1990s, when two experienced airport profes- sionals, William Fife (then the corporate vice-president of DMJM Harris–AECOM) and Donna Luh (an airport manager for the Niagara Frontier Transportation Authority), established a peer review group to help airports share information related to specific projects and issues (Schurr 2003). This group pro- vided structure to the professional connections already in place within the airport industry. Airport staff involved in the peer review group can request informal and formal feedback from their peers on a range of issues through a simple e-mail query or through organized conferences. The conference-style exchanges developed by Fife and Luh for this group provide a formal mechanism for discussing common challenges. According to interviewees, the conference-style exchange is typically a one-day event involving a host agency and 25 to 40 voluntary participants (airport staff and consultants). The host agency assembles the group to address a challenge it faces in regard to a project or program in development. Attendees discuss the host airport’s issues during the morning portion of the agenda, and then have the opportunity to raise challenges of their own during the afternoon discussions (Infanger 2003). Topics range from big-picture progress on major development projects to narrowly-focused subcompo- nents of a specific effort such as baggage or security. As noted by the interviewees, these peer reviews are typi- cally open to airport professionals (including consultants) who want to share their experiences and learn from others in the industry. The host airport may pay for lunch for the attendees, but the participants generally cover their own expenses— because, as several interviewees noted, everyone benefits from the discussions and the professional connections made, so it is a win for the host airport and a win for the participants. Although feedback concerning the conference-style exchange is overwhelmingly positive, a few cautionary notes about this format emerged during the interviews. First, per- spective matters. Each airport is unique, and the distinctions are accentuated when airports of different sizes and structures are compared. Although airports do face many of the same challenges, how an airport best addresses those challenges is affected by the specific characteristics of its environment. Not all ideas shared in a large forum, therefore, will resonate with every participant. Second, conference-style exchanges may be more useful to very senior staff than to lower-level managers. The discussions tend to be more strategic rather than practical because there isn’t time to go into details. Third, some interviewees warned that the size of the group can be overwhelming, and make it difficult to ensure that everyone has an opportunity to con- tribute. Similarly, a consensus response to a proposed idea, which is sometimes the host airport’s intent, can be difficult to reach with a group of this size. One interviewee noted that such conferences used to take place on a regular basis, perhaps several times a year. Although these reviews appear to occur less frequently, some agencies do still initiate them. For example, one respondent was planning to attend a conference-style exchange organized by the Port Authority of New York and New Jersey in August 2012. The goal was to bring together a group of 40 to 50 airport staff and consultants from around the country to identify issues related to planning and designing terminals of the future and to discuss those issues in breakout groups. Event organizers targeted participants based on their experience and expertise, rather than chapter two CURRENT PRACTICES IN CONDUCTING PEER REVIEWS

6 issuing an open invitation, but participants were still expected to cover their own travel expenses. It is important to note that industry organizations also pro- vide active forums for peer interactions through topical con- ferences, seminars, and workshops. For example, ACI-NA, NASAO, FAA Airports Division, Airport Consultants Coun- cil, and others sponsor events about the state of industry practices or workshops in specific subject areas such as pol- icy development, project delivery, security, wildlife hazards, financial management, and environmental impacts, among others. Participants from across the country pay to attend face-to-face conferences, which provide valuable opportuni- ties for networking and learning. Examples Outside the Airport Industry The conference-style format is common outside the airport industry as well. In the late 1990s, William Fife worked with Jerry Premo, who was also at DMJM Harris (now AECOM), to apply this format to the area of public transportation (transit). Like the reviews hosted in the airport industry, these one- or two-day events for transit professionals explore a specific proposed project or program of the host agency, followed by a roundtable discussion of other pressing transit issues. Topics have included security, clean-fuel options, drug and alcohol testing, environmental remediation, project management, and community support. In at least some cases, participants submit the topics they would like to discuss in advance (Schurr 2003). Several other versions of conference-style exchanges within the highway industry emerged in the literature review. In 2004, NCHRP funded a project called Peer Exchange of State and Metropolitan Planning Issues (Peer Review Capacity Building Partnership . . . 2012). Under its umbrella, several peer exchanges focused on different transportation planning issues, such as reliability measures and innovative transporta- tion planning (Hoffman et al. 2006). The peer exchange plan- ners invited representatives from state DOTs, metropolitan planning organizations, and public transportation agencies from around the country to discuss common challenges and innovative approaches related to these topics for the benefit of the broader transportation planning community. Each event lasted one or two days and involved fewer participants than the open-invitation conference-style reviews described previously. However, they shared the goal of mutual exchange and ben- efit, with no one organization dictating the entire agenda. To help focus the discussions at these peer reviews, NCHRP project planners asked the participants beforehand to answer questions about their agencies’ experiences in the selected topic area, such as their use of innovative financing tools or their implementation of disaster response strategies. The responses helped to provide a foundation for the discussions (Hoffman et al. 2006). The contractors for the project produced separate reports for each event to help distribute the results. Also in the highway sector, a number of research-oriented organizations partnered to organize a biannual conference on winter highway maintenance. In 2007, the first National Winter Maintenance Peer Exchange was a 1½-day event involving 86 attendees from 35 states (Scott 2008). Representa- tives from state, local, and federal transportation agencies and research organizations gathered to brainstorm on research needs, meet with vendors, and discuss common challenges. Event organizers sent out a survey to participants in advance ask- ing them to identify their “most critical winter maintenance challenges” (Scott 2008, p. 3). The responses were used to organize roundtable discussions at the conference and were incorporated in the final report. Each sponsor agreed to pur- sue action items following the conference (such as funding a high-priority research project) and to report progress at the next peer exchange and on a designated website (http://www. westerntransportationinstitute.org/professionaldevelopment/ national-winter-maintenance-peer-exchange). The sponsors of the National Winter Maintenance Peer Exchange handled the planning and logistics for the event. Sponsors included FHWA, AASHTO, Montana State Uni- versity, and two national research programs focused on winter maintenance, Clear Roads and Aurora. Both Clear Roads and Aurora are multi-state consortiums funded by state DOT con- tributions through the FHWA Transportation Pooled Fund Program (see www.pooledfund.org). Participants in the peer exchange either paid their own travel costs or were sponsored through their agency’s contributions to Clear Roads or Aurora. The event was so successful that it has become a biennial event, held again in 2009 and 2011 and planned for 2013. HOST-FOCUSED REVIEWS Examples Within the Airport Industry The peer review format most frequently described in inter- views for this report is a host-focused review. It is similar to a conference-style exchange in that it brings together peers from multiple organizations to share experiences and offer feedback. However, this type of review is smaller in size (15 to 25 participants); focuses primarily, if not entirely, on the host agency’s needs and topics; and typically has a more structured agenda. According to interviewees, the host airport brings in a panel of three to seven outside experts to meet with selected staff, either onsite at the airport or in a nearby hotel. The host airport covers all travel costs, lodging, and meals for the visiting experts, but the visitors donate their time. These exchanges usually last two to three days to allow for more in-depth discussion of the topics, though this is not always the case. Airport managers choose to host this type of peer review to get feedback from experts in the industry about alternative approaches they have developed for a project, gain validation for a proposed approach, brainstorm solutions to a challenge, and/or learn lessons from others who have dealt with very

7 similar situations. According to interviewees, airport managers have used these host-focused reviews to tackle a wide range of topics, including the overall master planning process for a terminal renovation or runway reconstruction, people-movers, concessions, environmental impacts, art in public places, security systems, or contract negotiations. Some reviews are organizationally focused (e.g., how to structure an informa- tion technology department), and others are more functionally focused (how to best design and locate a baggage system). Some interviewees had hosted multiple reviews of this type during the life of a large project. The makeup of the participants for a host-focused review is important to ensure a successful experience. Many inter- viewees emphasized the need to find individuals who are well- respected within the industry, have demonstrated success in handling similar challenges, and who come from similarly structured and sized organizations to increase the likelihood that the comments are applicable to the host airport’s situation. Some airport managers rely on their own professional con- nections to assemble these expert panels, while others enlist the help of outside consultants or the consultants already under contract for design or construction work. Several inter- viewees said that the idea to hold a peer review came from their master planning or design consultants as a way of get- ting outside confirmation or constructive feedback. In such cases, funding for the review can be built into the consultant contract as part of the project budget. Other interviewees indicated they had received support and encouragement to perform a review and recruit experts from staff in professional organizations and consulting firms. Examples Outside the Airport Industry State DOTs have also made extensive use of host-focused reviews. In 1994, FHWA mandated that research, develop- ment, and technology transfer (RD&T) programs in state DOTs hold periodic peer reviews aimed at improving prac- tices (Battey n.d.). The meetings are intended to help managers evaluate and improve their programs’ effectiveness by learning from the experiences of others. States can use federal funding from their programs to cover 100% of travel and other costs for the exchanges (Moulden 2010), whereas typical RD&T activi- ties require a 20% contribution of state funds. As mentioned earlier, FHWA moved away from the term “peer review” and began using “peer exchange” to further emphasize the collab- orative nature of these meetings (Harder 2001). Like the host-focused reviews in the airport industry, these FHWA-mandated peer exchanges are initiated by a single research program, use a structured agenda focused on the issues defined by the host agency, last two to three days, and include four to seven peer panelists from other organizations (Moulden 2010). The agendas may be wide-ranging, focusing on the host’s program management plan; or may address spe- cific aspects of research management, such as implementing research results, developing partnerships, or managing con- tracts (AASHTO 2012). In all cases, the expectation is that participants will engage in candid, detailed discussions that will support the development of planned actions for program modifications. Research program directors have found the peer exchanges to be “very valuable,” resulting in “improvements to the quality of their programs” (Battey n.d.). In 2004, the Wisconsin DOT (WisDOT) Research Pro- gram began offering logistical and financial support for peer exchanges to other offices within the department, using the same model required by FHWA for the research programs. Funded as technology transfer projects, these peer exchanges offered a quick-turnaround method for answering questions and solving problems—an alternative to the traditional multi- year research project (Peer Exchanges . . . 2010). Between 2004 and 2011, the WisDOT program funded and supported nine peer exchanges on a wide range of topics for the depart- ment, including real estate issues, privacy policies, social media, high-speed rail, and vehicle titling issues. PERFORMANCE AUDITS Examples Within the Airport Industry Although both the conference-style exchange and host- focused review provide a mechanism for peers to review a host’s practices (or proposed actions) and provide feedback, the performance audit format is more explicitly an evaluation defining and prompting specific improvements. A number of industry organizations and associations offer audit services to airports on a range of topics, such as safety (including runway safety, wildlife hazard management, and winter operations), business processes, risk assessment, and customer service. Some airports look directly to their peers for assistance with auditing their processes or procedures. For example, several individuals interviewed for this report explained that they have contacted representatives from an airport in another city or representatives from a nearby airport managed by the same regional authority to help prepare for FAR Part 139 inspec- tions. These “pre-inspections” give the airports an opportu- nity to have outside reviewers cast a fresh eye on operations, so the airport can then make any changes recommended by the peer reviewers before the official inspection takes place. Examples Outside the Airport Industry Water and wastewater utilities have found a more formal peer audit format called QualServe to be helpful. Provided by the American Water Works Association and the Water Environment Foundation for the last ten years, the program helps utilities “improve performance and increase customer satisfaction” (QualServe Fact Sheet 2011, p. 1). The reviews involve a guided self-assessment, on-site peer assessment, and report with recommendations. QualServe digs deep into the organization’s inner workings across business areas. According to John Anderson, project manager of utility

8 quality programs at the American Water Works Association, the self-assessment survey goes to more than half of the host utility’s employees and contains more than 400 questions. The results are compiled and analyzed to identify problem areas that warrant attention during the on-site review. The peer panels assembled for the QualServe reviews typically comprise four high-level, experienced managers from other utilities who have completed a week of training for the on-site assessment. They are selected for their specific business expertise and volunteer their time for both the training and the review. Participation in the program demonstrates a strong commitment within the utility industry to supporting peer professionals, much like the commitment to peer support demonstrated in the airport industry. The review team spends a week on-site with the host agency, interviewing hundreds of staff through one-on-one meetings and focus groups. At the end of the week, the leader of the peer team discusses the findings with the general manager and executive staff, and then presents the results to utility staff broadly in an auditorium using PowerPoint slides. The QualServe reviews help utilities identify morale prob- lems and streamline operations through an independent eval- uation. The utilities are encouraged to track their progress in addressing problem areas after the review, and to compare themselves with other utilities through an annual survey and through regional benchmarking workshops. A QualServe peer review costs $29,000 to $65,000 (2012), depending on the number of peer experts assembled (three to five) and the number of days on-site (four or five). The cost covers travel expenses for the peer team; processing of the self-assessment survey; and the experts’ time in training the volunteers, pre- paring for the review, and facilitating the review (QualServe Fact Sheet 2011). EXECUTIVE PEER ADVISORY TEAMS Several interviewees for this report shared their experiences with using an executive peer advisory team when carrying out large airport construction projects. The goal is to obtain feedback and advice from a small group of peers on a regular basis throughout the life of the project. Instead of attend- ing a single event, participants were called upon to attend in-person meetings or consult by telephone multiple times over several years. In one instance, an interviewee had personally assembled an advisory team for a large project. The team included a retired airport director with experience on mega projects (very large, multi-year construction projects), a consultant with experience in airline relations, a county transportation department head with expertise in roadway construction, and an airline representative. This team was integrally involved throughout the programming, design, and construction phases of the project, periodically attending meetings with the air- port staff and consultants, providing candid advice to the airport director, and even voicing opinions to the project team on the director’s behalf. All but one were paid for their time as well as travel expenses—a departure from the other peer review approaches encountered. This director thought the executive peer advisory team provided a valuable alter- native to the more common host-focused method. The direc- tor suggested that participants at host-focused meetings have a harder time disconnecting from how their own airports operate, and that consultants are often focused on generat- ing future business. This can make it more difficult to get both candid and unbiased opinions relevant to the host’s project. In another case, the airport manager interviewed had assem- bled an ongoing advisory team at the suggestion of, and with the help of, the contractor hired to complete the environmental impact statement (EIS) for an airport-wide improvement pro- gram. Because of the complexity of the project, the consultant wanted to bring in an advisory team that could share lessons learned on similar projects and offer guidance as challenges arose. The consultant contracted with a small team of strategic experts to attend four or five workshops related to stakeholder outreach, regulatory issues, strategies for the EIS, and envi- ronmental mitigation. The peer team included airport directors or former directors who had handled the types of challenges expected to arise on the project related to wildlife impacts, noise pollution, and community relocation. Once again, these advisors were paid for their time, but their role did not include providing recommendations. They shared what had worked for them and acted as a sounding board and resource for the air- port. The airport manager on this project found this expert input invaluable and often tapped into the group by phone or e-mail. SITE VISITS Tours of Airports Sometimes the best way to learn from others is by observ- ing their practices first-hand. Two managers interviewed for this report embarked on what they called an “industry tour” over the course of their mega project for Seattle–Tacoma International Airport. They assembled a team of community stakeholders that included county executives, economic devel- opment corporation representatives, and media representatives to tour airports in North America and Canada. The stakeholders visited different airports during the programming and design phases to see what worked, what did not work, and what they wanted for their own project. These one-day visits exposed the team to innovative approaches to art in public places, retail and food vending, baggage systems, and customer service. The goals were to learn from the practices of others and build sup- port among stakeholders for planned actions by the airport. Each airport visit was carefully planned to make effective use of time. The airport director contacted his counterparts at the destination airports to request the visit and to develop a schedule of activities. The host airports proved very helpful and

9 flexible, volunteering staff to lead tours and answer questions. The stakeholders who participated also volunteered their time, and the airport covered their travel expenses. Site Visits at Single Airports Several interviewees said they take advantage of a smaller- scale version of the industry tour by making one-day site visits to peer airports. The visit may be prompted by a spe- cific project, such as the development of a new de-icing pro- gram or planned implementation of a new software system; or it may simply be an opportunity to see multiple departments in action at another airport. According to one airport manager interviewed, it is impor- tant to get senior level support at the host airport in advance so the visit runs smoothly. Airport directors or senior managers make the initial contact with the host airport, describe what they hope to learn and discuss during the visit, and send specific questions; the host airport lines up staff that can best address these during the tour. The cost to complete a site visit is relatively low, just the travel expenses for staff members who make the trip. The host airports donate their time and have been very willing to help. One airport manager interviewed for this synthesis budgets about $1,500 a year for site visits, which could cover the travel costs of four people to a nearby airport or one person to an airport farther away. Conducting site visits benefits the airport staff making the trip but also encourages reciprocal efforts and promotes better relationships. The site visits have included international airport exchanges as well. Peer-to-peer learning on a small scale is also possible within a single airport system. One manager interviewed said that he encourages staff within his airport system to review each other’s work. “I have project managers in five airports and can take them from one airport to listen to a presentation by another airport and get an internal review fairly quickly and inexpensively,” he said. “They don’t have to worry about offending the other staff because they don’t work for them.” Site Visits Outside the Airport Industry In 2008, AASHTO created a program through the NCHRP that formalizes and documents site visits among peers in the highway industry, with the goal of “facilitat[ing] information sharing and technology exchange among the states and other transportation agencies, and identify[ing] actionable items of common interest” (US Domestic Scan Program 2012). The NCHRP Domestic Scan Program funds and administers up to five scans per year. Each scan involves a team of eight to ten members visiting two to six sites over a one- or two-week period. The team observes practices and technologies at work in the host agen- cies, and then creates a report and implementation plan to facilitate use of the practices at home agencies and across the country. The scan topics are suggested by state DOTs and FWHA across a wide range of transportation subject areas. Completed scans have addressed accelerated construction, winter maintenance, motorcycle safety, and bridge manage- ment, to name a few (US Domestic Scan Program 2012). NCHRP funds the management of the scan process, which includes identifying promising technologies and host sites, defining scan duration and content, preparing materials, and reporting. Scan team members and the staff at the host sites volunteer their time, but NCHRP reimburses their travel expenses. It is a substantial commitment with equally substan- tial benefits. By tracking the program effectiveness through interviews with scan participants and those with whom par- ticipants shared results, NCHRP has found that participants are inspired to implement and share what they have learned. Past participants have shared scan findings with colleagues in their own agencies, with upper management, with local organizations, and with participants in national and inter- national meetings and conferences. The hands-on learning is “indispensable, providing an opportunity to interact directly with those who developed the relevant technologies” (Casey et al. 2012, p. 7). In addition, the professional connections made through the scan last long after it takes place. TECHNICAL ASSISTANCE Examples Within the Airport Industry The following organizations provide peer-to-peer technical assistance in the airport industry: • ACI-NA • AAAE • NASAO • Airport Consultants Council • Various state airport associations • Various regional airport associations. These industry organizations provide on-site or online train- ing in many aspects of airport management and operations, develop guidance documents and tool kits, and provide other resources to assist with professional development. In addition, airport professionals seek out technical guidance and assis- tance on specific topics as needed through airport-to-airport communications. Examples Outside the Airport Industry A notable example of organized industry technical assistance is the Peer-to-Peer (P2P) Program created by FHWA more than a decade ago to help highway professionals access expertise on specific topics. FHWA P2P staff manages the clearinghouse of public and private sector peers certified

10 in specific skill sets. Agencies contact the P2P Program (typically in a formal request) for technical assistance and are matched with peers who can help them [“Peer-to-Peer (P2P) Program Guidelines” 2012]. The program is funded by FHWA as a service to the states. Some airport professionals have taken the opportunity to access this support in technical areas that overlap with the highway industry, such as concrete paving and lighting issues. Technical assistance from peers can include telephone con- versations, e-mail exchanges, web conferences, on-site visits, or peer exchanges (similar to the host-focused review format). An individual may simply need assistance with interpreting a technical document or may want to hear from multiple peers about their successful practices in implementing a new pro- gram. FHWA maintains separate P2P programs to serve dif- ferent transportation professionals in areas including traffic control devices, work zones, scenario planning and visual- ization, roundabouts, and freight professional development. ONLINE FORUMS This final group of peer review formats is included to high- light the value of tapping into the experiences of peers through less formal channels. Several interviewees for this report indicated that they make use of listservs (online groups) to seek out quick-turnaround feedback from peers. Participants e-mail single questions or short surveys to an established online group, and receive e-mails and phone calls from group mem- bers describing their experiences. One interviewee marveled at the number of responses he received to his contracting questions, either e-mails with sample documents attached or phone calls from peers willing to be candid about their expe- rience. On the other hand, another interviewee cautioned that it is important to consider the perspective of the respondents when weighing the advice provided. Each airport is unique and what works for one may not work for another. The for- mat doesn’t support in-depth exploration of a topic, but it is a starting point and provides a glimpse into industry practice at no cost. A number of airport industry organizations maintain e-mail listservs to help professionals connect with one another and have embraced LinkedIn (www.linkedin.com) as a tool for supporting peer discussions. Through these informal online groups, participants can post ideas, request feedback from peers, and stay abreast of new practices within the industry. This format for peer interaction continues to evolve as users modify the forums to fit their needs. Here are a few of the industry-sponsored LinkedIn sites where airport professionals currently connect with their peers: • Airline & Airport Professionals Worldwide • Airport Aviation Ground Training • ACRP • Airport Design—Engineering & IT Systems • Airport Environmental Professionals • Airport Industry Professionals • Airport Innovators • Airport IT Projects • Airport Management • Airport Managers • Airport Operations Information Exchange • Airport Planners • Airport Public Safety and Operations • Airport Security • ACI-NA • AAAE • Aviation Professionals • International Airport Review.