National Academies Press: OpenBook

Conducting Airport Peer Reviews (2013)

Chapter: Chapter Three - Making the Most of Peer Reviews

« Previous: Chapter Two - Current Practices in Conducting Peer Reviews
Page 11
Suggested Citation:"Chapter Three - Making the Most of Peer Reviews." National Academies of Sciences, Engineering, and Medicine. 2013. Conducting Airport Peer Reviews. Washington, DC: The National Academies Press. doi: 10.17226/22555.
×
Page 11
Page 12
Suggested Citation:"Chapter Three - Making the Most of Peer Reviews." National Academies of Sciences, Engineering, and Medicine. 2013. Conducting Airport Peer Reviews. Washington, DC: The National Academies Press. doi: 10.17226/22555.
×
Page 12
Page 13
Suggested Citation:"Chapter Three - Making the Most of Peer Reviews." National Academies of Sciences, Engineering, and Medicine. 2013. Conducting Airport Peer Reviews. Washington, DC: The National Academies Press. doi: 10.17226/22555.
×
Page 13
Page 14
Suggested Citation:"Chapter Three - Making the Most of Peer Reviews." National Academies of Sciences, Engineering, and Medicine. 2013. Conducting Airport Peer Reviews. Washington, DC: The National Academies Press. doi: 10.17226/22555.
×
Page 14
Page 15
Suggested Citation:"Chapter Three - Making the Most of Peer Reviews." National Academies of Sciences, Engineering, and Medicine. 2013. Conducting Airport Peer Reviews. Washington, DC: The National Academies Press. doi: 10.17226/22555.
×
Page 15
Page 16
Suggested Citation:"Chapter Three - Making the Most of Peer Reviews." National Academies of Sciences, Engineering, and Medicine. 2013. Conducting Airport Peer Reviews. Washington, DC: The National Academies Press. doi: 10.17226/22555.
×
Page 16
Page 17
Suggested Citation:"Chapter Three - Making the Most of Peer Reviews." National Academies of Sciences, Engineering, and Medicine. 2013. Conducting Airport Peer Reviews. Washington, DC: The National Academies Press. doi: 10.17226/22555.
×
Page 17

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

11 SELECTING THE RIGHT FORMAT There are several factors to consider when selecting a peer review format to use, such as the desired outcomes and staff capabilities. Perhaps the most important takeaway from those interviewed for this report is that the peer review process should be flexible and customized to meet an airport’s needs. All review formats were found to be valuable, no matter the size or structure; and airports are employing numerous varia- tions of each model. The key is to recognize the limitations of each approach when determining the best fit for a given situation. For example, a large conference-style exchange offers opportunities to network and to learn about effective practices being used around the country. Interviewees generally found this format productive and valuable to both the host and the participants. However, the conference-style exchange does not allow for the same in-depth discussions and problem- solving as a host-focused review. Some interviewees also commented on the challenges of facilitating a large group and providing opportunities for everyone to contribute. The host-focused review is the most common “bang for your buck” peer review. The smaller size makes it possible for all peers to participate in the round-table discussions, and the longer duration means that more issues can be addressed, or addressed in more detail. Interviewees reported very good experiences with this format, especially because of the “high caliber” of the peers involved. Airport managers reported selecting this format to get input on master planning processes, review design approaches, discuss contracting practices, and review vendor proposals; and to explore plans for improvement on a wide range of areas, such as baggage, security, informa- tion technology, and concessions. However, the peer team travel costs for host-focused reviews may be prohibitive for a smaller airport. One common way in which airports have addressed the challenges of planning and carrying out conference-style exchanges or host-focused reviews is by working with an out- side consultant. Most of the interviewees contacted described a close working relationship with a representative from a national association, the program manager on their design team, or an independent consultant in the industry. The con- sultants provide a wide range of support services, recruiting experienced peer experts to participate, drafting agendas, facilitating the peer review, and documenting the discussions. Sometimes the host airport hires the consultant, and other times the services are provided through membership in an industry organization. Using an outside consultant is by no means necessary to hold a successful peer review, but many airports have found the assistance valuable when internal staff is not available to take on peer review responsibilities. Table 1 presents the range of the peer review formats described in chapter two, with guidance to help managers select a format that will best suit their needs. As illustrated by the case examples in chapter four, choosing one peer review format for a project does not necessarily mean excluding the others, and the biggest or most expensive approach isn’t necessarily the best. Nor does every peer review need to fit into a single model. Interviewees shared a variety of modified approaches to gathering peer input, such as: • Contracting with third-party firms to review the work completed by design consultants; • Bringing in consultants not involved in a project to help develop Request for Proposal (RFP) language or evaluate vendor proposals; • Traveling to another airport to meet with senior staff about successes and lessons learned on a recently completed project; and • Encouraging staff at neighboring airports to visit one another to share successful approaches in the field. TIMING CONSIDERATIONS Interviewees agreed that a peer review can be beneficial at any time in the project development process, but several respondents thought that the most useful reviews take place before critical decisions have been made by the host agency— that is, when they can still make a difference in the develop- ment of a project or implementation of a program. However, a number of successful and influential peer reviews have been held at the end of a process, validating what worked well and informing future practices. Interviewees described a range of effective peer review timing: near the beginning of a new project or when a new director or team comes on board; partway through a project, when there is still an opportunity to change direction, if needed; chapter three MAKING THE MOST OF PEER REVIEWS

12 TABLE 1 PEER REVIEW FORMATS Format Goal Size Duration Cost Pros Cons Conference-style Exchange Get help from peers to solve a strategic problem. Participate in a broad industry information exchange on a range of topics. 20 to 40 people; includes your airport staff and consultants, visiting airport staff, and visiting consultants One full day Participants pay to attend (flight, hotel, meals) and donate their time. Host airport may cover snacks and a meal. Strategic discussions are useful to high-level airport managers. Exposure to wide range of innovative ideas and practices All participants learn, not just the host airport Low cost for host airport Great networking opportunity • Strategic discussions are typically geared toward the highest level airport decision maker. • Large size makes consensus difficult. • Wide range of perspectives offered; may not all be applicable • No time to discuss the host airport’s issues in detail Host-focused Review Get detailed input from peers related to specific project or program issues 15 to 25 people; includes host airport staff and consultants and staff from four to seven visiting airports 2 or 3 days Host airport pays travel costs for visiting participants. Visiting participants volunteer their time. There may be charges for contracting a facilitator or planner. Discussions result in implementable recommendations. Time to discuss the host airport’s issues in detail Moderate cost for the host airport Great networking opportunity • More expensive than conference-style exchanges Performance Audit Complete a self-assessment of current practices and receive formal peer evaluation for trouble spots Small team of peers or industry experts provides review of practices. Up to a week Costs are low if inviting volunteer peers from another airport to review host airport practices. Costs may vary if accessing audit services from industry organizations. Formal assessment and evaluation results in specific improvement goals • Can be more time- consuming than other types of reviews Executive Peer Advisory Team Get candid advice from single peer team over the course of a project 3 to 6 peers Multiple meetings or interactions over the course of several years The host pays the hourly rate for the peers’ time and reimburses their travel expenses. Unrestricted access to experts for consultation Unbiased assessments of project challenges • Costs can be high. Industry Best Practices Tour See multiple approaches to 5 to 10 individuals on the Multiple one- day visits over Host airports volunteer their time. Costs Opportunity to see multiple approaches in • More planning involved in an airport traveling team the course of include the travel person coordinating multiple • • • • • • • • • • • • • function several years expenses for the visiting team. trips Single Airport Site Visits See a project or program in action 1 or 2 staff members travel to meet with a handful of staff at the host airport. One day The host airport volunteers its time. Costs include travel expenses for the visiting team. Hands-on experience and observations Great for establishing long-term connections • Input is limited to the airport visited. Technical Assistance Learn a new skill or area of expertise Varies. May involve computer- based training, in- person classes, or field experiences with instructors Varies depending on the number of topics covered Some services are available through industry associations for little or no cost while others are more expensive. Airport advancements supported using the latest information, technology, and practices. Front-line workers involved in peer experiences and professional development opportunities • Formal industry assistance is available in predetermined subject areas rather than in response to the unique needs of the airport. Online Forums Get an answer to a question quickly Reach a listserv or other online group with hundreds to thousands of airport professionals Feedback comes quickly— within a few weeks. Free or minimal (for compiling results/feedback) Input is fast and free. Input includes a wide range of perspectives that represent the “pulse” of the industry. No need to figure out who has the information to offer. Those who have information to share will do so. • Input is not very detailed. • Input comes from peer airports that may not be from similarly structured or sized airports. • • • • • • •

13 or multiple times throughout the life of a large project. Here are examples of interviewees’ comments on timing: “Any point the project team needs to validate assumptions or strategies to help a project be successful.” “They were about 50% through their design. They contacted us about a month in advance.” “Don’t go too far down the road with so many issues before having a review.” “The program was just getting started, and we thought it would be a good icebreaker and good for planning and organization of teams.” “The host was well into their planning process and at a point where they had defined projects and had sticker shock. This was a good time. Get to the point where you have some projects defined and see how they meet or don’t meet needs, and bring in smart people to react to them. Have them think about proposed solutions you’ve already identified but not tell you how to do it.” “It’s good to hold the review early on if you’re starting some- thing new. Look at it as an iterative process.” “It’s better to question in the early stages than to get down the road. A peer review can happen in different stages but earlier is better when you have the possibility of making changes if you need to. Don’t do it at the end of the project but early on when there’s enough substance to vet issues of interest.” “We held a couple of peer reviews on design—one in initial planning looking along the lines of functionality of the facility. The second review was prior to initiating a contract or RFP for construction. We looked at constructability, phasing—how we build it and keep the airport operational at the same time. The third review was when the design was advanced but [we] could still look at aesthetics.” The interviewees also discussed the time needed to plan for the review. They most often cited a few months as the length of time spent developing the agenda, inviting participants, and organizing the logistical details. However, most partici- pants were contacted no more than a month in advance. None of the participants complained that they did not have enough time to obtain approvals, make travel arrangements, or other- wise prepare. Site visits for the industry tour, however, were scheduled several months in advance to ensure the best prices on flights and the availability of government rates for hotels. ESTIMATING A BUDGET Getting feedback from peers can cost nothing (when send- ing a short survey to an industry listserv) or many thousands of dollars (when hiring an executive peer advisory team). Choosing the right review format will necessarily be influenced by the available budget. As one interviewee put it, “Do what you can afford.” Here is an overview of estimated costs for several peer review formats actively used within the airport industry, as described by interviewees. Conference-style exchanges: Participants pay their own way and donate their time, so costs to the host are fairly minimal. The host will need to pay for any materials to be distributed at the event and typically provides lunch and snacks for the group; and hiring a consultant to help with planning and/or facilitation will increase the costs. Host-focused reviews: The host airport covers the travel costs of the visiting peers (airfare, hotel, meals), which add up quickly depending on the number of people invited and the length of the event. Holding a two-day event with five vis- iting panelists could cost well under $10,000. If closer to 10 panelists and many internal staff are invited (and meals and snacks are provided), a three-day event may rise to between $15,000 and $20,000. One airport consultant indicated that $100,000 was budgeted to cover the planning and travel costs for three peer reviews for a large airport construction project. The costs may be lower if some of the planning and reporting is handled by airport staff or through other nonpaid means. ACI-NA and Airports Council International–World offer sup- port for peer reviews (identifying and recruiting participants, assisting with agenda development, facilitating, and reporting) as a service to members. Executive peer advisory teams: Expert panelists assem- bled for this type of review are paid an hourly rate for their time (in one case, $200 per hour). The host airport also covers the advisory team’s travel to the in-person meetings. The peer advisory team provides assistance over several meetings and several years, which adds up: One interviewee estimated that it cost approximately $200,000 to cover all advisory team expenses, but emphasized that was a fraction of the $1 billion project cost, and well worth it. Industry tours: The initiating airport typically pays the travel expenses for the team that participates in the tour of airports, although additional individuals may want to join the group at their own expense. The tour participants (both visitors and hosts) donate their time. The costs depend on the number of airports visited and the size of the team traveling. One airport director estimated $50,000 had covered the costs of an entire multi-year tour spanning nearly a dozen airports. Single airport site visits: A one-day site visit typically involves sending one or two people to another airport. The host airport staff members volunteer their time, so the cost is limited to travel expenses. If an overnight stay is necessary, costs might range from a few hundred dollars (for a visit to an airport within driving distance) to $2,000 (for an airport farther away requiring air travel). DEVELOPING AN AGENDA The work that goes into developing and communicating the components of the agenda is the most important factor influencing a successful peer review. Over and over again interviewees emphasized the importance of having an agenda that was carefully thought out and structured enough to help

14 the team accomplish the desired goals. Whether organizing a large conference or a small day trip to a nearby airport, it is critical for an agency to identify the purpose of the review and clearly communicate expectations to all involved. The goals of the peer review drive the agenda, which in turn drives the list of invitees. As one interviewee said, “Know your agenda before you contact folks.” Another commented, “Make sure you have well-defined goals so you don’t waste anyone’s time. Have clear communications to make sure people understand why they are there and what is expected.” For the frequently used host-focused review, interview- ees reported the most experience with the following basic agenda structure: The host presents the problem or issue, the peers provide feedback and/or brainstorm solutions, and the group summarizes (and perhaps reports) the recommenda- tions. In some cases, the peer review panel met separately from the host staff to discuss the issues and then make rec- ommendations for presentation to the host team. Articulating the problem or issue provides the context for the discussions with the peers. Many of the interviewees commented that it is helpful, if not critical, to provide the par- ticipants with at least some of this background information in advance: the history of the project, documents showing work completed so far, an outline of issues to address, and guidance on the type of feedback desired from the peer participants. Here are a few interviewees’ comments on this topic: “It would be better to hear about the host airport process more ahead of time. It would be more efficient, and we could come with recommendations. You don’t want to spend too much time listening to presentations. Know going into it what you can and can’t change.” “Give as much info as you can beforehand in [a] digestible format.” “It’s useful to get information as a participant a few weeks in advance so you can educate yourself about a project and know what you’ll be asked about. Some reviews you show up and hear everything there.” “The first day the host team presented to the group the detail behind the slides, so it was really well done. By taking a day to have the key people talk to the group and say what they’re facing and the decisions they’re making, they opened themselves up to helpful assistance from the group.” Regardless of the amount of information provided in advance, the interviewees found it necessary for the host airport to outline the review goals at the beginning of the event. Interviewees had varying experiences regarding how the peers provided input to the host airport. In some cases they made formal presentations about their own organizational structure and the lessons learned on similar projects. This was helpful for illuminating each peer’s perspective and provided initial ideas regarding what might work for the host airport. In other cases, the group jumped right into a roundtable dis- cussion with questions and answers. The decision to have peer presentations may depend somewhat on the time available and the number of participants. Advance communication plays an important role in this case as well. Interviewees sug- gested telling the peers specifically what they should cover in their presentations and how much time they will have. Here are examples of interviewee thoughts regarding peer presentations: “Everyone should present on things that are relevant to the host airport. Could have been improved with more guidance regarding expectation for presentation.” “It’s not useful to have ten or 15 people giving presentations on lessons learned. Require the peers to come with their experience so they can discuss issues and stay away from participation that is off topic.” All interviewees said that the host-focused reviews they participated in included active discussions with the peers, either in question-and-answer sessions or in working groups that pored over documents and provided specific feedback. This was the part of the agenda that yielded the most value for both the host agency and the participants. In some cases, the host airport sent questions to be addressed in advance (which further helped the peers prepare), but in other cases there were no set questions used even at the event. Several par- ticipants found more informal discussions productive as long as all of the key topics were addressed. The following are inter- viewee thoughts on the discussion portions of the agendas: “Informal discussion was good. It’s good to be guided but not too specific.” “One-on-one dialogue helped them give us their feedback. It’s the best way to communicate goals and objectives.” “The workshop setting was our ‘aha!’ They looked at the docu- ments and gave us feedback. There was better dialogue.” “We went through introductions and gave a short synopsis of where we were at in program decisions. A wonderful, open dis- cussion. The peers came in with the challenges they’re facing and what went well in their experience.” “We had a structured agenda with a matrix and timeline and worked through it. Facilitated discussions with questions. Everyone was able to be candid.” The final part of the agenda usually involves a wrap-up or reporting session. This may be an informal recap of the key issues discussed, or the group may develop a list of formal recommendations. The nature of this last session may depend on the original goals for the review. In some cases, an airport manager wants the peer team to present its reactions and recommendations to an executive team. In other reviews, the host airport may simply want food for thought on a proj- ect rather than a consensus opinion on the best approach. Interviewees suggested structuring the agenda to support the desired goals. As with selecting a peer review format, there is a lot of flexibility in designing an effective peer review agenda. Some

15 host agencies include field trips to help visitors further under- stand the project, some set up breakout groups to delve into multiple topics at once, and some arrange welcome dinners the night before the meeting to help the peers and host air- port staff get to know one another. Whatever the approach, interviewees thought it was important to communicate expec- tations in advance, including timing, format, logistics, and expenses. This includes communications within the host agency as well. As one interviewee reflected, “I didn’t take the opportunity to meet with my staff at the very beginning about expectations about what would happen at the review, which would have made it better for everyone.” Sample Agenda Format for a Conference-Style Exchange One-day format • Invited speakers present the national trends related to the topic. • Host airport presents issues with a project in progress. • Visiting airports each present lessons learned on a related project. • A facilitated group discussion addresses common issues and challenges. Sample Agenda Formats for a Host-Focused Review One-day format • Host airport provides an overview of the project to discuss and the decision points at hand. • Visiting peer panel members participate in a hands-on review of the host airport’s relevant documents for the project, providing feedback and thoughts based on their own experiences. • Participants discuss pros and cons of the alternatives presented and create an informal recommendation for the host airport. Two-day format Day 1 • Host airport presents details about a project in progress and any questions or issues encountered. • Visiting peer panel members present their lessons learned on similar projects. Day 2 • Host conducts a facilitated group discussion regarding opportunities for the host airport. • Host facilitates the final wrap-up. Three-day format Day 1 • Host airport presents an overview of the project and spe- cific topics on which it needs feedback from the group. • A facilitated group discussion is held on topics 1 and 2. Day 2 • A facilitated group discussion is held on topics 3, 4, and 5. • Host conducts a tour of the airport or project discussed. Day 3 • Host facilitates the final wrap-up and lessons learned. Refer to Appendix C for agendas from completed peer reviews provided by interviewees. SELECTING PARTICIPANTS The airport industry enjoys an extremely collaborative, sup- portive environment in which airports welcome the oppor- tunity to share what they have learned and offer assistance to others. Every interviewee commented on the willingness of other airports to participate in peer review activities. Peer reviews offer all participants the benefits of expanding pro- fessional networks and learning new things. One interviewee said, “It’s an honor to be selected as a peer reviewer. You want to do your best for the group.” Therefore, the challenge in selecting participants for a review is not in finding those who are willing to help but in finding those who have the perspec- tive and experience to be most helpful. Peer reviews involve a combination of internal and exter- nal participants. Internal participants are typically staff and consultants at the host airport (or the staff traveling to another airport for a site visit), but may also include those working with the airport on a project, such as consultants, airline rep- resentatives, and community stakeholders. These individuals are invited to participate based on the topics being discussed, what they can contribute to those topics, and what they need to learn. For example, it may make sense to have selected techni- cal staff (architectural, structural, operations, etc.) available for part or all of the program to help explain project details and hear the feedback firsthand. Managers may want to invite an airline representative to strengthen airline support for actions initiated as a result of the review. Several interviewees, however, cautioned against making the size of this internal group too large. Including 20 or 30 inter- nal attendees can make facilitation very difficult or minimize the peer contributions. The following are some comments offered by interviewees related to internal participants: “We kept internal staff to a minimum. We had subject matter experts on our team but were most interested in feedback from the panelists. We didn’t want to be busy defending our ideas. You’re in a listening mode versus talking mode. This is important.” “Sometimes an airline representative is involved, which is important if it’s a large development program that needs airline support.” “Who you have there depends. You might have senior leader- ship and staff, airline representatives, elected officials (though not usually). Consultants are good at bringing specialized expertise.

16 The public may be there for certain types of master planning. Could be suppliers if the review is specialized and they’re subject matter experts.” The external participants make up the peer panel that offers feedback to the host airport. At a conference-style exchange, 15 or more airports may be represented. However, at the more common host-focused reviews, interviewees indicated the panel was limited to three to eight peers. A few interviewees believed that three peers were plenty to offer different perspectives, but one was disappointed by this low number and expected more. The respondents talked more about the desired back- grounds of the panel members than about how many are best to have in the room. Interviewees most often described the peer experts as “high caliber,” “experienced,” “senior level,” and “well respected in the industry.” They looked for panel- ists who had experienced similar challenges and had subject area expertise related to the peer review topic(s). In some cases it was important to find representatives from similarly structured airports so that they could best relate to the host airport’s challenges and offer implementable solutions. The following are some specific thoughts shared by interviewees on the ideal makeup of the external peer panel: “We cast a broad net knowing we wouldn’t get everyone. Some were from airports, a couple of university professors, a consultant for an airline, a former FAA high-level person.” “Have the airport representatives come from similarly sized and structured airports, because ideas will resonate more and be relevant.” “For one we had the heads of planning or facilities at airports; for one we brought in architects and program managers. You don’t want more than five people unless a lot are locals. You add your own folks as well. As many as ten gets expensive.” “You can bring in senior people with a lot of experience or lower level deputy directors with hands-on responsibility. We’ve brought in academics in operations research, a lieutenant general from the Air Force on active reserve, policy people, and a former FAA administrator.” “The trick is to get the right people on the panel who have exper- tise in the area the airport wants to address.” “We had big and small airports, which was better. The concepts are the same no matter what you’re spending on a project.” “Be mindful when choosing people to participate to get the right fit. If the subject can be applied regardless of governance struc- ture, it’s less critical; but if digging into issues related to state laws, you need to have this in mind when choosing.” “Avoid peer reviews where most of the participants know each other. It leads to group think.” “The shotgun approach to selecting participants isn’t always best. When you send out requests to the [peer advisory group] you need to know the perspective you’re hearing from. Spend the time to find the right participants.” “Big airports can run away with discussions. Define expecta- tions up front and make sure ideas are implementable.” One interviewee did say that he “wasn’t a fan of the tradi- tional peer review,” favoring the executive peer advisory team approach: “People bring their bias of how things would work in their locations. They come in on a voluntary basis to give you their perspective, and most people are wrong. The consul- tants [who attend] want to drum up business.” To address this problem, he handpicks peer experts he trusts to be candid and knowledgeable, and pays them for their time. Most, but not all, of the interviewees had enlisted the help of an outside consultant (paid or unpaid) to help in planning the peer review and selecting participants. Several interviewees commented on the benefit of this assistance and referred to the “vetting” process needed to ensure that the right panel of experts participates. They said that the consultants suggested potential peers based on their extensive contacts, and the air- port executives did the same. An initial personal contact was made by phone by whoever best knew the selected peer. USING A FACILITATOR Most of the interviewees said that a designated facilitator participated in the reviews to keep the discussions on topic and encourage contributions from all participants. Sometimes the host airport executive initiated the discussions and asked the facilitator to help keep the group on track throughout the agenda. Often the facilitators came from within the airport industry and were very knowledgeable about airport issues. Although the interviewees considered subject area expertise important, finding someone who is skilled at guiding dis- cussions and handling conflict may be even more important. Only one of the interviewees expressed dissatisfaction with a facilitator who was unable to calm and redirect a group when a discussion went off track. Facilitators take on a range of leadership responsibilities. Often they participate in the planning and agenda development to help lay a strong foundation for the discussions that will take place. During the event, they ask questions to prompt discussion; encourage participation by all attendees; guide transitions between topics and sections of the agenda; recap the discussions, issues raised, and decision points; check in with the group as a whole to make sure everyone is comfort- able with how the agenda is progressing; and check in with the host agency to make sure the discussions are providing the information it needs. Given this crucial role in both planning and expediting the event, it is not surprising that many airports sought out consultants within the airport industry to fulfill it. The Research Center for Leadership in Action at New York University (2007) created a guidance document to help facilita- tors successfully plan for and lead peer reviews. The following are some of the key ideas suggested in this document, which echo the experiences of those interviewed for this synthesis: • Send information to participants about the topic to be discussed before the event. This will provide a common ground for discussion without using up more time than necessary at the event itself.

17 • Ask the participants to prepare thoughts on their own experiences related to the topic. This may involve having them answer a list of questions in advance or prepare a PowerPoint presentation to address specific points suggested. • Consider including time in the agenda for both story- telling (sharing real-life experiences and outcomes) and the transfer of concrete skills (through field observations or computer-based training). • Vary the format of the agenda to keep participants engaged and energized. An agenda that uses only round- table discussions on many topics may not provide needed breaks from conversation. An agenda that primarily involves formal presentations by numerous participants can feel burdensome and slow. Facilitators also need to ensure that there are plenty of breaks and opportunities for participants to network and recharge. • Provide an opportunity for evaluation. This can involve requesting informal feedback throughout the event and/or a formal evaluation at the end of the peer review. Cur- rently, the use of evaluations in peer reviews is noticeably lacking in the airport industry. None of the interviewees indicated that they had administered or completed a for- mal evaluation after a peer review took place. When asked about evaluations, several interviewees commented that they wished they had incorporated this step into their peer review process to find out what worked especially well or what could be done differently at the next peer review. • Make sure the discussions are documented. If possible, provide participants with written results of the peer review before they leave or follow up very soon after with a report. The facilitator can help ensure that the right information is captured. The report helps guide the host agency in taking next steps and helps the peer participants demonstrate the value of attending to their supervisors. The following are some of the specific comments provided by interviewees regarding facilitation: “[The consultant and airport director] facilitated together. [The director] set out [the] agenda and [the consultant] helped lead through the process.” “[The consultant] got us through the agenda and elicited par- ticipation. His background made it so there was never a lull. He kept the focus on what [the host airport] wanted, which was key. Each airport needs a clear facilitator who is knowledgeable about the issues or something could get missed or lost.” “She was able to get hidden agendas out of people.” “[The airport director] facilitated since he knew everyone.” “There was no designated facilitator. The host airport ran things, and each project manager handled their piece.” “[The consultant] facilitated so we could spend our time think- ing and discussing.” REPORTING ON THE OUTCOMES Interviewees described several methods used for document- ing the discussions and outcomes of the peer reviews. In a few cases, the host airport thought it was important for the group to reach consensus about an issue and then verbally report recommendations to a stakeholder or executive group assembled at the very end of the event. In other cases, no report was developed at all. Typically, however, a designated note-taker captured the range of discussions and distributed the report for review by participants after the event. Often the note-taker was the consultant brought in by the airport to help plan and/or facilitate the peer review. One interviewee said, “It’s always important to develop a report so that it can be used to motivate staff and communicate with the media.” However, the timeliness of the report may influence its impact. Several interviewees commented that the report was developed months after the peer review, so they were no longer interested in reading it. A few airports took a strategic approach to reporting. They created comment sheets or matrix documents that the group completed during the event. These documents supported development of an airport action plan that was used to track progress in the months after the peer review. Overall, inter- viewees found the most value in reports that didn’t “sit on a shelf” but were used to support subsequent steps. The struc- ture of the report can have an impact on its usability. As one interviewee said, “It would have been better for the report to have thoughts prioritized and not just be a dump of the dis- cussion.” A timely report that clearly presents action items, recommendations, and/or key findings can provide strong support for implementation and communication after the peer review. As airport managers think through all of the aspects of effective peer review planning described earlier, it may be helpful to consult a resource that will be available in the fall of 2013. NCHRP Project 17-52, “Using Peer Exchanges to Improve the Effectiveness of a State’s Strategic Highway Safety Plan,” will result in a guidebook to help states in plan- ning and conducting peer reviews related to implementation of their highway safety programs. Although the intended audience is highway planners and not airport managers, the guide may include useful templates and direction that are relevant to any peer review planning effort. For more information, see the project description on the NCHRP web- site at: http://apps.trb.org/cmsfeed/TRBNetProjectDisplay. asp?ProjectID=2976.

Next: Chapter Four - Case Examples »
Conducting Airport Peer Reviews Get This Book
×
 Conducting Airport Peer Reviews
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s Airport Cooperative Research Program (ACRP) Synthesis 46: Conducting Airport Peer Reviews explores the range of peer review approaches being used by airport sponsors, identifies similar efforts outside the airport industry, and documents both effective practices and challenges in conducting peer review activities.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!