Virtual Public Involvement: Lessons from the COVID-19 Pandemic (2022)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

8  C H A P T E R 3 Summary of Findings from Phase 1 Section 1: Overview of the transition to VPI during the COVID-19 pandemic VPI readiness Most transportation planning agencies had already begun to implement some type of VPI before the COVID-19 pandemic. In focus groups, agencies talked about using social media, digital surveys and participating in the Federal Highway Administration’s Every Day Counts initiative that encourages technological innovation. In addition, smaller planning agencies often had, if nothing else, a website where citizens could contact the agency or engage with projects. For example, Pennsylvania DOT and Louisiana Department of Transportation and Development (DOTD) offered public meeting presentations and project information on their website. Wisconsin DOT shared experiences pre-pandemic of webcasting and utilizing YouTube Live to stream events. Several nonprofits that serve overlooked populations shared that they had some experience with VPI before COVID-19. For example, the Great Lakes ADA Center is one of 10 federally funded regional ADA centers whose mission is to promote voluntary compliance with the Americans with Disabilities Act of 1990. The center shared that they were well poised to transition during the pandemic to an increased usage of VPI as they had undertaken virtual engagement for many years, convening audio conferences in the late 1990s and initiating the use of a virtual platform in the early 2000s. In addition, as a multistate resource, the center has maintained a website presence and offered email and phone support for many years. Similarly, the Cincinnati Adult Education Solutions program, which supports diverse and transient communities seeking GED or high school equivalent education, reported that their transition to virtual offerings was easier for staff because of their existing online distance-learning program. Transitioning to VPI during COVID-19 The first COVID-19 case in the United States was identified in January 2020 and by mid- March 2020, many states and cities were issuing shutdown orders. The shift from in-person to virtual came with very little warning and imposed difficulties for most agencies. Agencies quickly went from working partly or entirely in person to entirely remote and, simultaneously, had to figure out new tools and strategies for internal and external planning and virtual engagement. Agency staff had to learn new platforms and develop and adopt policies for virtual engagement on a significantly accelerated timeline. All of this took place during a time of extreme uncertainty. Much of the economy was shutting down, and the duration of the pandemic was unclear, with agencies not knowing whether to expect shutdown orders to last for weeks, months or years. Many agencies said they may have made different decisions at the onset of the pandemic if they had known how long COVID-19 would impact daily life.

9    Navigating the initial transition to VPI was the most difficult task for most agencies. One participant noted that early in the pandemic, it was unclear which virtual engagement tools were most effective and reliable, so the agency cycled through many tools to find the most suitable. Another noted that their agency was reluctant to invest in virtual tools until late 2020, given the uncertainty surrounding the pandemic. Training materials and guides to using virtual tools were not readily available and learning on the job was common. The skills and preparation needed for virtual meetings were very different from those required for traditional in-person interactions and posed new issues to address, such as network outages, uninvited participants disrupting meetings and constituents who either did not have access to technology or did not know how to use it. New protocols had to be developed, including identifying backup meeting hosts and understanding how to utilize virtual breakout rooms. Over time, agencies improved at implementing virtual engagement through trial and error and knowledge-sharing among peer agencies and internally.   Challenges and opportunities  After agencies became more comfortable planning and implementing virtual meetings and other VPI strategies, new challenges emerged. Virtual meeting fatigue was common, with staff and the public worn down from frequent virtual interactions that required staring at a screen for an extended period of time. Some members of the public became reluctant to participate in long virtual meetings at night after working in a virtual environment all day. One agency said they switched to having shorter meetings during the workday, typically around lunch hour, and found that this helped spur participation. Another mentioned having “office hours” that required less preparation and time commitment from staff while still allowing the community to ask questions and provide feedback. Agencies also found creative solutions to address issues of technological access, from partnering with community organizations and hosting hybrid events to working with local libraries to help people access the technology required to participate virtually. Focus group participants and survey respondents repeatedly stressed the importance of partnerships with community organizations for establishing trust and spreading information about meetings and other engagement opportunities. The challenges that came with virtual engagement also presented new opportunities for agencies. Recording virtual meetings is significantly easier than recording in-person meetings, so agencies began posting meetings and presentations online. This action allowed people who missed the initial meetings to watch asynchronously and helped to make the meetings more accessible. Other tools for disseminating information on-demand were developed, including self- guided websites, informational videos posted online and even site visits using drones. Agencies also increased their social media usage to promote their material, share information and distribute electronic surveys. Over half of the respondents from the survey for this report indicated that their agency used digital newsletters and email blasts to solicit feedback and send project information during the pandemic, suggesting that using VPI tools beyond just virtual meetings was commonplace. Another effective strategy was promoting virtual engagement through more traditional means. Since a major concern was reaching groups that may not have experience with or access to technology, promotion of virtual events could not be effectively done solely through digital strategies like social media, websites, and emails. Agencies also reported using various traditional strategies, including mailings, text messages, printed flyers, and radio, newspaper, and

10    bus advertisements to promote virtual meetings and project information. Over half of survey respondents reported holding hybrid events where there were opportunities to engage both virtually and in-person. Reaching targeted audiences Though varied, agency experience with virtual public engagement was generally positive during the pandemic. Some agencies had better experiences reaching seniors and other vulnerable groups with virtual engagement compared to in-person engagement. In contrast, other agencies found that these groups were harder to reach virtually. Many groups that were difficult to engage in person were still difficult to engage virtually. The barriers to in-person engagement, such as transportation, childcare and time, were replaced by new obstacles, including the need to have familiarity with VPI tools. However, while not universal, most agency representatives conveyed that virtual engagement was a net positive and has increased overall public participation. One downside of VPI that most agencies noted was the difficulty of determining who exactly they were engaging. It is challenging to assess participants' demographics in virtual meetings. Demographic information is also limited with on-demand engagement, where agencies can track the number of people who view material on a subject but may not be able to reliably determine who is using the on-demand tools. Finally, virtual surveys tend to be much easier to disseminate but may allow a single person to respond anonymously multiple times, potentially making the information gleaned from the survey less reliable. These are challenges with VPI that agencies are still grappling with today. Moving forward Moving forward, most agencies expect to use a combination of in-person and virtual engagement. Now that the agencies have the infrastructure for VPI in place, they can continue to take advantage of VPI while also offering in-person options. As virtual and in-person public involvement present different accessibility challenges, offering both options may be the best strategy for ensuring that as many people as possible who want to participate can. This hybrid approach presents new opportunities and challenges for agencies to work through. One focus group participant noted that the staffing required for virtual and in-person meetings is largely similar, although the responsibility for staff is very different. It is also important that agencies ensure that messaging is consistent both in-person and online. The public may misunderstand or even distrust the agency if messaging differs for different audiences. These challenges are worth working through to ensure that planning agencies do all they can to facilitate effective and equitable public engagement. Section 2: VPI training and resources utilized Staff training As noted, study participants shared primarily consistent experiences at the onset of the COVID-19 pandemic in the United States, with most noting their team had to transition rapidly

11    to virtual engagement complicated by varying levels of experience and knowledge of how to do so. Most participants did not experience the benefit of receiving formal VPI training. When asked in the study survey to specify the most common internal department or agency challenges with implementing VPI, lack of staff training was the most cited response (27 percent) followed by lack of internal VPI policy/guidance (19 percent). Notably, the types of internal challenges encountered did not meaningfully vary by respondent type. Forty-two percent of persons responding to survey questions about staff training reported that they had received some training. Among the three respondent types, more DOTs (60 percent) reported receiving VPI training compared with their MPO (46 percent) and RPO (22 percent) peers. By far the most frequent source of VPI training or technical support was furnished by internal staff, followed by the Federal Highway Administration’s (FHWA) guidance document entitled “Temporary Virtual Public Involvement During the COVID-19 Pandemic,” state/local government resources, peer organizations and customer support from the staff of specific VPI platforms such as Zoom. Among those who reported receiving training, more RPOs reported using non-FHWA federal resources as well as state/local government resources. At the same time, MPOs and DOTs were more likely to have used the FHWA’s “Temporary Virtual Public Involvement During the COVID-19 Pandemic” guidance document. Most focus group participants indicated they did not receive formal training on VPI. Instead, most shared they “learned on the job” or “learned by doing.” Some emphasized that implementing VPI demands learning a new skill set. As a participant from Utah DOT explained, planning a VPI event is similar to hosting a radio broadcast, necessitating the creation of a detailed broadcasting plan and designation of assigned roles for a producer, director, on-air talent, etc. Several focus group participants discussed receiving informal guidance/training from fellow staff members or consultants. For example, a participant from Wisconsin DOT noted relying on two staff members who had experience with VPI, including video creation and managing online comments. An interviewee from Goodwill International – a nonprofit organization focused on creating job training programs, employment placements and other community-based programming – similarly shared that staff who were more experienced with virtual tools were informally enlisted to train and support other staff. Regular support from consulting firms might be one likely reason that lack of formal staff training opportunities did not significantly impede VPI implementation for some agencies. For example, Pennsylvania, Vermont, and Louisiana DOTs explained that consultants took the lead in implementing VPI, especially for large-scale projects. Similarly, the largest share of survey respondents, 59 percent, indicated that a combination of internal agency staff and consultants/contractors typically implemented their VPI during the pandemic. However, this finding varied by respondent type. Specifically, while 77 percent of DOTs utilized VPI support from consultants/contractors in addition to internal staff, only 44 percent of MPOs and 40 percent of RPOs did so. The latter two organization types instead relied primarily on internal staff for VPI implementation. While formal VPI training was not a common experience, informal training and peer-sharing activities helped to fill the gap. For example, Ohio DOT developed a free online VPI training module targeted to project management staff, local public agencies, and consultants. This user- friendly module explores the differences between a virtual open house and a virtual public

12    meeting; provides tips for conducting virtual meetings; offers instructions on using VPI platforms; and provides a series of templates for flyers, postcards, mail inserts, newsletters, paid advertisements, and other useful public notification materials. The Chicago Metropolitan Agency for Planning developed in-house VPI training that included staff instructional guides for using specific platforms. Goodwill International explained that at the onset of the pandemic, program partners, including Google, supported Goodwill team members with technical assistance and training to provide virtual service delivery to their consumer base. The literature review also documents VPI training offerings provided by some agencies. For example, Massachusetts DOT developed a manual entitled “Guidelines for Successful Virtual Public Meetings,” which presents legal guidance, regulations, checklists, and recommendations for VPI. Tips on promoting accessible engagement are highlighted as are engagement tools and techniques such as live polling, surveys, and breakout sessions. In addition, the benefit of preparing a “run of show” outline, assigning roles to each staff member and conducting event rehearsals, is discussed. The state of Maine’s Greater Portland Council of Governments produced a 24-page white paper entitled “Inclusive & Accessible Virtual Engagement: Lessons from the Field,” which provides an overview of VPI best practices to promote accessible and equitable outreach. The report emphasizes the importance of training staff to implement VPI so that “regardless of the platform, moderators must be good listeners, work collaboratively with the community and understand how to positively synthesize ideas, strategies and actions to meet the needs of the community.” Volunteer training The topic of VPI training for organization volunteers, in addition to staff training, was raised in several interviews with nonprofit organizations supporting vulnerable populations. For example, the National Federation for the Blind (NFB) reported that they had purchased approximately 100 Zoom platform licenses for usage among their 52 NFB affiliates located throughout the country and offered training for staff and volunteers on how to use the platform. The AARP California state office interviewee similarly discussed the need to train their diverse cadre of volunteer teams, some of whom advocate on specific state and other policy issues, as well as district liaison volunteers who are paired with each member of Congress in California’s delegation. In addition, AARP’s local advocacy teams received training on VPI and advocacy work using the Zoom platform, and were taught platform features, including polling and breakout rooms. Other resources supporting VPI adoption among agencies As formal VPI training was not typically offered, study participants shared other strategies to acquire VPI skills. The most commonly cited was participating in internal and external peer exchanges. DOT and MPO focus group participants shared that peer exchanges on virtual engagement practices were valuable in supporting adaptation to this new engagement environment and contributed to improved outcomes. Some participated in external VPI-focused peer exchanges, while others established internal VPI working groups. Ohio DOT established an internal public involvement group that convened monthly to discuss best practices and lessons learned, while Maine DOT reported participating in a New England

13    regional VPI peer exchange. Several MPOs shared this sentiment regarding the value of peer exchanges. For example, Ohio’s Miami Valley Planning Commission reported that their peer exchange with Ohio DOT allowed participants to discuss VPI challenges and platform specifics. Other participants, including Caltrans, Maine and Ohio DOT, mentioned that their awareness of and involvement in the FHWA Every Day Counts program (EDC) was helpful as they sought to navigate strategies for successful VPI planning and implementation. As described by FHWA, “EDC is a state-based model that identifies and rapidly deploys proven, yet underutilized innovations that make our transportation system adaptable, sustainable, equitable and safer for all.” VPI is a topic covered in EDC-5 (2019-2020) and EDC-6 (2021-2022). Training needs moving forward Survey respondents, focus groups and interview participants discussed various ongoing needs related to virtual engagement training that would be helpful to their staff moving forward. For example, a survey question asked respondents to select the three most desired forms of assistance that would help their organization to improve or expand the implementation of VPI. As shown in Table 1, the two most cited requests were directly related to training. Specifically, more than 40 percent of respondents requested best practice resources and demonstrations, while 36 percent indicated online training and technical assistance support would be helpful. Close to one-quarter requested VPI peer exchange workshops, hands-on training, and technical assistance to support their continued VPI implementation. Table 1 – Requested assistance to improve or expand usage of VPI (N=196) Type of Assistance Requested  N  Percentage  Best practice resources and demonstrations  82  42  Online training and technical assistance  70  36  Funding support for VPI platform purchase  58  30  Updated VPI guidance  47  24  VPI peer exchange workshops  47  24  Development of internal VPI policies  47  24  Hands‐on training and technical assistance   42  21  Better technology infrastructure for constituents   3  2  Study focus group and interview findings afforded the collection of more detailed information on topical interests and needs related to VPI training. Key topical areas of training interest or need, as discussed, focused on the following:  Training on VPI technologies, resources and platforms – While most participants have learned on the job or more rarely via formal training how to use one or more VPI technologies and platforms, many continued to express a need for additional training and informational resources on the types of VPI technologies available, how to successfully deploy features of specific VPI platforms and how to best utilize these various tools. Some expressed interest in learning more about specific tools or platforms, and several suggested that the development of

14    a tool matrix to assist agencies in identifying the most appropriate tool(s) for the type of engagement being planned and audience(s) being targeted would be extremely valuable.  Virtual meeting facilitation – Another frequently repeated request among study participants was for training on virtual meeting facilitation strategies and techniques. As several shared, a distinct skill set is required to implement virtual meetings successfully. For example, virtual facilitators must know how to encourage, moderate and monitor virtual participation, including handling “dead air” and inappropriate or disruptive participants. Protocols must be developed and adhered to that focus on preparing for unforeseen circumstances in the virtual setting. For example, backup facilitators and presenters should be identified, with clear and distinct roles and responsibilities assigned to staff, etc. Overall, virtual meeting facilitation skills are not necessarily intuitive; thus, training on developing and honing such skills remains a need.  Virtual material presentation – Related to virtual meeting facilitation is virtual material presentation. Participants reported that best practices for communicating information online or in a virtual setting are not necessarily the same as in-person engagement. Participants requested training and guidance on strategies for material presented to diverse audiences so that agencies can convey the information they seek to impart consistently and concisely. They indicated needing more information on the selection of appropriate graphics, color palettes, fonts and similar features.  ADA compliance and equitable engagement – An ongoing need for ADA compliance training when undertaking VPI was discussed, with a participant from Caltrans explaining, for example, how their agency is extremely conscious of the need to ensure equitable engagement. Participants in the focus groups discussed difficulties with captioning and preparing accessible recordings and language translation. One suggested that the development of an “ADA VPI tip sheet” would be helpful. Finally, increasing and perhaps formalizing knowledge-sharing on VPI practices and strategies within and among agencies was discussed as a largely unmet training-related need that would yield diverse and enduring benefits. Specifically, multiple participants noted the benefit of information sharing, especially as agencies seek to plan and implement hybrid engagement activities and consider using new and evolving VPI technologies and platforms. In addition, increased institutionalized knowledge-sharing practices would assist in raising awareness and understanding of how to make the best use of diverse VPI tools and platforms to engage the public fully and equitably. Section 3: VPI tool use during COVID-19 The VPI toolbox The VPI toolbox is extensive and growing, with dozens of techniques and platforms available to transportation agencies. FHWA’s EDC VPI initiative featured eight categories of tools and techniques: mobile applications, project visualizations, do-it-yourself videos, crowdsourcing tools, virtual town halls, mapping tools, all-in-one tools, and digital tools to enhance in-person events. Additional types of tools and several ways of categorizing them were identified in the literature review/practice scan for this study. To understand VPI tool use during COVID-19 and

15    build a useful knowledge base for the VPI guide, this report distinguishes between tools, tool features and specific platforms.  Tools are broad categories of engagement techniques, such as virtual meetings, surveys, interactive maps and social media. These tools have a variety of functions, with some primarily geared toward gathering input, others intended for outreach or promotion and some designed to share information in support of an engagement process.  Tool features include characteristics that may enhance or define a tool or its application. For example, the features of a virtual meeting tool might include a registration requirement, the ability to call in from a landline phone or an option to use breakout rooms. The features of a survey might include multilingual options or the availability of specific question types such as trade-off bars or image comparisons (visual preference).  Platforms are specific software products, including 1) commercially available platforms such as Zoom meetings, Mentimeter polling, or MetroQuest surveys; 2) proprietary products offered by consultants or vendors as part of a project contract; and 3) the various custom tools that many agencies have developed for in-house use. Broad tool categories and tool features are of greatest interest to practitioners of VPI. Information gained about the effective, efficient, and equitable use of tools such as virtual meetings or surveys is expected to be more enduring than information on specific platforms. Tool features also are important in understanding the nuances of tool use and assessing both agency and public preferences for different types of virtual engagement. VPI tools used by survey respondents The survey asked respondents to indicate the types of VPI tools they used during the COVID- 19 pandemic as well as those they expected to use post-pandemic. They could select any that applied from a list of 17 tools. The results are shown in Figure 1. The tools most commonly reported were virtual public meetings, social media, dedicated project websites or webpages, email blasts and electronic surveys. Each of these was used by at least 75 percent of the respondents during the pandemic. Also used were project visualization tools, informational videos, and digital newsletters. Virtual public meetings were used by 92 percent of the respondents, making this the most prevalent choice. This finding is consistent with the literature review, which showed that virtual meetings were the main focus of much of the VPI guidance and peer-sharing materials developed early in the pandemic.

16    Figure 1: – VPI Tools used during pandemic and expect to use post-pandemic   These responses also were analyzed by agency type (state DOT, MPO, or RPO). For all agency types, virtual meetings were the tool most reported. The greatest variation was seen in informational videos: 81 percent of DOT respondents reported using informational videos, compared to 56 percent of MPO respondents and just 28 percent from RPOs. RPO respondents also reported lower use of websites and social media than state DOTs. 14 12 28 34 35 46 46 43 57 62 119 117 126 147 150 150 162 162 12 13 23 33 42 43 47 50 57 60 123 124 127 150 158 163 165 181 0 20 40 60 80 100 120 140 160 180 200 Other Drive‐In Meetings Virtual Reality Crowdsource Tools Telephone Town Halls Comprehensive VPI Platforms Text Messaging Project Telephone Hotlines Mobile Applications Online Collaboration Tools Digital Newsletter Informational Videos Project Visualization Tools Electronic Surveys Email Blasts Dedicated Project Websites or Webpages Social Media Virtual Public Meetings Used During Pandemic Expect to Use After Pandemic

17    Respondents also were asked about their experience hosting hybrid events during the pandemic, defined as an event that included both in-person and virtual participation. Over half of respondents (58 percent) reported having hosted such hybrid events. Expectations for future tool use The top five tools used during the pandemic remain at the top of the list of tools that agencies expect to use post-pandemic. For most tools, respondents’ expectations signal either a slight drop or no significant change in use post-pandemic. The largest expected change is for virtual meetings, with 92 percent using these during the pandemic and 82 percent expecting to use them post-pandemic. A slight drop is expected in the use of dedicated project websites, email blasts, hotlines, videos, and telephone town halls. Respondents expect to use mobile applications, social media, electronic surveys, project visualizations, digital newsletters, text messaging, drive-in meetings, and crowdsourcing at essentially the same levels as they did during the pandemic. Their responses suggest slight increases in the use of online collaboration tools, comprehensive VPI platforms and virtual reality. The survey included an open-ended question regarding a VPI tool respondents had not yet used but would like to try in the future. GIS-based story maps (Esri) received the most mentions, followed by project mobile applications. Other responses included tools covered by other survey questions such as open house websites, virtual reality, videos, and social media. Tools of interest not captured by other survey questions included podcasts and analytics software. Most useful tools Respondents were also asked to provide an open-ended response about one VPI tool the agency had found “extremely useful.” The most common responses referenced online meeting platforms, including their effectiveness in reaching diverse audiences. Features of these platforms noted as being most useful included the ability to record and post meeting videos and the ability to standardize communication with the public. Other tools mentioned as extremely useful included online survey tools and project websites. Additional tools mentioned Apart from the list of tools included in the survey, several additional tools were identified in the literature review or mentioned by focus group or interview participants. These included story maps, message boards, mobile kiosks, tablet-based outreach at pop-up events and on-demand web-based open houses (some using simulated meeting rooms to mimic the experience of an in- person open house). Another practice mentioned was supplemental streaming of virtual meetings online or via local access cable television.  

18    Many nonvirtual methods also were used to substitute for in-person engagement during the pandemic, including telephone calls to individual residents, organizational stakeholders or community leaders who could represent or serve as liaisons to their constituents; mailings; door hangers and flyer distribution; and physical displays such as kiosks with project information. Some of these supplemental “offline” methods targeted the population as a whole, while others were specifically deployed to encourage participation by underrepresented groups, as described in a later section of this report. Some agencies held outdoor events early in the pandemic. Examples include “drive-in” meetings, charrettes in which participants were masked and spaced apart, and outdoor pop-up events. Other agencies experimented with socially distanced indoor events. Promotional tools In addition to public engagement tools, survey respondents were asked to identify the most effective resources to promote and spread the word about upcoming VPI events during the pandemic. As shown in Figure 3, social media was cited as the top method (68 percent of respondents), with the most used social media platform being Facebook (79 percent), followed by Twitter (65 percent) and YouTube (53 percent). Organization webpages or dedicated project websites and email blasts also were selected by more than 50 percent of respondents. “Offline” promotional tools made up the next four most effective resources, including partner networks (38 percent), mailed newsletters (33 percent), print newspapers (32 percent) and community leader networks (30 percent). An open-ended “other” category yielded additional responses, including making announcements about an upcoming VPI event during other meetings. Other promotional tools mentioned in the literature review included geotargeting via social media advertisements and the Nextdoor social media platform. One survey respondent noted that “paid social media took off during the pandemic, using animated graphics to pull people in.” Several focus group participants took advantage of social media management platforms such as Sprout Social to monitor the online conversation, schedule posts and capture analytics.   Figure 2: A socially distanced meeting for Utah DOT’s Little Cottonwood Canyon (EIS). Credit: Utah DOT

19    Figure 3 – Most effective resources to promote VPI events       3 6 7 12 17 23 24 29 29 32 35 36 41 58 63 65 74 116 122 133 0 20 40 60 80 100 120 140 Press Release‐2% Public Access TV‐3% Ads on Local Public Transit‐4% Blogs‐6% Text Messaging‐8% Attend Community Events‐12% Phone Calls‐12% Educational Project Videos‐15% Portable Message Boards/Signs‐15% Local Radio‐16% QR Codes‐18% Printed Flyers and/or Door Hangers‐18% Digital News Sources‐21% Trusted Community Leaders‐30% Print Newspapers‐32% Mailed Newsletters or Postcards‐33% Networks of Partner Agencies/Organizations‐38% Email Blasts‐59% Organization Webpages or Dedicated Project Website‐62% Social Media‐68% Survey Responses

20              Postcard  promotion for  Hawaii DOT  forum includes  call‐in number  for those  without  internet service  (Credit: Hawaii  DOT and  Victoria Ward,  Ltd.)       Trends in the application of virtual tools In addition to increased use of VPI tools, the pandemic spurred new ways of using these tools. Of particular note are trends in the use of project videos, websites, enhanced virtual meeting formats and comprehensive engagement platforms. Project Videos Many agencies developed short prerecorded informational videos as a substitute for the type of information typically provided at in-person meetings, then publicized the availability of the videos and a way to comment online. For example, staff at North Carolina DOT found videos to be one of the most effective strategies for reaching a broad audience during the pandemic. Another trend was the development of a series of short topical videos. In one example, practitioners created topical videos prior to the release of an environmental document for a complex project. The videos were made available in multiple languages to help inform the public before a virtual public meeting. The use of prerecorded presentations has become an official policy at some agencies. For example, if project managers choose to hold an in-person meeting at Iowa DOT, they are also required to have a recorded presentation that goes live at noon on the day of the meeting. People Figure 4: Postcard from Hawaii DOT

21    interested in watching the recording are asked to sign in. The favorability rating is a feature of the Public Involvement Management Application, or PIMA, tool used by Iowa DOT, as described here: “And anyone can watch that meeting. We ask people to sign in; they can make comments throughout the PowerPoint; they can start and stop; they can choose their favorability if they don’t like the project [or] if they’re leaning in favor; and we can track that throughout the entire life of the project. So, we can see how people are engaging with us.” Websites While dedicated project websites have been in widespread use for many years, they took on a new urgency during the pandemic. One focus group participant observed that websites “have gone from something that was somewhat perfunctory to what is now a huge tool to engage large numbers of people.” Another participant noted a trend toward “simpler websites where people can get basic information quickly, versus more content dense websites. Prior to COVID-19 there was a lot of interest in ‘How can we get people to spend more time on our website and engage with all of the material?’ And now we’re seeing more ‘How can we make this simpler and shorter so people can engage and then go about their business?’” The pandemic also saw increased experimentation with websites designed to encourage asynchronous virtual engagement, including online open houses in varied formats. Beyond basic information such as text, images, slide presentations and survey questions, some online open houses include videos, interactive maps, comment and collaboration tools such as an idea wall, visualizations, and game-style learning opportunities. The emergence of immersive, virtual reality meeting rooms was a notable new format for online open houses during the pandemic. However, many agencies created online open houses in a more traditional website format. For example, early in the pandemic, the Corvallis (Oregon) Area MPO created a bilingual online open house with an interactive project map and comment form to provide virtual engagement for their Transportation Improvement Program. The virtual format helped engage more people throughout the region than in-person open houses had in the past. Asynchronous and on-demand engagement Study participants noted that asynchronous virtual engagement provides self-guided opportunities for participants to learn at their own pace and at times convenient for them. In addition, several survey respondents found that it helped increase the quality and quantity of feedback, with comments reflecting a greater understanding of the material. The literature review identified a variety of tips from practitioners for effective use of on- demand open houses. These include using a reading level checking tool to ensure that materials are presented at an appropriate level for the public; tracking where people spend time on the site; closing the open house when appropriate; archiving the information and circling back to tell people how their feedback was used. To provide direct interaction with project teams, online open houses can be coupled with scheduled Q&A sessions announced on the website. For an inclusive online open house, recommendations include designing it for usability by people with disabilities; using plain language, offering simple feedback options, and providing language alternatives; designing content that is scalable to small screens (phones); and pre-testing open house websites with people unfamiliar with the material.

22    Enhanced virtual meetings The pandemic allowed agencies to develop virtual meeting facilitation skills and experiment with real-time polling and more advanced features, including virtual breakout rooms and collaborative tools, such as whiteboards and virtual sticky notes. Different ways of integrating language translation and accommodating persons with disabilities were developed as meeting platforms evolved. For example, study participants noted the use of Zoom breakout rooms to hold informal conversations or to provide access to information in different languages. Many practitioners identified polling as a best practice since it provided interactivity and the ability to “see in real time what other people are thinking.” This was particularly important when platform settings did not allow for open dialogue in the chat. While most meeting platforms provide built-in polling capability, some study participants prefer the greater flexibility of external tools that provide variety in the question and response types and are more visually interesting. Several practitioners found success in using collaboration tools such as virtual whiteboards with small stakeholder groups but expressed reservations about using such tools in large public meetings due to the challenge of teaching a large group how to use them. Another important trend was the practice of providing multiple ways to view and participate in a virtual meeting by streaming the meeting on YouTube or Facebook Live in addition to the primary virtual meeting platform used. For example, the Great Lakes ADA Center shared that the center uses the Zoom feature that enables simultaneous event streaming to YouTube, as some people are more comfortable/familiar with YouTube. A similar option is available for streaming from Zoom to Facebook simultaneously. By connecting to Facebook and YouTube via the Zoom platform, the accessibility features remain intact, which was important to the ADA Center. Agencies also used telephone town halls and television or radio broadcasts to provide similar information and feedback opportunities to a broader public, including rural dwellers and others with limited broadband service. Moreover, simulcasts enabled agencies to provide meeting information in multiple languages. The literature review, focus groups and interviews yielded numerous tips for successfully conducting virtual meetings. Examples include developing a detailed “run of show” to help manage the many moving parts of a virtual meeting and scheduling adequate time for preparation and practice. Other suggestions include building in redundancy, with alternate hosts and facilitators ready to intervene in case the primary host loses their internet connection. The need to adapt presentations and facilitation methods to accommodate call-in participants also was mentioned. Many tips specifically address the engagement of underrepresented groups in virtual meetings, as described in a later section of this report. Comprehensive engagement platforms The pandemic saw the evolution and increased use of several comprehensive engagement platforms that combine engagement tools, contact data and comment tracking, among other features. Some of these “all-in-one” platforms can incorporate area demographic data and mapping to help plan and monitor participation and include language translation capabilities and comprehensive analytics. A unique example is the Reach tool, a mobile web application pioneered by Iowa DOT and other states, that sends personalized group texts about any of the agency’s projects and allows recipients to make comments and sign into virtual meetings directly from their phones.

23    Factors affecting tool selection The agency survey asked respondents what factors had most influenced their selection of VPI tools, with the ability to select all that applied from a list of 11 possible factors. The most common response was tool functionality, followed by the audience the agency was seeking to engage and the agency’s familiarity with the tool. Other common factors were the cost of the tool or platform, the anticipated number of participants the tool could reach and the project’s scope or complexity. Less common factors included the project geography, funding source, timeframe, and data protection. Figure 5 shows the number and percentage of responses for each influencing factor.   Figure 5 – Factors influencing VPI usage   In discussing tool selection, a focus group participant stressed that agencies need to be clear about why they are engaging people, then set the tools around that goal: “Are they trying to get concrete information? Then a survey is fine. Are they trying to build consensus among stakeholders? Then you need some interactive workshops, and doing that virtually is very, very difficult.” Other tool selection considerations that emerged in the literature review, focus groups and interviews included:  Ease of use for the public  Eliminating potential barriers to participation, such as the need to download software or register for a meeting in advance  Allowing anonymity and privacy for participants 19 30 42 43 54 81 90 99 104 116 123 0 20 40 60 80 100 120 140 Protection and Ownership of Data‐10% Project Funding Source‐15% Event Timing (e.g. Morning, Day, Evening)‐21% Project Timeframe‐22% Project Geography (e.g. Urban, Rural)‐28% Project Scope and/or Complexity‐41% Anticipated Number of Participants‐46% Tool/Platform Cost‐51% Familiarity With Tool‐53% Audience(s) Seeking to Engage‐59% Functionality of Tool‐63% Survey Responses

24     The ability to manage participation  Security (As an example, concerns about disruptive intrusion by unwanted guests affected certain agencies’ willingness to use some virtual meeting platforms.)  Ensuring accessibility for screen readers and other accessibility-related needs  Access to customer support for technical assistance Also mentioned was the need for clear agency policies on whether in-person, virtual or hybrid meetings were to be used at different stages of the pandemic. Project managers had more discretion to make this decision in some states than in others, and some practitioners felt a need for more guidance. Focus group participants also shared their thoughts on the choice between virtual and in-person meetings and different ways of combining them as hybrid engagement expands, as discussed in the “Looking Ahead” section of this report. Practitioners repeatedly stressed that just as with in-person methods, there is no one-size-fits- all VPI tool. Tool selection and implementation should be tailored to the needs of the community and the purpose of the engagement, and multiple engagement options should be offered. One participant urged others to think about tool selection as a holistic process: “People should not feel pressure to use a tool because it’s what everyone else is using or it’s what the agency has paid a subscription for…Look at how you might address the deficiencies in one tool by complementing it with another.”   Section 4: Accommodating populations with specific needs The survey, focus groups and interviews all included questions and discussions about using virtual public involvement to engage vulnerable populations. For the purposes of this study, vulnerable populations include, but are not limited to, older adults, persons with disabilities, minority populations, veterans, low-income persons, and Limited English Proficient (LEP) populations. According to the literature review/practice scan findings, some see the advent of VPI as a solution to increase engagement among vulnerable groups, facilitating more convenient participation and requiring less time than attending an in-person meeting. However, others see VPI potentially excluding marginalized voices due to the need for reliable internet service and digital skills. Guidance from the practice scan also points to the need to carefully plan an approach to identifying, inviting, and including vulnerable populations in the outreach and engagement process. It is important to plan early in the project to understand who needs to be included, where these populations are located and what some specific barriers to participation might be. Practitioners suggest creating community profiles to understand who you are reaching out to, virtually or in-person, then deciding on the right tools and the comprehensive use of different languages and translation as needed.

25    Removing barriers and increasing overall participation When survey respondents were asked whether they thought VPI tools had contributed to engagement among vulnerable/underrepresented groups, 68 percent reported they strongly agreed or somewhat agreed, while only 5 percent somewhat or strongly disagreed. Many research participants from the survey and focus groups noted that virtual public involvement removed many barriers to participation that previously existed for these groups, including lack of transportation, childcare, and time and schedule constraints. With virtual meetings, participants can more easily multitask, such as tending to responsibilities at home while also participating in virtual events. With asynchronous VPI, people could participate at times convenient to them, making participation more accessible for those with non-traditional work hours. It also was noted that VPI helped create a more equitable space for discussion, with online participation providing a comfort level to people who might not be comfortable expressing opinions in person for various reasons. This finding was consistent with interview participants from nongovernmental organizations directly serving these vulnerable groups. These organizations, including AARP California, National Federation for the Blind, The Great Lakes ADA Center, and Adult Education Solutions Cincinnati, stated that attendance at virtual events/workshops exceeded in-person versions of the same events before COVID-19. Some specific findings follow:  Adult Education Solutions reported that virtual learning allowed them to serve more students and improved overall attendance and participation due to increased flexibility and reduced need to travel.  The Great Lakes ADA Center noted that VPI provided an opportunity for in-home engagement for persons who may typically require a Personal Care Attendant (PCA) if they need to travel.  National Federation of the Blind stated that virtual platforms increased attendance significantly at national and chapter meetings. For example, before COVID-19, they typically had about 3,000 participants at the national convention. However, during COVID-19, participation rose to approximately 10,000. They also increased the total number of meetings offered because of virtual engagement, from 700 meetings per month to an average of about 2,000 meetings per month.  AARP California found that participation at events increased dramatically due to virtual engagement, especially for advocacy work. They reported larger turnouts for their “lobby days,” doubling attendance by legislators and volunteers. They also increased participation at local city council meetings and other official meetings. Before COVID-19, it was difficult to get participation from older adults at these events due to issues including scheduling, transportation and discomfort with driving at night. Virtual meetings removed all of these obstacles.  The Adult Education Center of Cincinnati found that virtual learning allowed them to serve more students, increasing participation and improving attendance.

26    Lack of technology and digital divide While some agencies found success engaging low-income and other vulnerable groups with virtual outreach and participation options, other agencies reported difficulty, especially for those lacking reliable internet connectivity. The literature review showed that the role of internet access in enabling or limiting VPI participation is complex. The digital divide goes beyond the availability of broadband service to include digital literacy, members of a household sharing computers and devices, bandwidth quality, and lack of private workspace for remote school, work, and other activities. More than three-quarters of survey respondents reported that the digital divide has posed a very or somewhat substantial barrier to engaging vulnerable populations during the pandemic. The most cited barriers were lack of online access (internet, wi-fi), lack of access to other technology (computers, smartphones, tablets, etc.) and technical difficulties. In particular, more than 60 percent of RPOs cited lack of internet and lack of access to other technology as challenges. When survey participants were asked to identify groups most difficult to engage, “people without access to technology” (39 percent) was the most frequent answer. Figure 6: Groups most difficult to engage virtually   Focus group participants also noted that the lack of high-speed internet and quality internet connection limited participation from some. They noted difficulty reaching populations that did not have the technical skills or infrastructure to participate virtually. There were also instances where participants did not have adequate internet speed to access materials effectively. For example, in rural areas with low broadband speeds, it is difficult to stream videos and load large files that are posted online (such as mapping platforms). Some participants may only have online

27    access via cell phones, with small screens that make it difficult to see certain displays and detailed graphics. The ADA Center noted that many low-income customers they serve might have unlimited minutes on cell phones but not unlimited data plans and accessing large files and data-heavy websites/meetings could be costly. They also recounted that to better serve populations with low internet speeds, they requested all speakers and panels to turn their cameras off (captioning was offered for the those who are deaf or hard of hearing). Others noted that older adult populations and those rural areas lacking internet and cell service were particularly hard to reach through virtual engagement. Some agencies also have populations in their region/state that do not use electronic technology for religious reasons. Strategies that worked to increase participation among the vulnerable The literature review identified various strategies practitioners use to increase participation and overcome the digital divide. These include one-on-one assistance; the use of telephone technology; on-demand engagement that can be done at any time; the use of other low-tech options, such as paper surveys/materials; and partnerships with local organizations. When survey respondents were asked what strategies they had used to increase public participation among vulnerable/underrepresented populations during the pandemic, the three most cited were posting meeting materials and recordings online; sharing project information via traditional outlets such as mail, newspapers, and radio; and promoting project information and events utilizing social media advertising. Partnering with local community networks or leaders also was frequently noted, with the most common partners being local nonprofits, advocacy groups, community centers and schools. According to the survey, the most common engagement strategies to increase participation among vulnerable groups varied by agency type. Relative to other agencies, respondents from RPOs were most likely to partner with local community networks (52 percent), while respondents from DOTs were most likely to share information via traditional outlets like mail and radio (57 percent). Respondents from MPOs were most likely to use various strategies, notably posting meeting recordings online (81 percent) and using social media advertising (56 percent). One-on-one training and other technical assistance According to findings in the literature review, one-on-one training or assisted participation is another strategy that social service providers have used to encourage participation in virtual activities by vulnerable individuals. This finding was consistent with insights from focus group participants and interviews with nongovernmental organizations and was particularly true for older adult participants and people with certain disabilities. Focus group participants noted the challenges many older adults faced with the new virtual engagement medium. Some participants found that older adults needed smaller, more intimate discussions, especially older adults who also had some disability, such as those who are deaf, hard of hearing, blind, or have low vision. Small breakout groups (five to six attendees) were helpful for these individuals, particularly when explaining complicated or technical aspects of projects. In addition, several participants described offering some form of one-on-one or on-call assistance for those unfamiliar with online platforms, such as providing “practice”

28    sessions/training or individual troubleshooting for technical issues as they arose. One participant included information about these one-on-one sessions on the meeting invitation for any resident that needed an orientation on how to use the online platform. Another offered residents an opportunity to sign up to reserve time with a project manager for a one-on-one phone conversation about a project, while one participant sent mailings to older adults at senior facilities/housing complexes to alert them of upcoming engagement opportunities. This same participant also arranged with property managers to pass out information directly to the residents. Interview participants shared similar experiences. During the discussion with AARP, it was noted that most of their volunteers (all of whom are older adults) adapted quickly to virtual platforms. Still, about 30 percent had difficulty with the transition. Their primary tool for addressing these challenges was one-on-one assistance and direct support. For example, Goodwill International provided training to Goodwill Ambassadors, who then trained clients on how to access the internet, use mobile devices, etc. They also partnered with the U.S. Department of Labor to offer hundreds of training sessions to groups with low technical skills, including older adults and low-income residents. Access to technology Some research participants found ways to provide technology to residents and service customers to reduce the digital divide. For example, Cincinnati Adult Education Center worked with Cincinnati Bell and other local providers to offer students free wi-fi hot spots for class and schoolwork. They also developed a technology loaner program whereby they purchased laptops and allowed students to borrow them, although it was noted that there were issues with trust and logistics in distribution. Goodwill International partnered with Comcast and Verizon to offer free internet programs and distribute free devices to customers and employees. The Goodwill interviewee noted that connectivity is an essential life skill now. They have added access to internet and technical skills to their list of basic needs, such as food, housing, and transportation. One survey participant partnered with a local library to provide computer access for constituents to participate in virtual engagement activities. On-demand and “office hours” Several focus group participants recounted the use of “office hours” engagement whereby staff would be available in a Zoom room or by phone during certain posted hours, with some noting regularly scheduled occurrences and others as a one-off for a project. These flexible times were helpful for residents who did not want to or could not commit to a longer engagement or meeting but had specific questions or wanted/needed a more personalized experience. As described earlier in the “Tools” section of this report, many agencies also used websites or other platforms to create an on-demand virtual engagement experience. These on-demand events provide residents the utmost flexibility in terms of participation, allowing them to navigate materials at their own pace and at a time that is most convenient for them. However, others noted that on- demand engagement does not provide intangibles, such as building trust and creating a collaborative environment to solve problems. Virtual meeting accommodations According to the virtual engagement experiences of the Great Lakes ADA Center during the pandemic, VPI has increased the frequency and amount of engagement among persons with

29    disabilities but only when the events and platforms were made accessible. The Great Lakes ADA Center advised that the best strategy is to “plan and make it [events] accessible from the beginning because you never know who your audience is going to be.” The National Federation for the Blind noted that for those who are blind or have low vision, being at home with their own computer and assistive technology made participation in events much easier. Interview, focus group and survey participants all described other important considerations and tactics for ensuring equity and accessibility for people with disabilities to participate, especially in virtual meetings. Among the considerations were the following:  Zoom was found by the ADA Center to be the most accessible virtual meeting platform. As explained in the interview, “from audio quality, from camera quality, from accessibility for keyboarding, interface for people who use text to speech or people who use screen readers…, Zoom is really the most current, commercially available, high-volume platform.”  Though some participants who are deaf or hard of hearing prefer to read lips of speakers on camera, if that is not an option due to bandwidth, closed captioning is a viable option. However, captioning on most popular platforms is limited to only one to two lines of text. This can be difficult to follow. Some services can be added to virtual meeting platforms to allow scrolling functionality and improve the overall experience for these users.  Agencies need to ensure that there is adequate color contrast on all materials posted online and in the website design to improve visibility.  It is good practice to send materials remediated for ADA audiences at least 24 hours in advance of meetings for review ahead of time. Those using screen readers cannot read and listen to a meeting at the same time.  Requiring persons to engage virtually via camera can be anxiety-evoking for some persons with disabilities, including mental illness, and thus offering an accommodation not requiring participation via camera is important.  Ensure that any posted recordings of virtual meetings are made ADA compliant and fully accessible. The Great Lakes ADA Center used Camtasia software to screen record the live meeting to ensure the interpreter remains visible at all times.  Screen readers will read aloud the comments in chat as they are posted, distracting screen reader users from hearing the conversation or presentation effectively. This can be confusing and disruptive for participants. The ADA Center does not recommend relying heavily on the chat function for this reason.  Screen readers can also have difficulty with screen sharing and PowerPoint presentations if the presentations are not fully ADA accessible. Agencies should provide all presentation materials, and PowerPoint slides must be made accessible before exporting and sharing as a PDF. Announce at the start of the event how to access copies of accessible materials.  When presenting during a virtual meeting, describe all images and videos for those who are blind or have low vision as well as for those joining by phone.  

30    Importance of telephone participation The telephone has continued to be an effective method of virtual engagement, especially for those with limited access to or comfort with other forms of technology. The literature review found that many practitioners recommend using a virtual meeting platform with a call-in option. Survey and focus group participants also noted that the call-in feature for Zoom and other virtual meetings was very important for participation, both for those without access to a computer or smart device and those in rural areas that are not well connected by broadband. Focus group participants described the need to adapt presentation materials to accommodate those participating by phone and not overly rely on visuals. For example, staff should describe and explain images in virtual presentations rather than just refer to them. Another idea mentioned was using a staffed telephone number for questions and comments before, during and after a virtual meeting. The ADA Center also found that those without reliable (or any) internet access could successfully engage virtually via phone. Many individuals could dial in to listen to the audio of a virtual meeting and send questions via telephone or email before staff to share live with participants. Participation by phone was also critically important for the National Federation of the Blind, whose membership includes a large percentage of people without computers or internet access. Telephone town halls (TTH) also proved to be very effective, as described by several study participants. TTH were helpful because they provided agencies with a way to target specific populations using voter registration data and could provide simultaneous translation when needed. Using TTH, residents are called directly and can opt-in on the spot to hear the presentation. This method takes the project to them instead of relying on the public to come to the project. One participant noted the importance of finding the right vendor as their experiences with vendors varied. This participant recently had a very successful TTH for a statewide Electric Vehicle Plan with almost 300 participants. Another participant conducted a TTH during the pandemic with residents from the Navajo Nation with over 100 participants, compared with 20 attendees at an in-person meeting convened for the same project before the pandemic. Combining virtual and in-person methods for vulnerable groups The Phase 1 literature review found that a clear best practice for VPI engagement is to include offline methods in the promotional campaign. These can consist of traditional outreach methods, such as mailers, door hangers, flyers, posters, and temporary signs (including digital displays, variable message signs and billboards). Also important is media outreach, including ethnic radio, TV, and newspapers. Social media is also an essential promotional tool for reaching persons of diverse races, incomes and educational levels and rural residents. Findings from the survey and focus groups are consistent with this approach. Many research participants described deploying a combination of virtual and physical outreach, including various engagement techniques that were regularly used before the pandemic. Focus group participants provided specific examples of tactics used to increase participation. These included mailing postcards, sending printed copies of materials by mail, posting flyers in public and heavily visited spaces in targeted communities, using text messages to promote an event, employing roadside display signs, distributing through community-based and local organizations and other resources to boost participation. Others also used radio, newspaper, and bus advertisements for promotion.

31    Some participants stated that these low-tech tactics yielded little improvement in engagement. For others, it made a huge difference. These tactics were important for targeting specific populations that could be impacted by a project, ensuring they received timely information about upcoming events. One participant described walking neighborhoods before each meeting, leaving door hangers and handing out flyers. A participant from Arizona DOT recounted how important mailers were for one of their statewide projects that impacted rural communities throughout the state. Mailers were an effective way to get the word out about the project and about opportunities for public participation. They could also accommodate multiple languages on the mailers, including English, Spanish and Vietnamese. Partnerships with community organizations The literature review found that many experts believe that choice of VPI tools is less important than an agency’s commitment to proactively engage community residents by working with local organizations and actively including them in the outreach process. When survey respondents were asked about strategies that they employed to increase participation from vulnerable communities, “Partner with Local Community Networks or Leaders” was one of the most popular answer choices. Forty-eight percent of Metropolitan Planning Organizations and Rural Planning Organizations, and 42 percent of state DOTs that answered this question selected this response. Many focus group participants also noted the importance of working directly with community-based nonprofit organizations and local community leaders and activists, even when conducting virtual engagement. These organizations and individuals serve as a “trusted advocate” for the community and are helpful in recruiting participants, cohosting virtual events, and making community members feel comfortable and willing to share and participate. When asked about the types of community organizations their agencies partnered with during COVID-19 to engage with vulnerable and underrepresented populations, the top survey answer was local or regional nonprofits (41 percent of respondents), followed by advocacy groups and community centers (both at 35 percent). Schools (34 percent) and libraries (31 percent) closely followed the top three responses. Less frequent responses included places of worship, senior centers, local food banks and assisted living facilities. Other open-ended responses included homeownership associations, Chambers of Commerce, and healthcare networks. Many agencies reported success in partnering with these community organizations in a variety of ways, including:  Organizations distributed materials to community members and shared information on social media to raise awareness about opportunities for participation.  Some organizations convened groups to participate in a virtual meeting in a central location. This was especially helpful for groups that lacked access to technology at home. One organization held a hybrid neighborhood meeting at a church that typically broadcasts its religious services. Church staff handled the broadcast technology for the community meeting, and residents easily participated because they were familiar with how to access the streaming services.  Some of these organizations also assisted in supporting access to technology for participants. For example, one agency partnered with a library to provide computer access for constituents.

32     Some agencies partnered with community organizations to hold targeted virtual listening sessions for specific groups who might not be comfortable in larger virtual meetings, such as LEP/non-English speakers, people of color, people living with disabilities, people with lower incomes and youth.  Community organizations can assist in fact-checking and “ground-truthing” information. One focus group participant partnered with nonprofit organizations representing vulnerable groups to improve the quality of the data collected for use in project maps so they more accurately reflected the experiences and perspectives of community group members. Community organizations also can help host in-person meetings or events and did so even during the COVID-19 pandemic when virtual engagement alone was not sufficient to facilitate participation from community members. For example, one participant described a project that was to include a virtual charrette in a predominately Black community with low internet access. Based on discussions with local community organizers, who cautioned against a virtual-only approach, the project team also held a masked and socially distanced event in City Hall. As the community organizers expected, almost no one attended the concurrent virtual event. For another contentious project, a participant recounted asking leaders from an environmental justice community if they could act as a bridge to residents. They recommended holding an outdoor meeting (and providing dinner to attendees) before the planned virtual meeting to allow for informal discussion with the community. Following the virtual meeting, they also distributed a mailer specific to this community with a postage-paid card so residents could provide comments via mail. AARP recounted working with a small nonprofit in the St. Gabriel Valley Chinese Community in Southern California to plan a community outreach event in Mandarin to share information about AARP services and programs. In describing the meeting, the interview participant used the term “transcreation” instead of translation, noting that being truly inclusive requires creating an experience unique to the community on their terms, not merely providing language translation. A few focus group participants from regional planning organizations also described formal programs they had implemented to recruit and retain organizations and individuals for ongoing engagement assistance. In some cases, participants and organizations are financially compensated for their time and expertise. Conversely, some DOT participants stated that they would like to incent and support participation from organizations in underserved and vulnerable communities, but current federal regulation do not permit this. They described this limitation as a significant barrier to working with community organizations to assist with outreach. Engaging rural populations Engaging rural populations, including some religious communities and native tribes, posed particular challenges. Both survey and focus group participants recounted specific strategies and engagement techniques to reach these residents. For communities with limited use of technology (e.g., certain religious groups), some agencies provided printed copies of materials and distributed them at local stores. One participant from Pennsylvania DOT noted that local stores sometimes have internet access even though individual community members do not. This can serve as a place where a group could gather to join a virtual meeting. Participants also recounted approaches to engaging with Native American tribes, including sending materials by mail (surveys, USB drives, etc.), hosting customized virtual, hybrid or in-person meetings and working with tribal leaders to encourage participation at larger virtual events.

33    Others noted that in more rural areas, virtual meetings allowed a broader range of perspectives from geographic regions to interact. For example, one participant described hosting a virtual charrette in Missoula, Montana, that engaged native tribes several hours north. This result would have been impossible to achieve if they had hosted only an in-person meeting. With in-person meetings, agencies would have had separate meetings in each community, but virtual meetings allowed more mingling and an exchange of perspectives from remote locations. Several focus group participants saw this as a huge advantage of virtual meetings. LEP residents and accommodating multiple languages Focus group participants noted significant challenges in conducting VPI in multiple languages, specifically during virtual meetings. Participants noted a need for additional training on accommodating multiple languages and ensuring an equitable experience for all participants. They also recounted some examples of tactics that had been successful. One participant developed project videos in multiple languages on various topics in advance of a public meeting, stating that these videos were effective for non-English speakers because they could be paused or repeated to facilitate understanding of technical concepts. Another participant used the closed captioning tool in their virtual meeting platform, which instantly translates discussions into another language and provides real-time captioning. This was more cost-effective than hiring an interpreter; another participant noted that the interpreter they had hired for a meeting never was used. One focus group member noted that YouTube can accommodate language translation with captioning via its artificial intelligence (AI) platform. She also mentioned offering various language-enabled phone lines, adding that if they use a language translator for a virtual meeting, they give persons speaking that language a separate meeting link to connect them to the translator. Other participants mentioned using Spanish-language simulcasts and Facebook Live in partnership with local organizations to broaden participation from Spanish-speaking communities or holding back-to-back meetings virtually in different languages. Interview participants also discussed engagement with LEP residents. La Raza provides legal counsel to immigrant communities, many of whom are LEP. They have found that most clients have cell phones, but very few have computers or laptops, so most communication during COVID-19 took place via phone calls and personal cell phones. Fortunately, because of the pandemic, staff were licensed to give legal advice over the phone. However, they maintained that it was still necessary to meet face-to-face with clients while maintaining safety controls and social distancing, especially when dealing with highly sensitive issues, such as litigation. In addition, AARP described a desire to conduct virtual events and volunteer training in multiple languages simultaneously (Spanish and Chinese were mentioned), but they do not currently have the capacity to do it. Challenge of collecting demographic information One of the key challenges noted by several participants is difficulty in identifying who is participating in a virtual meeting and whether the targeted vulnerable population(s) is being served. There is no easy way to collect demographic information or require participants to provide this information. It is also important to track attendees to understand who is using the virtual tools and when to include alternative low-tech options. One participant indicated that they

34    collect geographic information for their online surveys by looking at IP addresses, although they cannot collect racial, ethnic or income information. Another participant recounted a scenario where extensive effort was made to reach certain communities for a survey. They received 5,000 responses, but because the effort was almost entirely virtual, the lack of in-person engagement became a political issue. There was no way to demonstrate that lower-income residents participated. One participant relied on hybrid meetings with community-based organizations and churches hosting to compensate for this challenge. These hybrid events provided a better sense of the demographic makeup of participants and whether the team was reaching the targeted individuals. Another participant used a professional focus group firm to recruit residents from targeted demographic groups to ensure participation among certain environmental justice communities in the outreach process. Section 5: Looking ahead: future of virtual and hybrid engagement Participant thoughts on the future of virtual engagement In a similar way that authorities say there is currently no end in sight for the COVID-19 pandemic, there is also no end in sight for virtual public involvement. Based on the research this study has conducted, even if an official end of COVID-19 is declared, it is evident that VPI will continue to be used to engage the public. In fact, 69 percent of the survey participants reported they were extremely likely to use VPI tools moving forward, and an additional 24 percent said they were somewhat likely to use VPI in the future. Only two participants said they were somewhat unlikely to use VPI, and no one responded that they were extremely unlikely to use it. Further, for most VPI tools, survey respondents’ expectations signal either a slight drop or no significant change in use post- pandemic. The tools most likely to be used post-pandemic include virtual public meetings, digital newsletters and email blasts, electronic surveys, social media, project visualization tools such as interactive maps and 3D renderings and a dedicated web presence for projects. Many focus group participants noted that virtual attendance has become an expected method of engagement, both from agencies planning their outreach and the public providing their input. As one put it, “Across the board, it’s become a given that if we’re doing in-person [engagement], we’re also doing virtual now, too.” How hybrid engagement has been implemented As it has become safer to gather in groups, participants noted that agencies are once again hosting in-person meetings. With virtual engagement continuing, a hybrid approach is now becoming standard, and participants felt that hybrid meetings, in particular, would become a regular occurrence as they provide convenient options for stakeholders and the public. “… there are benefits and limitations to both in-person and virtual. In a virtual meeting, you don’t have the opportunity for mingling, to grab a cup of coffee before the event, where you’re able to do some data gathering and/or influencing or just relationship-building. It’s harder in a virtual setting – you can build it in, it’s just harder and not as natural. Likely need both virtual and in-person going forward.”

35    Most study participants indicated that they have already begun conducting hybrid meetings, defined as an event that includes both in-person and virtual participation. A majority of survey respondents said they had hosted hybrid events during the pandemic, with the agencies representing rural populations implementing hybrid meetings at a higher rate (as those areas were more likely to conduct in-person meetings during the pandemic). Hybrid meetings can take different forms, with some agencies conducting an online meeting simultaneously with an in- person event, some hosting separate live virtual meetings using online meeting platforms, and some implementing in-person meetings and creating on-demand/self-guided versions of materials by posting presentations and other resources online. Challenges and considerations Hybrid engagement presents challenges for practitioners, starting with whether hybrid meetings are legal under state open meeting laws. While states may amend their open meeting laws and acts to allow hybrid meetings, in the short term practitioners in some areas are very purposeful in their use and are cautious in fully adopting the practice. Additionally, consultants in the focus groups noted it was important to have guidance or a policy from agencies on whether projects could, or should, do in-person, virtual, or hybrid engagement. Participants noted several considerations to determine which version of a hybrid meeting is appropriate. Practitioners should consider their audience and their objective as a first step. If the goal is education, on-demand, self-guided information would be suitable. However, building trust and long-term relationships with the community and working through solutions are better done via an in-person component. Additionally, participants noted it is important to consider the subject matter and how easy it would be for someone to navigate individually without a facilitator. If people can most likely comprehend the information to be communicated without assistance, an on-demand approach is acceptable. Ultimately, having multiple options available to the public was considered a best practice, allowing people to go online at a convenient time or attend a meeting – in person or virtually – where they can talk to a project official and get questions answered. To the extent possible, it is important to create consistency between the virtual and in-person participation options, ensuring that participants have the same access to information and opportunities to participate. There are also staffing considerations for hybrid meetings. Many participants noted that facilitating a virtual meeting significantly differs from facilitating an in-person meeting, requiring different skills and preparation. The same is true for hybrid meetings that bring in a virtual component. Survey respondents who had held hybrid events reported that technological issues were the greatest challenge when hosting the event. Focus group participants who had hosted hybrid meetings stressed that the person facilitating the in-person portion of the meeting should not also be required to operate the technology for the meeting. They also noted that it was challenging for facilitators to make sure everyone (virtual and in-person) felt like they were “in” the meeting. Most agencies indicated they had a dedicated team for each type of engagement. Further, the format and audience of the meeting may impact whether it is reasonable to host the in-person and virtual components at the same time. Hybrid meetings can be a seamless experience for smaller stakeholder meetings where it is vital to have everyone in the same meeting but not everyone is available to travel to the in-person

36    meeting. However, these hybrid meetings require high-quality audio in the conference room and benefit from a designated staff person to ensure the technology functions properly. Alternatively, for large public meetings or complex meetings with many moving pieces, participants noted it was easier and more practical to separate the virtual from the in-person, instead of having them at the same time. As one observed, “Doing them as an actual hybrid meeting can be pretty tricky. We found that it is not preferred – there are so many things that can go wrong, and you have to have almost double the personnel to handle it. We prefer to do them separately, even if they might be the same night, but treating them as two separate events.” For either of these formats, participants encouraged releasing information virtually ahead of any live meetings. This would provide time for the public to review materials and to attend meetings prepared with specific questions. It was also noted that materials remediated for ADA should be online no later than 24 hours before meetings for independent viewing as those stakeholders cannot use screen readers and listen to the meeting simultaneously. Public involvement professionals throughout the study noted that posting materials online for on- demand engagement is a best practice and a baseline strategy to build upon. This approach allows the public to engage with the content or watch the meetings at their own pace and at a convenient time. However, with in-person engagement returning, public involvement practitioners are now challenged with ensuring project teams do not simply revert to in-person- only engagement. In fact, some agencies now require a hybrid model; for example, if a project chooses to do an in-person meeting, they also must post a recorded presentation that goes live online the day of the meeting. Several participants noted that the ability to record a live virtual meeting and post it online afterward makes it easy for those who missed the meeting to access the same content as those who attended, and it can increase participation with minimal effort from the project team. Posting recordings provides a “two for the price of one” option for agencies. As one participant described, “You could have 100 people at a meeting and hundreds more viewing the recording within the week or two following that meeting. Almost like having another meeting at little or no cost.” It was also noted there is a danger while hosting separate hybrid meetings that there are different audiences at each event. If these audiences do not see or hear each other, it can give participants the impression that people are being left out of the process, that the agency is not listening, and, importantly, these audiences are not afforded the opportunity to hear and learn from one another. Beyond hybrid meetings, agencies are exploring how they can translate other in-person experiences to an online, on-demand experience. For example, one participant indicated that their agency would continue offering site “visits” using drone technology and in-person tours. Section 6: Conclusion The Phase 2 of this research will develop a guide on best practices for virtual public engagement for transportation agencies. Focus group participants requested best practice guidance on several topics: preliminary planning, tool selection, virtual meeting logistics, hybrid meeting facilitation, accessibility accommodations and equitable approaches. Resources that provide guidance on these topics, as identified by focus group participants and the literature review, tend to focus only on specific elements of VPI such as virtual meetings, rather than covering all aspects of virtual engagement, likely due to the extensive and complex considerations for successful VPI deployment.