Command-Level Decision Making for Transit Emergency Managers (2013)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

CHAPTER 3: FINDINGS Phase I Findings 3.1 Training Needs Analysis The interview data and literature review revealed the individual knowledge and skills critical for effective transit command-level decision making in rapidly developing emergency incidents, and differentiated the critical tasks and skills required for each role. From the data collected, we identified five concentration areas in ground transit emergency response (see Table 1). Additionally, the data revealed four primary command-level roles relevant across all five transit concentration areas. These primary roles mirror ICS operations center roles and include the Transit Emergency Manager who is responsible for the overall strategic command of the emergency response effort, the Transit Operations Coordinator who is responsible for managing the tactical functions such as coordination with field operators, etc., the Transit Planning Coordinator who is responsible for the collection, evaluation, forecasting, dissemination, and use of the information about the emergency incident and status of resources, and finally the Transit Logistics Coordinator who is responsible for managing logistical support such as personnel, vehicles, equipment, and supplies. During an incident, these roles work within the transit emergency operations center and coordinate with Emergency Support Function (ESF) 1 (Transportation) based in a local or state emergency operations center. Detailed profiles for each of these roles can be found in Appendix B of this report. Table 1: Transit Concentration Areas and Their Associated Command-Level Roles Transit Concentration Area Command-Level Role (See Appendix B for Detailed Profiles) 1. Buses (Public and Private)  Transit Emergency Manager  Transit Operations Coordinator  Transit Planning Coordinator  Transit Logistics Coordinator 2. Railways (Heavy, Light, Commuter Rails) 3. Bridges, Roads and Highways 4. Tunnels (Above and Below Ground) 5. Maritime (Cruise Ships, Ferries, etc.) In addition to identifying the concentration areas and transit command-level roles, the data also revealed five primary functions that transit agencies often must achieve during an emergency. These emergency response functions were then categorized between regular and emergency services. Moreover, these functions will help inform and drive the design and development of the learning objectives (Task 3). Table 2: Primary Task Functions of Transit Agencies with Associated Service Types Transit Emergency Response Function Regular Service Emergency Service 1. Life Safety X 2. Property Conservation X 3. Evacuate or move people “quickly and efficiently” X X 4. Move responders into and out of the areas X 5. Provide Resources (offering additional routes, increased service clearing roads, highway/roadway or waterway accessibility, etc.) X X The data also revealed three high-level cognitive processes that transit agency decision makers must perform during emergencies. A cognitive process is a series of interdependent actions executed during multi-agency response to an emergency. These actions form an evolving response pattern aimed at resolving the crisis. Through these actions, it is possible to employ, maintain, and revise plans. These processes involve activities such as leveraging previously established relationships with other transportation authorities, establishing morale, and establishing internal communication and information flows that promote effective multi-agency (public and private) transit response. As described below, there are three main cognitive processes which include: Develop Situation Awareness, Synchronize Information and Resources, and Execute Actions and Decisions. Page 7

 Develop Situation Awareness  Identify, gather, and prioritize information to understand the situation: Effective transit agency command-level decision makers are able to determine quickly how and where to identify, gather, and prioritize information necessary to take action regardless of transportation mean or mode (buses, roads, bridges, boats, etc.).  Recognize context of the situation and predict future needs: Effective transit agency command-level decision makers are able to construct a coherent picture of unfolding events and see the overarching implications and potential public needs of possible actions.  Synchronize Information and Resources  Coordinate and communicate internally and externally: Effective transit agency command- level decision makers are able to collaborate internally and with outside agencies to gain resources to achieve mission goals and objectives. For example, if the evacuation need calls for 10,000 people to be moved to a safe location and additional buses are required, then a public agency may coordinate with private transit agencies to acquired additional vehicles.  Acquire, prioritize and allocate available assets to meet the transit needs of the public: Effective transit agency command-level decision makers are able to assess and meet transit needs of the public. For instance, this function may entail calculating the number of people who need to be moved with the number of available resources.  Execute Actions and Decisions  Recognize decision points: Effective transit agency command-level decision makers are able to recognize decision points during a crisis and take action quickly. For example, a Transit Emergency Manager and/or Transit Operations Coordinator can make a decision to stop or interrupt services if presented with a critical situation or if there is a significant public need.  Maintain mission priorities: Effective transit agency command-level decision makers are able to recognize their primary mission which is to protect human life by quickly and efficiently moving people and providing resources. Finally, we also identified several prerequisite training courses that transit command-level decision makers should take prior to participating in a TERA exercise. These training courses include ICS 100, 200, 300, 400, 700, and IC-801 Transportation (see Appendix L for prerequisite online training outlines relating to each recommended course). 3.2 Learning Objectives Based on findings revealed in the Training Needs Analysis (Tasks 1 and 2), terminal and enabling learning objectives were designed for each identified role, incident phase (red=activation phase, blue=operations phase, and black=demobilization phase), and performance tasks (see Appendices D1-4 of the Phase I report). The terminal learning objectives describe the major intended outcomes expected from learners while performing tasks within TERA. To clearly and concisely communicate how learners will achieve intended outcomes, we wrote enabling learning objectives that describe precisely how the terminal objectives would be achieved. The enabling learning objectives were then categorized by learning level in accordance to Bloom’s Taxonomy (Bloom, B.S., M.D. Englehard, E.J. Furst, W.H. Hill, and D.R. Krathwhol, Taxonomy of education objectives: the classification of educational goals, Longmans, Green, New York, 1956) and then matched with an appropriate TERA system strategy or function (e.g. send an email or call using the address book). Finally, performance measures in the form of time increments were paired with the enabling learning objectives to indicate the duration learners would have to achieve a task. To adhere to sound Instructional Design practices, we also segmented the Tasks, Conditions, Standards, and Expected Actions. • Task(s) are descriptions of action(s) learners will perform. • Condition(s) are criteria for measuring how the tasks will be performed. • Standard(s) are guidelines for how the tasks should be performed. • Expected Action(s) are the anticipated task performance activity. Page 8

After writing the learning objectives, system strategies, and performance measures, we then compared them to the HSEEP Exercise and Evaluation Guidelines which specifies evaluation criteria and standards for the mirrored ICS capabilities. The comparative analysis helped us to verify the evaluation requirements needed to train and assess the learning objectives. 3.3 Scenario Timeline After creating the learning objectives, we developed a prototype scenario timeline (see Appendix E) that outlines and represents the major events and time segments. The scenario timeline also formed the general situation by presenting background information and broad event descriptions that clarifies who, what, where, why, when, and how for each simulated inject or stimulus. Simulated injects or stimulus are the driving components of TERA and will assist learners while performing tasks. For each simulated inject, we applied and documented several instructional design requirements to include:  Inject Type: What form of communication or information will the inject take? While operating in simulated environments, injects can take many communicative or information-based forms such as emails, phone calls, faxes, live conversations, video news reports, etc.  Time: At what occurrence on the timeline will inject(s) appear?  Inject Content: What information or material(s) make up the simulated inject or stimulus?  Incoming and Outgoing Recipient(s): Who will the information be conveyed to and from?  Response(s)/Feedback: What are the standards response(s) to the inject(s) and/or stimuli?  Performance Standard: What are the approximate expected actions of the learner to the task in question?  Consequences: What are the positive and negative outcomes for completing or not completing the task? 3.4 Scenario Recommendations During Phase I, we surveyed a variety of sources to find candidate emergency management exercise scenarios to be developed for TERA. Surveyed sources include the TCRP panel-recommended scenarios for this project, the National Planning Scenarios (http://www.fema.gov/pdf/media/factsheets/2009/npd_natl_plan_scenario.pdf ), transit emergency management doctrine, and existing exercise systems. We specifically included the list of scenarios implemented in the base Emergency Management Staff Trainer (EMST) system to determine if these scenarios could be expanded to include transit command-level training roles. Expanding these scenarios would provide increased value by providing both transit-specific exercises and collaborative training with other emergency agencies. The scenarios from the survey were entered into a spreadsheet and duplicates, scenarios not widely applicable to transit, and entries that were consequences rather than scenarios were marked. Commonly, emergency management scenarios are divided into Chemical, Biological, Radiological, Nuclear, Explosive (CBRNE) and Natural Disaster categories. In surveying the candidate scenarios, there were very few applicable nuclear or radiological scenarios, and the normal all-hazards categories as defined by the National Planning Scenarios did not encompass all candidates. We decided, therefore, to modify the categories to be terrorism based (CBRNE, physical attacks, hostage situations), natural disaster, or accident. Page 9

For each scenario, we noted the primary task functions for each (drawn from the Training Needs Analysis), and any differentiators for those task’s functions. For example, many scenarios required property conservation, but only some required large scale assessment of structural integrity of assets such as bridges or stations. Others required decontamination of assets. A summary of this differentiator data for each scenario is given in Table 3. Table 3: Scenarios with Task Function Differentiators Source/Scenario Category Task Function Differentiators/Notes TRB A-36 RFP Cyber attack (loss of power) Terrorist Continuity of operations Flooding Natural Disaster Provide resources, determine structural integrity of transit assets, coordination of clearing/repair to reach damaged regions Regional evacuation Duplicate Consequence; must be combined with an incident such as Hurricane or Hazmat Chemical or Biological release Terrorist Security, life safety, property conservation (decontamination), move responders in and out Multiple explosions Terrorist Security, life safety, property conservation Transit revenue vehicle collision Accident Move responders in and out, rerouting Emergency Management Staff Trainer (EMST) Earthquake Natural Disaster Determine structural integrity of transit assets, coordination of clearing/repair to reach damaged regions, move responders in and out Flood Duplicate Hurricane Landfall Natural Disaster Regional evacuation Pandemic Influenza Natural Disaster Continuity of operations, addressing public fear Terrorist attack with multiple explosions Duplicate Wildfire Natural Disaster Rerouting Tornado outbreak Natural Disaster Rerouting Severe weather at National Convention Natural Disaster Rerouting with high user capacity Civil unrest at National Convention with attacks on transit assets Terrorist Security, life safety with high user capacity National Planning Scenarios 10 Kiloton Improvised Nuclear Device Terrorist Large scale evacuation, shelter in place in stations, loss of power and services Aerosol Anthrax Terrorist Decontamination, addressing public fear Page 10

Source/Scenario Category Task Function Differentiators/Notes Pandemic Influenza Duplicate Plague Terrorist Continuity of operations, addressing public fear Blister Agent Terrorist Security, life safety, crowd control, move responders in and out, decontamination Toxic Industrial Chemicals Terrorist Evacuation, rerouting, decontamination Nerve Agent Terrorist Security, life safety, crowd control, move responders in and out, decontamination Chlorine Tank Explosion Terrorist Small scale evacuation in contaminated area Major Earthquake Duplicate Major Hurricane Duplicate Radiological Dispersal Device Terrorist Rerouting, evacuation from contaminated zone, shelter in place in stations, decontamination Bombing using Improvised Explosive Devices (IED) Duplicate Food Contamination Terrorist Terrorist based scenario as specified by NPS summary, not highly applicable to transit Foreign Animal Disease Terrorist Terrorist based scenario as specified by NPS summary, not highly applicable to transit Cyber Attack Duplicate SME Meetings Hurricane Duplicate Terrorist attack with multiple explosions Duplicate Blizzard Natural Disaster Shelter in place in station, inability for transit assets to operate Terrorist attack with suicide transit operators Terrorist Internal security, life safety Public Transportation System Security and Emergency Preparedness Planning Guide Bomb threat Terrorist Threat procedures Unusual or out of place objects Terrorist Threat procedures Chemical agent release Duplicate Vehicle Born Improvised Explosive Device (VBIED) Terrorist Security, life safety. Subset of multiple explosions scenario. Improvised explosive device in station/vehicle Terrorist Security, life safety, continuity of operations. Subset of multiple explosions scenario. Page 11

Source/Scenario Category Task Function Differentiators/Notes Armed hijacking, hostage, or barricade situation in station/vehicle Terrorist Security, life safety, continuity of operations Chemical, biological, or nuclear release in station/vehicle Terrorist Security, Life safety, decontamination. Secondary explosive device directed at emergency responders Terrorist Security, must be combined with another scenario Physical or information attack on control system Terrorist Internal security, continuity of operations Physical or information attack on dispatch system Terrorist Internal security, continuity of operations Improvised Explosive Device detonated near fuel facility Terrorist Internal security, continuity of operations Airport Emergency Response Operations Simulation (AEROS) Aircraft incident Accident Not applicable across transit agencies Bomb incident Duplicate Crowd control Duplicate Consequence; must be combined with an incident such as Blizzard or Hazmat Hazmat incident Duplicate Severe storm Duplicate Power failure Accident Continuity of operations Sabotage Duplicate Structural fire Accident Continuity of operations Water rescue Natural Disaster Primarily on-scene activities Fuel farm fire Accident Continuity of operations To determine scenario priority, we considered the following criteria: 1. Each scenario’s applicability to transit emergency management. 2. Each scenario’s applicability to transit emergency operations center (command and control level) mitigation activities as opposed to primarily on-scene activities. 3. Each scenario’s applicability to Tier One and Tier Two transit cities. 4. Each scenario’s applicability for all specified exercise participant roles. 5. Each scenario’s likelihood of occurring. 6. Ability to develop engaging storyline with multimedia injects. Two additional criteria describe diversification considerations that were taken into account when group scenario topics. 7. Ability to exercise a diverse set of Command and Control (C2) mitigation activities across recommended scenarios. 8. Ability to provide all-hazards scenario set. Page 12

The following scenarios were approved by the panel: 1. Riverine flooding with hazmat (prototype) 2. Active shooter terrorist attack with multiple explosives 3. Hurricane with regional evacuation 4. Hazardous materials release 5. Cyber attack (loss of power) 6. Earthquake Outlines for the recommend scenarios are found in Appendix A. Page 13

Phase II Findings 3.5 Field Test Evaluation Summary Three field tests utilizing the prototype system were executed in Phase II. The primary questions considered in the evaluation were: • What are the participant’s feelings and attitudes towards achievement of critical learning objectives during the TERA exercise? • Did TERA present realistic content and an environment for participants to accurately perform their assigned role? • Did TERA help participants gain a better understanding of how to make decisions during an emergency event? • Should TERA be incorporated into training or education courses for transit agencies? The first set of questions measured participant reaction responses on intended learning objectives for TERA. Additionally, the second set of questions measured participant reaction responses on realism of TERA scenario content, their user experience, and decision-making ability while playing the simulation. Finally, the third set of questions measured participant reaction responses to general expectations for the simulation tool, whether or not TERA should be incorporated for use in transit professional development, and areas users like best and least about using the tool. The evaluation form is included as Appendix E. 3.6 Field Test 1 Evaluation Results At the conclusion of the first field test, participants were asked to rate their progress on five learning objectives intended for the simulation exercise using a one-to-five Likert scale to measure their improvement on critical emergency knowledge and skills (1 = no improvement, 5 = exceptional improvement). The learning objectives included the ability of participants to manage information and communication during a flood disaster, maintain focus on incident priorities and objectives, assess the situation and select the best course(s) of action, reflect upon the simulated experience and discuss the reasons for the decision, and identify and use transit assets and resources as needed. Overall, participants felt they made slightly above average progress on understanding the intended learning objectives (combined mean = 3.3, SD = 0.209). Moreover, participants evaluated not only intended learning objectives for the simulation exercise, but also provided both written and numeric feedback rating and summarizing their feelings and attitudes on the general use and presentation of TERA. Participants indicated they felt strongly that TERA presented realistic content (mean = 4.25, SD = 0.5), was an excellent tool for providing simulated learning experiences (mean = 4.5, SD = 0.577), they would participate in a future exercises (mean=4), and finally the tool should be incorporated into professional development training and education courses for their transit agency (mean = 4.25, SD = 0.957). 3.7 Field Test 2 Evaluation Results At the conclusion of the second field test, participants felt they made above average progress on understanding the intended learning objectives (combined mean = 3.65, SD = 0.223). Additionally, participants indicated they felt strongly that TERA presented realistic content (mean=4), was an excellent tool for providing simulated learning experiences (mean=4.5, SD=0.577), they would participate in a future exercises (mean=4.75, SD=0.5), and finally the tool should be incorporated into professional development training and education courses for their transit agency (mean=4.25, SD=0.957). 3.8 Field Test 3 Evaluation Results At the conclusion of the third and final field test, participants felt they made above average progress on understanding the intended learning objectives (combined mean = 3.75, SD = 0.306). Also, participants indicated they felt strongly that TERA presented realistic content (mean = 4.5, SD = 0.577), was an excellent tool for providing simulated learning experiences (mean = 4.25, SD = 0.5), they would participate in a future exercises (mean = 5), and finally the tool should be incorporated into professional development training and education courses for their transit agency (mean = 4). Page 14

3.9 Combined Evaluation Results Overall, participants felt they made above average progress on understanding the intended learning objectives (combined mean for all field tests = 3.56, SD = 0.5995). Moreover, participants evaluated not only intended learning objectives for the simulation exercise, but also provided both written and numeric feedback rating and summarizing their feelings and attitudes on the general use and presentation of TERA. Participants indicated they felt strongly that TERA presented realistic content (combined mean = 4.25, SD = 0.5), was an excellent tool for providing simulated learning experiences (combined mean = 4.41, SD = 0.577), they would participate in a future exercises (combined mean = 4.58, SD = 0.519), and finally the tool should be incorporated into professional development training and education courses for their transit agency (combined mean = 4.16, SD = 0.957). 3.10 Field Test Comments and Lessons Learned In the first field test, while all participants were involved in transit emergency management, not all provided participants had experience in their assigned exercise role. Only one of the participants performed his assigned exercise role as part of his job function. Senior agency emergency management personnel were on hand to give advice and guidance, which ensured the exercise went smoothly despite having some inexperienced participants. For the subsequent tests, we requested that participants have experience in their assigned exercise role in order to provide more relevant data for the training effectiveness evaluation. Also in the first field test, we conducted a short “Learn the Interface” exercise which had generic tasks such as “Send an email.” At the end of this exercise, some of the participants were still a bit hesitant with the interface. Once they started working the flood scenario however, they were engaged by the storyline and reported that the experience was “fun” and “enjoyable.” (In one case, it was truly a remarkable change in attitude and body language from “I should have called in sick today” to “Once I got into it, I really enjoyed it.”) In response to this, we developed a new “Learn the Interface” exercise which frames the tasks in a realistic context, namely a severe storm scenario, in an attempt to engage the participants earlier. This short scenario is not meant as a full exercise, but as a familiarization tool. We utilized this contextual “Learn the Interface” scenario in the second and third field tests. Following are some participant comments from the field tests: • “TERA is easy to use and friendly to the non-techie.” • “TERA kept all participants engaged at a higher level than most other exercises I have participated in.” • “TERA is a tool that can be used in a group or individual setting, so it could be used more often than other training products I have seen.” • “TERA is potentially more effective than a paper and pencil table top exercises.” • “TERA made me aware of what I needed to learn and practice.” • “TERA kept me engaged and on my toes with additional injects.” • “TERA was very realistic.” • “TERA was highly interactive and the training was beneficial.” Page 15

Phase III Findings 3.11 Deployment The 2012 Transportation Hazards and Security Summit and Peer Exchange was utilized as a platform to evaluate the final system with two scenarios. The training event was deemed a success as each of the exercise goals was met. Constructive feedback, which is outlined in detail in Appendix G, was generated to enhance future exercises. The following captures highlights from the feedback: • TERA is a valuable system that will drastically benefit the transportation industry as the industry is focused on collaboration. • It is incredibly beneficial to be able to log-in to a training event from multiple locations as gathering personnel is often a struggle when conducting exercises. • The system has tremendous potential and opportunity to broaden the scope of training. The transportation industry is looking forward to the upgrades and updates that will expand upon the system. Recommendations for the road ahead are as follows: • Expand upon TERA with more scenarios and robust roles. • Develop a transportation-centric version of the training platform. In doing so, consider joining all exercise platforms under one name. • Expand upon existing relationships to incorporate additional audiences into TERA. To properly prepare for the event, three lead state exercises were facilitated at the following locations: Washington, California, and West Virginia. Each state received complete and all-inclusive training on TERA in order to develop “peer assistant trainers” for the Security Summit. An After Action Review (AAR) for each exercise is featured in Appendix G. TERA trainers also led ten first-look state exercises. The first look state exercise consisted of an introduction briefing, a demonstration and/or a staff training event via a teleconference. The first-look exercises were conducted to create awareness and acquire TERA access prior to the Security Summit. The following states participated in the training: New Jersey, Minnesota, New Hampshire, Alaska, Mississippi, Wisconsin, Ohio, Wyoming, Massachusetts and Maryland. At the Security Summit, states were arranged by the four AASHTO regions and one non-AASHTO region. Each entity was assigned a training room, a TERA facilitator, and a lead state exercise participant to assist with the exercise. The exercise day (22 August 2012) began with an introduction briefing for all participating personnel. This briefing explained the history of TERA and its relationship to the National Guard product known as the Emergency Management Staff Trainer (EMST). It also addressed how to acquire a password, the different functions within the interface and a “Learn the Interface” training session. The morning session closed out with an individual training session where users were able to rehearse the flood and active shooter scenarios within TERA. The afternoon session launched the staff training component of the exercise. Each respective room was able to lead a staff training exercise on the flood scenario within TERA. The exercises were either a group effort (i.e. led by an AASHTO region representative) or an individual state effort that linked the participant with their state colleagues operating from a remote location. Regardless of the format, each participant was able to play one of the following roles: Operations Coordinator, DOT Representative, Planning Coordinator, Logistics Coordinator, Emergency Manager, Finance/Administration, Law Enforcement, or Public Information Officer within a scenario of their choice. Following the staff training component of the exercise, participants gathered for a working AAR. The first portion of the AAR was devoted to organizational development. Participants expressed how TERA assisted their organization in handling the scenario mission. The second portion was dedicated to a TERA analysis. This portion allowed participants to identify TERA upgrades and adjustments. Additionally, all participants were able to fill out a written feedback form to communicate their opinions. Page 16

The final portion of the exercise day was devoted to TERA enhancements. TERA facilitators led the discussion on the near-term goals for interface enhancements. The six release scenarios were finalized based on this feedback. The scenario scripts are found in Appendix H of the contractor’s final report, and the TERA scenario tasking broken out by role in Appendix I of the contractor’s final report. All issues and recommendations from field testing, the summit training event, and beta testing are summarized in Appendix J of the contractor’s final report. Final status is provided for each item. A follow-on exercise and train-the-trainer event was subsequently planned and executed in conjunction with the 2013 Transportation Hazards and Security Summit and Peer Exchange in August 2013, under Modification 2 of the contract. Overall Accomplishments under NCHRP-funded Modification 2 • We expanded TERA into the DOT domain as a part of this project. We also added an airport EOC scenario (Fuel Farm Fire) and a rail incident scenario (Crossing Incident), both of which were developed outside the scope of this project but which we felt would enhance TERA’s capabilities. • As a result of adding the DOT, airport, and rail scenarios, and in response to suggestions received at the 2013 Summit, we renamed TERA, replacing Transit with Transportation, to be the Transportation Emergency Response Application. We updated the logo to reflect this change: • We created an informational website to promote the webinars and on-site exercises at www.about.tera.train-emst.com. The website includes an updated informational video and brochure. Customize scenario for DOT • We conducted teleconference meetings with DOT Subject Matter Experts (SMEs) from West Virginia DOT and Caltrans. During these meetings, we discussed the available scenarios and chose the Flood for expansion into the DOT domain. • We worked with the SMEs to determine the set of roles to be exercised in the DOT emergency operations domain. While there are many DOT support roles that are supported by the simulation, these are the roles that can be executed by a live player in an exercise. The roles are o Department of Transportation Emergency Manager o Department of Transportation Logistics o Department of Transportation Finance/Administration o Department of Transportation Operations o Department of Transportation Plans o Department of Transportation Public Information Officer o Department of Transportation Safety Officer These roles conform to the guidance given by the National Response Framework and Incident Command System. • We worked with the SMEs to develop a spreadsheet of all injects to be delivered to the live roles during the Flood scenario. We implemented the DOT flood scenario with the same timeline and collective injects (e.g., TV reports, web news articles) as the existing Transit TERA flood scenario, so DOT and transit agencies can work together in a collaborative exercise if desired. • We executed the TERA test plan for the scenario, and deployed the completed scenario to the TERA server. Page 17