A Guide to Emergency Management at State Transportation Agencies (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

93 Plans should be routinely tested through training, drills, and exercises. This section covers training, drills, and exercises not only for the state DOT’s emergency plans but also for the agency’s broader emergency management program. Relevant state DOT case examples of training and exercise implementation are included. In addition, this section introduces the HSEEP approach to exercise development and evaluation. Once a plan has been developed and needed resources have been procured, personnel are organized and equipped; and training, exercises, and evaluation are conducted to develop required capabilities and competencies. Per the 2015 AASHTO Fundamental Capabilities of Effective All-Hazards Infrastructure Protection, Resilience, and Emergency Management, this capability involves the following: • Prepare DOT employees for their roles. • Understand and improve plans put in place. • Provide an opportunity to test plans and validate the effectiveness of training. There is no question that successful management of emergencies, especially large-scale or complex ones, requires well-trained and exercised personnel who can not only perform their assigned tasks but also work with personnel from other agencies and jurisdictions, and provide emergency services in other jurisdictions if requested. Emergency management training imparts to personnel the knowledge, skills, and abilities required to perform their roles, responsibilities, and functions in emergencies. Individual and role-specific training leads to team- or unit-level training that culminates in interagency and interjurisdictional training and exercises that allow personnel to practice and demonstrate what they have learned, and become familiar with key personnel from and foster collaboration with other agencies and organizations. Furthermore, as noted NCHRP Report 777, regional-level disasters and emergencies have high consequences but occur infrequently or not at all. Interagency and interjurisdictional exercises are essential in testing regional plans, EOPs, and mutual aid procedures. Exercises also help assess emergency management technologies, equipment, and facilities; the ability of personnel to mobilize them; and provide personnel with the opportunities to practice using them. In addition, having good documentation and an effective and consistent training and exercise evaluation process will help DOTs fulfill grant and program requirements; and engage in continuous improvement of plans, procedures, personnel, technologies, equipment, and facilities. While state DOTs fully understand the importance of training for emergencies and seek to implement high-quality training and exercise programs, they face numerous challenges that can impede their goal. The roles and responsibilities of personnel involved in emergency management continue to evolve and expand, and new threats and hazards emerge, while budgets decrease or remain stagnant. Training managers and supervisors are also faced with new or changing federal and state transportation and emergency management legislation, standards, and guidance. To S E C T I O N 6 Emergency Management Training and Exercises

94 A Guide to Emergency Management at State Transportation Agencies keep up with these challenges, state DOTs need to act strategically and create an emergency training and exercise program that is not only aligned with federal and state guidance but can also integrates changes and improvements and takes advantage of available resources and partnerships. For example, FEMA and the state EMA have a wealth of knowledge and preparedness resources on exercise planning and design, conduct, and evaluation, and there are emergency training and exercise opportunities offered at nominal or no cost to state agencies. Close coordination with the state EMA is also important because state agencies are usually expected to support the state’s NIMS and Stafford Act compliance and reporting requirements as well as the state’s efforts toward EMAP accreditation. Interagency training and exercises help ensure that key players are able to respond to emer- gencies efficiently and effectively, fulfill their roles and responsibilities in the EOP and other emergency plans, and coordinate and collaborate with each other using NIMS and ICS. Following national standards, such as NIMS, ICS, HSEEP, EMAP, and EMAC, and incor- porating them into training and exercises enhances the ability of state DOT personnel to work with other agencies and jurisdictions. While delivering training to large numbers of field personnel is costly and adhering to national standards and undertaking corrective actions may be time-consuming, the benefits of enhanced preparedness far outweigh the costs. Emergency Training and Exercise Needs Emergency training needs for state DOT personnel with emergency management roles can be determined by conducting a training needs assessment. The assessment will identify internal and external requirements and mandates, and employees’ current and potential responsibilities. The assessment results will provide the specific training required by job function or position. Personnel from maintenance and operations (M&O) field personnel to supervisors to CEOs and elected officials require some form of emergency training. In addition, individuals respon- sible for exercise planning, design, and development should have appropriate training as well. Some state DOTs offer training and assistance on emergency response and recovery and the FHWA emergency response program to local public works agencies. As state DOTs increasingly rely on contractors to perform all types of work, including emergency response and recovery work, state DOTs should ensure that those contractors selected for emergency management roles are both qualified and trained. Where appropriate and feasible, state DOTs may choose to include contractors in their training and exercise programs. In addition, some state DOTs include other emergency response providers, such as police, fire, and local public works agencies, in their training and exercise programs. Appendix E of NCHRP Synthesis 468 provides a needs assessment form used by Vermont Agency of Transportation (VTrans) to determine the training needs of their employees. National preparedness guidance and mandates include the NRF ESF #1 and ESF #3 and the NIMS. Key national standards include the Occupational Safety and Health Administration (OSHA) and the MUTCD standards. State DOT personnel with roles and responsibilities delineated in plans require training. According to CPG 101, Version 2.0, “Whenever possible, training and exercise must be conducted for each plan to ensure that current and new personnel are familiar with the priorities, goals, objectives and courses of action” (CPG 101, Version 2.0, p. 4–26). Plans include agency and state preparedness plans; joint operational or regional coordina- tion plans; all-hazards evacuation plans; SOPs; mobilization plans; continuity plans; mitigation plans; recovery plans; codes and requirements; transportation and TIM plans; and hazard- specific response plans.

Emergency Management Training and Exercises 95 DHS/FEMA grants help DOTs participate in preparedness activities, including training and exercises. They normally require implementation of NIMS and may also have training, exercise, evaluation, and reporting requirements. In addition, areas of improvement identified in risk assessments, self-assessments, reviews, performance indicators, lessons learned, and corrective actions of exercises, incidents, and planned events can also impact training and exercise programs. The emergency training needs of transportation field personnel are described in NCHRP Synthesis 468. The transportation CEO’s role in emergency response is described in NCHRP Web-Only Document 206: Managing Catastrophic Transportation Emergencies: A Guide for Trans- portation Executives. According to the document, the CEO of the state DOT has the ultimate responsibility regarding the agency’s emergency management program and its preparedness and performance during emergency situations. According to NCHRP Web-Only Document 206, the CEO must ensure the following: 1. “An agency-wide emergency operations plan that gets reviewed and updated on a regular basis. 2. A training and exercise program of annual or greater frequency that involves the state director in at least one exercise. 3. A continuity of operations plan (COOP) plan and COOP site whose capabilities are assessed on a regular basis” (NCHRP Web-Only Document 206). In addition, training and exercise personnel should have function-specific expertise and training experience. Individuals responsible for exercise evaluation and exercise design and development should take relevant courses on the topics. Emergency Management Accreditation Program The 2016 Emergency Management Standard promulgates standards for training, exercises, evaluations, and corrective action processes. EMAP provides accreditation on its Emergency Management Standard to government emergency management programs, including programs of state EMAs and state DOTs. While no DOT is currently accredited, state DOTs support their states through compliance with the EMAP standards. The training section states that an accredited EMAP includes a formal and documented training program that includes a “training needs assessment, curriculum, course evaluations, and records of training . . .” and provides “the assessment, development and implementation of training for Program officials, emergency management response personnel and the public.” (2016 Emergency Management Standard, p. 11) An accredited EMAP program should have regularly scheduled training and a method and schedule for evaluation, maintenance, and program revision. Training should be based on current and potential responsibilities, and on hazards identified in EMAP Standard 4.1.1. In addition, personnel to be trained include all personnel with emergency management responsibilities, including key officials. Pro- gram records should include training participant names and types of training planned and delivered. With respect to exercises, evaluations, and corrective actions, an accredited EMAP “regularly tests the knowledge, skills and abilities, and experience of emergency personnel as well as the plans, policies, procedures, equipment, and facilities” (2016 Emergency Management Standard, p. 11). An accredited exercise program evaluates plans, procedures, and capabilities through various means, including periodic reviews, testing, post-incident reports, lessons learned, performance evaluations, exercises, and real-world events. The evaluation results should be documented and provided to stakeholders. An accredited exercise program also includes a process for corrective actions that prioritizes and tracks each action.

96 A Guide to Emergency Management at State Transportation Agencies Continuity of Operations COOPs are activated when an emergency impedes the state transportation agency’s ability to carry out its essential functions. The emergency may be due to a natural disaster or manmade event resulting in the destruction of key facilities or a pandemic incapacitating a significant proportion of key personnel. COOPS include information about essential functions, manage- ment of vital records, and devolution procedures, including orders of succession, delegations of authority, and movement to alternate facilities. The 2013 FEMA Continuity Guidance Circular 1 for non-federal governments specifies the following training and exercise frequency and topics for continuity capabilities and essential functions: Annual testing should be performed for the following: • Alert, notification, and activation procedures • Recovery of essential records, critical information systems, services, and data • Protecting and accessing essential records and information systems • Primary and backup infrastructure systems and services • Telework capabilities, including IT infrastructure Annual exercises should be performed for the following: • Essential functions capabilities • Internal and external interdependencies identified in the continuity plan • Continuity plans and procedures, including internal and external communications, deliberate and preplanned movements to an alternate site, required backup records and data to support essential functions, and use of telework sites if telework is a part of the continuity plan Annual briefings on continuity awareness for all personnel should be conducted. In addi- tion, annual briefings on continuity and devolution should be provided to Emergency Response Group and Devolution Emergency Response Group members. Periodic briefings should be held for managers about essential records. Quarterly testing of internal and external communications equipment and systems and annual testing and exercises of physical security capabilities at continuity facilities should be conducted. Biennial exercises should be performed for reconstitution and devolution procedures. The following annual training for Emergency Response Group and Devolution Emergency Response Group members should also be conducted: • Reconstitution plans and procedures • Activation of continuity plans • Communications and IT systems • Devolution option • Electronic and hardcopy documents, references, records, information systems, data manage- ment software, and equipment In addition, annual training on essential functions and roles and responsibilities should be provided to the following: • Leadership staff • Staff expected to telework during an activation • Staff who will assume the position of organization head or other key positions • Officials who will have policy and decisionmaking authority

Emergency Management Training and Exercises 97 NIMS and ICS NIMS is the essential foundation of the National Preparedness System and provides the template for the management of incidents and operations in support of all five National Plan- ning Frameworks. The 2011 NIMS Training Program document emphasizes the importance of training and states that training of emergency personnel “is critical to the success of NIMS implementation nationally” (FEMA 2011, p. 2). Responsibilities of stakeholders, including state transportation agencies, include identifying appropriate personnel to take NIMS training, ensuring all course delivery meets the standard contained within the NIMS Training Program and other training guidance provided by NIC, and credentialing emergency and incident management personnel. Credentialing is defined in the 2017 draft NIMS Guideline for the National Qualification System as “the process of providing documentation that identifies personnel and verifies their qualifications for parti cular positions.” These responsibilities align and dovetail with the concepts of personnel qualification and certification defined in the 2017 draft NIMS Guideline. Qualification is “the process of enabling personnel to perform the duties of specific positions and documenting their demonstration of the capabilities that those positions require” (NIMS 2017). Qualification involves completion of training, obtaining licenses and certifications, and meeting fitness requirements. Certification attests “that individuals meet qualifications for key incident functions and are competent to fill specific positions” (NIMS 2017). For state DOTs, NIMS and ICS training should be a priority for the following reasons as well: NIMS and ICS enhance the effectiveness of interagency coordination and response through a standardized approach to incident management. Implementation of NIMS is a requirement of federal preparedness grants and mitigation grants. Also, NIMS requires the use of ICS at TIM scenes. Furthermore, the following is stated in NCHRP Synthesis 472: “A good understanding and implementation of NIMS concepts and principles, including ICS, NIMS resource management procedures, and ICS record-keeping procedures and forms, facilitate successful integration of state DOT personnel into their state’s emergency organization and effective reimburse- ments” (NCHRP Synthesis 472, p. 48). NIC provides NIMS training and qualification guidance and maintains and distributes foundational documents and other resources. NIC identified core competencies for personnel qualification and a NIMS national curriculum, and for the Multiagency Coordination Systems. The 2011 NIMS Training Program supersedes the 2008 Five-Year NIMS Training Plan. It is based on these core competencies which, in turn, are based on operational needs. The NIMS Training Program includes a NIMS Core Curriculum, which includes baseline NIMS and ICS courses IS-700 and ICS-100 and additional training, including MACs, EOCs, mutual aid, and resource management. In general, incident complexity will affect training needs, and some flexibility is provided to entities in the implementation of NIMS training. For instance, agencies may develop their own training courses to suit their scheduling and budgetary needs and still meet the NIMS training requirements. While field personnel require basic NIMS and ICS training, personnel in EOCs, MACs, and TMCs; supervisors; senior management; and elected officials require additional or other training. Arizona DOT’s (ADOT’s) Emergency Planning and Management Training Matrix shows required emergency management and security training for DOT personnel by job category. The matrix is included in Appendix G of NCHRP Synthesis 468. Missouri DOT’s NIMS Training Guide recommends specific NIMS training courses for personnel at different levels, including emergency responders, first-line response supervisors, mid-level response

98 A Guide to Emergency Management at State Transportation Agencies supervisors, senior-level response managers and executives, elected and appointed officials, and support staff. In addition, NIMS and ICS should be used in all training and exercises, including use of ICS in exercise planning team structures; ICS should be used in emergency response, including response to traffic incidents. Five new NIMS documents were recently released for public comment: (1) NIMS Guideline for the National Qualification System, (2) NIMS Job Titles/Position Qualifications, (3) NIMS Position Task Books, (4) NIMS Guideline for the Credentialing of Personnel, and (5) NIMS Guideline for Mutual Aid (https://www.fema.gov/national-incident-management-system/ national-engagement). Since 2010, NIMS updates and publications, including the five recently released NIMS documents, provide new definitions, policy direction, and guidance. Existing training programs and content should be reviewed against the finalized documents and updated as needed. Also, check with the state NIMS coordinator and state EMA regarding NIMS compliance matters. Mutual Aid and Emergency Management Assistance Compact State DOTs provide and receive mutual aid to/from other states and organizations using mutual aid agreements and mutual aid plans. Mutual aid operational plans include a schedule of training and exercises for validation of plan design, concept, implementation and commu- nications, logistics, and administrative structure, and affording practice opportunities to emergency response providers (Draft 2017 NIMS Guideline for Mutual Aid). EMAC is an example of an interstate/tribe/territory mutual aid compact. Understanding and being able to execute tasks related to EMAC and other mutual aid agreements are important and require appropriate training. Emergency Management Performance Grant Program EMPG Program-funded activities include updating emergency plans, conducting training, and designing and conducting exercises to validate core capabilities, maintain current capabili- ties, and enhance capability for high-priority core capabilities with low capability levels. EMPG Program recipients and subrecipients are expected to address capability targets and gaps identi- fied through the annual THIRA and SPR process. EMPG Program recipients are also required to develop a multiyear TEP addressing THIRA risks and exercising and validating THIRA capa- bility requirements in a progressive manner. Recipients should develop and maintain a progressive exercise program and a multi- year TEP consistent with HSEEP. EMPG Program funds related to training should support NIMS implementation and emphasize NIMS Training Program core competencies. NIMS Training—Independent Study (IS) 100, IS 200, IS 700, and IS 800 are required for EMPG Program-funded personnel. In addition, they are required to complete either the courses in the Professional Development Series or the National Emergency Management Basic Academy. The EMPG Program also has the following exercise participation requirements: • No fewer than four quarterly exercises of any type and one full-scale exercise within a 12-month period are required. The exercises should increase in complexity and have common program priorities.

Emergency Management Training and Exercises 99 • EMPG Program-funded personnel are required to participate in no fewer than three exer- cises in a 12-month period. For allowable costs and other information, see DHS Notice of Funding Opportunity Fiscal Year 2016 Emergency Management Performance Grant (EMPG) Program. FEMA Training FEMA’s Emergency Management Institute offers complimentary, on-demand courses acces- sible to the public via the Internet. FEMA suggests the completion of at least the following independent study courses (http://training.fema.gov/IS/): • IS-1: Emergency Manager: An Orientation to the Position • IS-10: Animals in Disaster, Module A—Awareness and Preparedness • IS-11: Animals in Disaster, Module B—Community Planning • IS-100.a: Introduction to Incident Command System • IS-120.a: An Introduction to Exercises • IS-130: Exercise Evaluation and Improvement Planning • IS-197.EM: Special Needs Planning Considerations—Emergency Management • IS-200.a: ICS for Single Resources and Initial Action Incidents • IS-208.a: State Disaster Management • IS-230.a: Fundamentals of Emergency Management • IS-235: Emergency Planning • IS-288: The Role of Voluntary Agencies in Emergency Management • IS-366: Planning for the Needs of Children in Disasters • IS-547.a: Introduction to Continuity of Operations • IS-650.a: Building Partnerships with Tribal Governments • IS-700.a: NIMS—An Introduction • IS-701.a: NIMS Multiagency Coordination Systems • IS-702.a: NIMS Public Information Systems • IS-703.a: NIMS Resource Management • IS-704: NIMS Communications and Information Management • IS-706: NIMS Intrastate Mutual Aid—An Introduction • IS-800.b: National Response Framework, An Introduction • IS-860.a: National Infrastructure Protection Plan FEMA’s Emergency Management Institute offers classes in All-Hazards Position Specific Training Program annually as well. The courses are for personnel with responsibilities for managing complex incidents who are seeking certification for ICS command, general staff, unit leader positions, or current members of an Incident Management Team. Federal Reimbursement Programs Federal reimbursement programs, such as the FHWA Emergency Relief and the FEMA Public Assistance program, have complex processes and procedures. Fully understanding the program requirements can save DOTs time and expense and help them receive the reimbursement amounts to which they are entitled. In addition to documentation, damage assessment, debris management, knowledge of emergency contracting, repair versus replacement considerations, design and mitigation techniques, environmental and historical preservation regulations, and construction and procurement procedures are required. Each of these function areas has detailed requirements.

100 A Guide to Emergency Management at State Transportation Agencies As stated in NCHRP Synthesis 472, a “high level of preparedness leads to better outcomes (with respect to the FEMA and FHWA reimbursement programs) for state DOTs.” Training practices that were helpful to DOTs in obtaining successful reimbursements included providing training on the FHWA Emergency Relief and FEMA Public Assistance programs to personnel responsible for documentation and reimbursement, training in conducting assessments, using scenarios from prior disasters, providing training to local public agencies on both programs, providing training to state EMA personnel on the FHWA Emergency Relief program, and train- ing state DOT personnel for integration into the state EMA as FEMA project officers and project coordinators. Transit agencies should also achieve full understanding of FTA’s Emergency Relief program and comply with its requirements. Emergency Evacuations Exercises can assess the feasibility of an evacuation plan and train personnel. State transpor- tation agencies conduct and participate in exercises in areas with high likelihood of hurricanes and flooding to prepare for an evacuation. As described in the Texas DOT case study presented in NCHRP Web-Only Document 215: Incident Command System (ICS) Training for Field-Level Supervisors and Staff, after Hurricane Rita, DHS conducted a series of hurricane preparedness exercises in the Gulf Coast region to prepare for the 2006 hurricane season. Contraflow has been used by hurricane-prone areas for emergency evacuations. Contraflow is described as “a form of reversible traffic operation in which one or more travel lanes of a divided highway are used for the movement of traffic in the opposing direction” (NCHRP Report 740). Exercise results along with results of simulations and traffic analyses are used to fine-tune contraflow operations plans. Traffic Control and Management Traffic control during emergencies requires traffic management teams be able to manage and direct traffic on highways and critical intersections that lack active signalization and during contra- flow operations. MUTCD, TIM, work zone safety practices, and TMCs and ITS technologies are important elements in effective traffic control and management. HAZMAT protocols and OSHA guidance may be helpful in incidents involving HAZMAT and other incident types as well. MUTCD The FHWA’s MUTCD (available at http://mutcd.fhwa.dot.gov/ser-pubs.htm) is a national standard on traffic control devices on all roadways and bikeways open to public traffic. MUTCD also contains standards on TTC and TIM activities. TTC functions include the movement of road users around an incident, reducing the likelihood of secondary traffic crashes, and precluding unnecessary use of local roadways. Non-compliance with the MUTCD can result in the loss of federal-aid funds and tort liability. Because the MUTCD is detailed and comprehensive, sufficient training in the standard is required of emergency response providers. In particular, responders should be trained in safe practices around TIM areas to ensure their safety and the safety of motor- ists. The ITE Traffic Engineering Handbook recommends periodic training updates to train per- sonnel on new MUTCD policies or standards, industry practices, or DOT policies and procedures. The MUTCD is published by FHWA under 23 Code of Federal Regulations (CFR), Part 655, Subpart F. The following parts are applicable during emergency traffic management: • Part 4—Highway Traffic Signals • Part 5—Traffic Control Devices for Low-Volume Roads • Part 6—Temporary Traffic Control

Emergency Management Training and Exercises 101 • Part 7—Traffic Control for School Areas • Part 8—Traffic Control for Railroad and Light Rail Transit Grade Crossings • Part 9—Traffic Control for Bicycle Facilities Traffic incidents are classified into the following three classes of duration for the purpose of determining traffic control needs. These classes are as follows: • Major—expected duration > 2 h • Intermediate—expected duration 30 min to 2 h • Minor—expected duration < 30 min In addition to the FHWA’s MUTCD web page, MUTCD Training and Resources are also available from a number of organizations: • LTAP Offices • American Traffic Safety Services Association (ATSSA) • Institute of Transportation Engineers (ITE) • AASHTO • National Highway Institute (NHI) • International Municipal Signal Association (IMSA) • National Work Zone Safety Information Clearinghouse TIM Training State DOTs, tribal and local public works agencies and safety and service patrols address and manage traffic incidents on a daily basis. Field personnel also perform TIM activities during emergencies and disasters. The TIM Self-Assessment inquires whether the agency held at least one multiagency training session on the following topics: • NIMS and ICS 100 • Training of mid-level managers on the National Unified Goal • Traffic control • Work zone safety • Safe parking Also included is a question on whether all responders have been trained in traffic control following MUTCD guidelines. After the introduction of NIMS, NIMS concepts and elements were incorporated into TIM. The National Traffic Incident Management Coalition (NTIMC) National Unified Goal Strategy concerning Multidisciplinary NIMS and TIM Training “ensures that incident responders are cross-trained on scene roles and responsibilities and have a thorough understanding of the Incident Command System (ICS) as required in the National Incident Management System (NIMS)” (2010 Traffic Incident Management Handbook Update, page 5). Furthermore, the 2010 Traffic Incident Management Handbook Update states that TIM programs “at all stages of development can and should tap into NIMS resources to achieve ‘Preparedness.’” This integra- tion of NIMS into TIM practices provides field personnel with frequent on-the-job training of NIMS through the response to actual incidents. Traffic Management Centers and Technologies Because TMCs play an important role in traffic and incident management and emergency support, TMC staff should be appropriately trained. TMCs should conduct task-specific needs assessment and crosscutting needs assessment. These assessments may result in the identification of specific response scenarios, and development of interagency training

102 A Guide to Emergency Management at State Transportation Agencies programs and exercises. According to the FHWA, two training-related strategic actions that should be taken by TMCs focus on training and discussion-based exercises to train personnel on plans and procedures, and the certification of required personnel and equipment to ensure their preparedness. TMCs monitor and manage transportation systems and support incident management through the deployment of ITS and other transportation technologies. TMCs are often co-located with EOCs and work closely with emergency response personnel from their DOT; state and local police and fire departments; other state, local, and regional agencies; private contractors; towing companies; and other organizations to manage traffic incidents of all sizes and types. Work Zone Safety and Mobility Rule The Work Zone Safety and Mobility Rule that went into effect in 2007 requires training as well as periodic refresher training for all personnel in work zone transportation management and traffic control. Note that the Rule requires TMPs for all federal-aid projects and recommends TMPs for non-federal-aid projects as well. For significant projects, TMPs must have a TTC Plan and address traffic operations and public information and outreach. The National Work Zone Safety Information Clearinghouse (www.workzonesafety.org) supplies information on relevant training. Other sources include the NHI and professional organizations, such as the ITE, the American Traffic Safety Services Association, and the Inter- national Municipal Signal Association. The 2013 FHWA Work Zone Operations Best Practices Guidebook describes best practices by state DOTs, such as a TMP peer review process by Michigan DOT, New Jersey DOT’s Safety Program (including emergency plans and training) specification as a contractor requirement and its on-site traffic control coordinator training, and Virginia DOT’s Flagger Certification Program. OSHA As described on its website (www.osha.gov), the Occupational Safety and Health Act of 1970 created OSHA “to assure safe and healthful working conditions for working men and women by setting and enforcing standards and by providing training, outreach, educa- tion and assistance.” While not all DOTs are required to follow OSHA regulations, OSHA has a comprehensive set of useful standards covering workplace hazards. The OSHA stan- dards address many workplace hazards and hazard communication, including working with HAZMAT; personal protective equipment, fire protection; fall protection; construc- tion, maintenance, and other roadway activities; bloodborne pathogens; and emergency response. Various training resources, including the Hazardous Waste Operations and Emer- gency Response training, can be found on the OSHA website. For specific OSHA guidance, consult your compliance unit. HAZMAT Workers handling or transporting HAZMAT require specialized training, while all field per- sonnel would benefit from basic awareness and communications training. A useful resource is HMCRP Report 5: A Guide for Assessing Community Emergency Response Needs and Capabilities for Hazardous Materials Releases (2011).

Emergency Management Training and Exercises 103 Disclaimer: Complex federal environmental, safety, and health regulations, including OSHA regulations, need to be followed by state transportation agencies. For specific guidance, consult your compliance unit. Physical Security and Cybersecurity The 2013 NIPP calls for the strengthening of Critical Infrastructure Security and Resilience through the coordinated development and delivery of technical assistance, training, and edu- cation. Another of NIPP’s goals is promoting “learning and adaptation during and after exer- cises and incidents.” The 2013 NIPP also encourages broad participation in exercises to address diverse needs and purposes and addressing “cascading effects involving the lifeline functions” and determination of infrastructure priorities during response and recovery (2013 NIPP, p. 24). Physical security and cybersecurity training issues will be covered in the NCHRP 20-59 51A project, which is expected to result in an updated Security 101 guide. Planned Events Planned events present good opportunities to assess plans and provide personnel with addi- tional opportunities to practice their training. As described in NCHRP Report 777, Washington Metropolitan Area Transit Authority (WMATA) evaluates the movement of drivers and riders during the annual Fourth of July fireworks on the national mall in Washington, DC. Also, large- scale planned events (e.g., Superbowl, Presidential Inaugurations) require substantial prepara- tion, including training and exercises, and require the participation of many agencies, including DOTs and jurisdictions. Technologies Effective mobilization of technologies and equipment used in emergencies is important in emergency response. It should therefore be incorporated into exercises when possible. All tech- nologies to be used in emergency situations should be used in daily operations as well to ensure that personnel are able to use them. If a technology is complex, appropriate training should be provided prior to use. In addition, if a technology cannot be used in daily operations, personnel should receive appropriate training and the opportunity to use it in exercises. Hazard-Specific Training Since hazards and threats vary by state and region, specific state DOT training and exercise needs will depend on the likelihood of particular hazards and threats in the state or region. For instance, coastal regions are adversely affected by hurricanes, while regions with nuclear power plants need radiological response plans. Exercises Training alone is not sufficient to achieve qualification in emergency management functions. Experience and practice through exercises and actual events or incidents are essential. Exercises are beneficial for multiple purposes, including the evaluation of personnel, plans, procedures, equipment, and facilities. NIPP 2013 emphasizes “continuous learning and adaptation” through a call to action to learn and adapt during and after exercises and incidents, and rapidly incorpo- rate lessons learned into technical assistance, training, and education programs.

104 A Guide to Emergency Management at State Transportation Agencies Drills are a common form of exercise for state DOT field personnel and are used to provide training on specialized equipment or a specific function or procedure. TDOT’s Comprehensive Exercise Program (CEP) document lists 10 purposes for the program. 1. Exercise the Transportation Emergency Preparedness Plan, supporting plans, catastrophic annexes, and specific policies and procedures to ensure TDOT’s ability to respond effectively to the needs of the citizens and local jurisdictions during emergencies. 2. Exercise the Emergency Support Functions assigned to TDOT under the Tennessee Emer- gency Management Plan to respond effectively to the needs of the citizens and local jurisdictions during emergencies thereby improving individual and team performance, strengthening professional relationships, retaining skills, abilities, experiences, and practic- ing or clarifying response organization roles and responsibilities. 3. Institutionalize and document the TDOT emergency management exercise program and its principles to regularly test or practice the skills, abilities, and experiences within the community of emergency management for the state of Tennessee. The CEP will also validate or test the capabilities of TDOT policies, plans, procedures, organization, equipment, facilities, personnel, training and agreements for the response and recovery phases that will allow for the return of TDOT and the transportation infrastructure system to a normal status as soon as possible; and to establish exercise program pro- cesses, practices, goals, and objectives for TDOT emergency management stakeholders across the state. 4. Follow the State and agency policy and plan review cycle in order to validate the new or updated documents. 5. Establish a documented corrective action process/plan (CAP) and IP that will ensure constant improvement in emergency response capabilities in TDOT. 6. Comply with TEMA and DHS requirements and known emergency management best practices. 7. Exercise response operations and planning efforts according to contractual obligations. 8. Exercise emergency response operational plans for all PROBABLE and the more likely POSSIBLE hazards and threats to Tennessee. 9. Exercise the capabilities and legal guidelines to provide the service, assistance, coordination, and expertise to the citizens of Tennessee as described by the TEMP. 10. Support local jurisdictional training and exercise programs as best as possible. The 2013 HSEEP offers substantial exercise planning guidance. Many preparedness and homeland security grants require exercises and the development of AARs/IPs conforming to the HSEEP. The HSEEP document provides a set of guiding principles and a common approach to exercise program management; design and development; conduct; evaluation; and improvement planning. Designed to be flexible and adaptable, HSEEP is applicable to all types of exercises. The fundamental principles of HSEEP include a focus on capability-based objectives and exercise priorities informed by risk; guidance of the exercise program and individual exercises by elected and appointed officials; integration of the whole community where appropriate; and use of common methodology. HSEEP principles also include a progressive planning approach, with exercises temporally increasing in complexity, and alignment of exercises using a common set of priorities and objectives. In addition, HSEEP emphasizes the development of a multiyear TEP to schedule and coordi- nate the delivery of training and exercise activities. The TEP takes a progressive approach with exercises and training, becoming increasingly complex while adhering to the exercise program priorities. The training is designed to prepare participants for future exercises. Examples of multiyear TEPs are included in the Caltrans and TDOT case studies.

Emergency Management Training and Exercises 105 Close coordination with the state EMA ensures that DOT exercise and training activities support state priorities as well as DOT priorities. Regional or district offices should coordinate with regional EMA offices, which can offer various types of assistance with exercise planning and implementation. DOTs can also leverage limited resources by participating in exercises or training sponsored by EMA or other agencies, including local law enforcement and fire departments. The TEP results from a Training and Exercise Planning Workshop (TEPW). The TEPW deter- mines the strategy and format of an exercise program and program priorities based on input from elected and appointed officials. TEPW participants should include the following: • Elected and appointed officials • Representatives from relevant disciplines (federal, state, regional, local) • Personnel responsible for exercise management and conduct, including facilitators, control- lers, and evaluators CPG 101, Version 2.0, emphasizes the importance of considering the whole community, including special needs populations and the private sector, in the planning process and in exercises. Priorities are based on jurisdiction-specific threats and hazards, which may be based on THIRA and other risk assessments, corrective actions from actual events and exercises, external requirements, and accreditation standards (e.g., EMAP), regulations, and legislative requirements [e.g., OSHA, MUTCD, Nuclear Regulatory Commission (NRC)]. Once the program is underway, a rolling summary of outcomes provides an analysis of trends to inform elected and appointed officials, support reporting requirements, and modify the exercise schedule and objectives as appropriate. Exercise Cycle The HSEEP Exercise Cycle contains the following four elements: (1) exercise design and development, (2) conduct, (3) evaluation, and (4) improvement planning. Key activities in each element are listed as follows: 1. Exercise design and development. Identify exercise planning team, assign team members to schedule meetings, identify/develop objectives, design the scenario, create documentation, plan exercise conduct and evaluation, and coordinate logistics. Seek input from elected and appointed officials. 2. Conduct. Prepare for exercise play, and manage the exercise and wrap-up activities. 3. Evaluation. Planning for evaluation begins at the start of the exercise cycle. Performance of personnel, plans, procedures, equipment, and facilities (per 2016 EMAP) is evaluated against exercise objectives, and strengths and weaknesses are identified during the evaluation process. 4. Improvement planning. Corrective actions are tracked to completion. The 2016 EMAP requires that corrective actions be prioritized as well. The 2016 EMAP also requires that the products of evaluations—not just exercises but periodic reviews—testing, post-incident reports, real-world events, and post-incident reports be documented and distributed. Exercise Types There are two categories of exercises: discussion-based and operations-based. Discussion-based exercises. Seminars, workshops, tabletop exercises (TTXs), and games are less costly and time-consuming than operations-based exercises. Discussion-based exer- cises use a facilitator to direct discussions. They familiarize and train participants on or develop plans, policies, agreements, and procedures, such as development of operational concepts and procedures, transportation EOPs, public information dissemination strategies, and coordination

106 A Guide to Emergency Management at State Transportation Agencies of response and evacuation plans. Discussion-based exercises can also help in the creation of training. Operations-based exercises. Drills, functional exercises (FEs), and full-scale exercises (FSEs) are more realistic and conducted in real time. Operations-based exercises provide personnel with the opportunity to practice what they have learned. They are useful for assessing emergency plans, procedures, personnel, technologies, and equipment. Note that FSEs are considered having the highest realism and require the most resources and time in both exercise duration and planning. Table 13 contains brief descriptions of each exercise type, and Table 14 provides advantages and disadvantages by exercise type. Emergency Response Drills According to the 2013 NCHRP Report 740, there are four types of emergency response drills useful for assessment of emergency evacuation plans: 1. Plan walk-through: introduces the emergency plan, procedures, communications pathways. It yields ideas and comments on plans and can also be used as refresher training. 2. TTXs require participants to respond to a hypothetical crisis; duration of play can reach several hours. 3. Event simulations provide enhanced realism (e.g., using victims with fake blood, using actual members of special populations); duration ranges from 2 h to 8 h. 4. A full deployment drill is extremely realistic and involves multiple agencies and jurisdictions; this type of drill can last several days. Facilitated Exercise 2010 MTI Emergency Management Training and Exercises for Transportation Agency Operations describes another type of exercise, the facilitated exercise. The San Jose Metropol- itan Medical Task Force exercise chair created a format that divided a full-scale exercise into segments and incorporated refresher training. The facilitator guides the exercise play in each segment and ensures that no significant mistake has been made by the players. For example, if an important topic is left out of the Incident Action Plan, the facilitator will inform the players regarding its significance and will ensure that it be included in the Plan. The thinking behind this is based on andragogy—learners remember what they do, not what they hear. Therefore, instead of allowing players to engage in incorrect behaviors or make the wrong decisions, the facilitator takes steps to direct the exercise play in the appropriate direction. Refresher Training Refresher training for NIMS and ICS courses is not available through FEMA Emergency Management Institute (EMI) and has very limited availability through other sources. TDOT, however, does provide a 4-hour refresher NIMS and ICS course every 5 years (NCHRP Synthe- sis 468). In addition, exercises and real-life incidents provide personnel with opportunities to practice NIMS and ICS. Sandboxes Sandboxes can help participants envision actions and facilitate operations-based exercises by simulating certain actions and scenes. The sandbox can include model vehicles, buildings, traffic signals, and so forth (2014 MTI Exercise Handbook).

Emergency Management Training and Exercises 107 Drills A coordinated, supervised activity to do the following: —validate a specific operation or function in a single agency or organization; and —train on new equipment; develop or validate new policies or procedures; or practice and maintain current skills. Functional Exercises (FEs) Uses exercise scenarios with event updates that drive activity at the management level, conducted in a realistic, real-time environment to validate and evaluate: —capabilities, multiple functions and sub-functions, or interdependent groups of functions; and —plans, policies, procedures, and staff members in management, direction, command, and control function. Actual movement of people and equipment may not occur. Full-Scale Exercises (FSEs) The most complex and resource-intensive type of Exercise. FSEs involve multiple agencies, organizations, and jurisdictions. FSEs validate many facets of preparedness. (Derived from HSEEP Glossary 2013.) Discussion-Based Exercises Seminars Seminars provide an overview of authorities, strategies, plans, policies, procedures, protocols, resources, concepts, and ideas, such as the following: —develop or change plans or procedures; and —assess the capabilities of interagency or interjurisdictional operations. Workshops Similar to seminars, but participant interaction is increased, and the focus is placed on achieving or building a product. Successful workshops clearly define objectives and have the broadest possible stakeholder attendance. Tabletop Exercises (TTXs) Held in an informal setting, generates discussion of various issues for a hypothetical emergency. TTXs are used to enhance general awareness; validate plans and procedures; rehearse concepts; and assess the systems to guide the prevention of, protection from, mitigation of, response to, and recovery from a defined incident. Games Simulates operations in a competition to do the following: —explore the consequences of player decisions and actions; and —validate or reinforce plans and procedures or evaluate resource requirements. Operations-Based Exercises Table 13. Discussion-based exercises and operations-based exercises.

108 A Guide to Emergency Management at State Transportation Agencies Helpful Tips for Exercise Programs The following is based on the TDOT case study: • Establish relationships with members of state, local, and federal government emergency management communities, volunteer agencies, and private industry. • Build on existing relationships through regional personnel with key stakeholders. • Emphasize joint state and local exercises. • Design shorter duration exercises. • Consolidate exercise objectives and requirements to enhance efficiency. Design and Development The 2013 NIPP recommends the following regarding exercise design: “Design exercises to reflect lessons learned and test corrective actions from previous exercises and incidents, address Discussion-Based Exercises Advantages —Conducted in a safe, non-stressful environment at a lower cost than operations- based exercises. —Interaction among peers fosters learning. —Can assist in the identification of additional training needs. —May be helpful in developing future training content and scenarios. Disadvantages —Cost may become an issue if the exercise is held at a location difficult to access for some or all the participants. —Discussion-based exercises do not provide the realism that operations-based methods provide. Operations-Based Exercises, Drills Advantages —Provide hands-on experiential learning with respect to functions, activity, or equipment. —Provide a sense of urgency without the possibility of serious consequences. —Can help identify procedural and policy gaps. —May avoid comprehension problems related to literacy and language deficiencies. Disadvantages —Providing hands-on training to a large number of individuals can be time-consuming and costly. —Scheduling drills can be difficult due to scheduling issues of the field personnel, the instructor, and the facility or equipment. —Variables differ based on the individual, so consistent outcomes are not assured. —Personality differences between the instructor or mentor and the worker may cause issues. Operations-Based Exercises, Functional Exercises (FEs) Advantages —Offer experiential learning in a realistic setting. —Facilitate the retention of knowledge and skills. —Help identify units and individuals that would benefit from additional training. —Can help develop future training content and scenarios. Disadvantages —Arranging and scheduling FEs can be difficult and time-consuming. Operations-Based Exercises, Full-Scale Exercises (FSEs) Advantages —Offer very high realism, complex situations. —Help identify units and individuals that would benefit from additional training. —Can help develop future training content and scenarios. Disadvantages —Significant coordination, preparation, resources, and time are required. (Adapted from NCHRP Synthesis 468.) Table 14. Advantages and disadvantages of discussion- and operation-based exercises.

Emergency Management Training and Exercises 109 both physical and cyber threats and vulnerabilities, and evaluate the transition from steady state to incident response and recovery efforts” (2013 NIPP, p. 26). For design and development of individual exercises, exercise program managers will need to manage multiple elements, including the exercise budget and staff along with IT requirements, exercise tools, and resources; MOUs and other agreements; technical assistance; equipment and supplies; and documentation from previous exercises. Design and Development Steps Exercise design and development steps include the following: 1. Review elected and appointed officials’ guidance, the TEP, and other factors. 2. Select participants for an exercise planning team and develop an exercise planning timeline with milestones. 3. Select objectives and core capabilities. 4. Identify evaluation requirements. 5. Develop the exercise scenario. 6. Create documentation. 7. Coordinate logistics. 8. Plan for exercise control and evaluation. Key Meetings The following meetings are usually held to help move the design and development process forward. Of these meetings, the HSEEP notes that the following are considered essential meetings: Initial Planning Meeting (IPM) and Final Planning Meeting (FPM). Prior to these meetings, it is helpful to have identified the lead agency, funding sources, any labor restrictions, and HSEEP compliance issues (2014 MTI Exercise Handbook). • Concept and Objectives (C&O) Meeting identifies the scope and objectives of the exercise. Participants include elected and appointed officials, representatives from the sponsoring and participating organizations, and the exercise planning team leader. The C&O Meeting can be held with the IPM. • Initial Planning Meeting (IPM) determines exercise scope and identifies exercise design requirements and conditions, such as assumptions and artificialities. Exercise documenta- tion responsibilities and exercise details are determined; they include exercise objectives, location, schedule, duration, participant extent of play, and scenario variables (e.g., time, location, hazard selection). • Master Scenario Events List (MSEL) Meeting focuses on developing the MSEL. The meeting may be combined with the MPM or FPM. • Midterm Planning Meetings (MPMs) provide opportunities to discuss exercise organization and staffing concepts, scenario and timeline development, scheduling, logistics, adminis- trative requirements, draft documentation review, and any other issues. They also provide opportunities to engage elected and appointed officials. • Final Planning Meeting (FPM) ensures the readiness of all components and resolution of any outstanding issues. No significant changes should be made during or after this FPM. Exercise Planning Team The exercise planning team is led by a team leader. The team determines exercise objectives, designs the scenario, ensures that activities are aligned with the overall multiyear plan, and creates the Exercise Plan. Exercise Plans have clear objectives. They define exercise roles, responsibili- ties, and management structure based on NIMS and ICS; procedures for debriefings and Hot

110 A Guide to Emergency Management at State Transportation Agencies Wash; and AAR, IP, or Corrective Action Plan development. The exercise planning team also creates other needed documentation and coordinates logistics, including safety of participants. While agencies can and do conduct exercises, such as TTXs and FEs, to test plans and mobi- lization capabilities, there will be no opportunity to exercise an actual evacuation. NCHRP Report 740 stresses the importance of including as many agencies as possible and representa- tives of populations with special needs in the planning process. Helpful Tips for the Exercise Planning Team These and the remainder of the helpful tips in this section are based on the following sources: 2013 HSEEP, 2014 MTI Exercise Handbook, NCHRP Report 777, 2015 TDOT Comprehensive Exercise Plan, and 2013 NIPP. • In exercises and drills managed by another agency, participate in exercise planning to ensure the DOT role is realistic to operations. • In exercises and drills managed by your DOT, ensure all key stakeholders participate. • Plan for high-probability and low-probability events. • Consider special needs populations and pets. • Exercise planning team should keep key officials, state EMA, and other stakeholders in the loop. • Safety should always be top of mind. • Have a clear organizational structure based on ICS structure. • Use project management tools and processes. • Start evaluation planning and fill key evaluation roles at the start of this process. • Review the detailed checklists for discussion-based exercises, operations-based exercises, and facilitated exercises available in the 2014 MTI Exercise Handbook. • Ensure adequate play for all participants. • Exercise focus should be driven by the objectives, not by the scenario. • Limit the number of objectives (few rather than many). • Use the SMART guidelines to develop exercise objectives. Exercise Objectives The objectives selected for the exercise will drive scenario selection, evaluation, and all other aspects of exercise development. Therefore, exercise objectives should be prudently chosen with input from elected and appointed officials. Following SMART guidelines, assist planning teams in identifying and developing relevant exercise objectives: • Specific: Objectives address who, what, when, where, why. • Measurable: Measures should define quantity and quality, with a focus on concrete outcomes and actions. • Achievable: Objectives should be feasible for players. • Relevant: Objectives should be pertinent to the agency’s mission, goals, and strategic intent. • Efficient: A reasonable timeframe should accompany each objective. In addition, keep the exercise objectives to a feasible number and focus on exercising emergency functions assigned to the DOT by the state. The following are sample exercise objectives: • Mobilize resources for contraflow operations. • Track and document resources. • Conduct effective situational assessment. • Identify alternative transportation solutions. • Respond to an active shooter event.

Emergency Management Training and Exercises 111 The 2014 MTI Exercise Handbook presents a list of example objectives based on Wisconsin Emergency Management 2004 (Table 13, p. 128). They include the following: • Communications: “To determine the ability to establish and maintain communications essential to support response to an incident/accident and the immediate recovery, including establishing interoperable communications with first responder agencies.” • Damage Assessment: “To demonstrate the ability to organize and conduct damage assess- ment, including the collection of information to facilitate response by first responder organi- zations, support of overweight permits, and recovery activities.” • Emergency Public Information: “To determine the capability of the emergency public informa- tion system to disseminate timely and accurate emergency response information in languages and methods appropriate to the community; evaluate the ability to work with the media and maintain media monitoring and rumor control; evaluate the adequacy of the electronic sign- boards, travel information radio, 5-1-1 system, and agency website for maintaining timely travel information to the public.” • Emergency Evacuation: “To determine the adequacy of the evacuation plan for the juris- diction and the ability of officials to effectively coordinate an evacuation. Demonstrate the capability and procedures to provide access, egress and emergency routing (including contraflow where appropriate) to support mass care for persons displaced by a disaster in another community.” Roles and Responsibilities Key exercise roles are noted in the 2013 HSEEP, Table 4.1. The roles include the following: • Exercise Director oversees all exercise functions. • Evaluator observes, documents, and analyzes the exercise; the evaluator has expertise in the functional areas observed. • Lead Evaluator oversees team of evaluators; is familiar with all issues concerning the exercise; and is able to analyze capabilities. • Facilitator for discussion-based exercises keeps discussions aligned with exercise objectives; ensures all issues and objectives are covered. • Controller for operations-based exercises and some games plans and manages exercise play and timeline, sets up and operates exercise, directs pace of the play by providing injects and other information, and ensures safety of participants. • Senior Controller oversees exercise organization, including all controllers; manages exercise progress; oversees exercise setup and takedown; and debriefs controllers and evaluators. • Safety Controller monitors exercise safety during exercise setup, conduct, and cleanup. Scenarios A scenario is a narrative or timeline used in operations-based exercises and TTXs that “drives an exercise to test objectives” and is “informed by actual threats and hazards. . . .” (2013 HSEEP Glossary). Scenarios should include the following: “(1) the general context or comprehensive story; (2) the required conditions that will allow players to demonstrate proficiency and competency in conducting critical tasks, demonstrating core capabilities, and meeting objectives; and (3) the technical details necessary to accurately depict scenario conditions and events” (HSEEP 2013, p. 3–12). The MSEL supplements the exercise scenario with a chronological list of event synopses or injects; expected responses and responsible players; and objectives and core capabilities to be addressed. It is useful for more complex exercises. Sources of scenarios include National Plan- ning Scenarios, Public Transportation System Security and Emergency Preparedness Planning

112 A Guide to Emergency Management at State Transportation Agencies Guide, and TCRP Web-Only Document 60/NCHRP Web-Only Document 200: Command- Level Decision Making for Transit Emergency Managers (available at http://onlinepubs.trb.org/ onlinepubs/tcrp/tcrp_w60.pdf), which describes scenarios addressed in TCRP Project A-36, “Command Level Decision Making for Transit Emergency Managers”: flood, hurricane, earth- quake, power outage, HAZMAT, and active shooter. NCHRP Synthesis 468 provides a Blizzard Emergency Scenario and a Flood Scenario. The 2014 MTI Exercise Handbook notes the benefits of basing the scenario on an actual event. According to the authors, Edwards and Goodrich, “An actual occurrence increases believability. Theoretically based scenarios decrease believability. The more artificialities that are used, the higher the likelihood of misunderstanding and rejection by participants.” For instance, Washington DOT used the 50-vehicle pileup on Snoqualmie Pass that occurred in February 2007 to create a scenario for a TTX held in May 2007. Helpful Tips for Scenario Development • Be realistic and challenging but not overwhelming. • Focus on local threats and hazards. • Use subject-matter experts (SMEs) to create realism. • Use the capabilities of MPOs (e.g., GIS mapping, extensive databases, stakeholder connections). • Involve a broad, diverse team for scenario development. • Link MSEL entries to the EEG critical tasks to ensure the critical tasks and core capabilities can be demonstrated. • Select three recent events; then, select the scenario that best supports exercise objectives. • Can also be based on the following: – National Planning Scenarios – Findings of AARs and IPs – Threat and Vulnerability Assessments – Current or Historical Events (local, national, or international) – Scenarios Developed for Research Studies (e.g., TCRP Project A-36) • Consider the following when developing a scenario: – Cascading effects involving lifeline functions – Flexible uses of the transportation system – Resource prioritization strategies – Mobility options and needs of all travelers, including disadvantaged populations – Chain of authority when key personnel/decisionmaker is unavailable – Critical information collection and dissemination – Identification of infrastructure priorities in response and recovery Example scenarios for discussion-based transportation sector exercises are provided in the 2014 MTI Exercise Handbook. In addition, DOTs may consider seeking assistance from the state EMA and other agencies and resources from the Homeland Security Digital Library (HSDL), HSDL.org, which now includes content from the Lessons Learned Information Sharing (LLIS) program. The HSDL contains a trove of information and case studies on lessons learned. HSDL is jointly sponsored by FEMA and the Naval Postgraduate School’s Center for Homeland Defense and Security. HSDL contains lessons learned, AARs, case studies, innovative practices, and other material formerly contained in the LLIS program, in addition to many other documents on homeland security. Helpful transportation-specific resources include NCHRP Synthesis 468; the 2014 MTI Exercise Handbook: What Transportation Security and Emergency Preparedness Leaders Need to Know to Improve Emergency Preparedness; 2010 MTI Emergency Manage- ment Training and Exercises for Transportation Agency Operations; NCHRP Report 777; and the AASHTO Catastrophic Guide.

Emergency Management Training and Exercises 113 Exercise Documentation Exercise documentation is extremely important. Identification of strengths and areas of improvement cannot be accomplished without accurate and thorough documentation. The types of documents required for each exercise type are provided in Table 3.2 of the 2013 HSEEP: • Situation Manual for TTXs and Games for All Participants • Facilitator Guide for TTX and Game Facilitators • Multimedia Presentation for TTX and Game for All Participants • Exercise Plan for Drill, FE, and FSE for Players and Observers • Controller and Evaluator Handbook for Drill, FE, and FSE for Controllers and Evaluators • MSEL for Drill, FE, and FSE for Controllers, Evaluators, and Simulators • Extent of Play Agreement for FE and FSE for Exercise Planning Team • EEGs for TTX, Game, Drill, FE, and FSE for Evaluators • Participant Feedback Form for All Exercises for All Participants Note that TDOT follows HSEEP, EMAP, and EMPG Program exercise documentation guidance. Exercise Conduct Exercise conduct involves the following key steps: Step 1—Preparing for Exercise Play Generally, discussion-based exercises require less preparation than operations-based exercises. Both require room or site and seating setup, equipment checks, multimedia presentation and handouts, sign-in sheets, and feedback forms. Also, before an exercise, separate briefings are held for elected and appointed officials, facilitators, controllers, evaluators, players, observers, and actors. Step 2—Managing Exercise Play For discussion-based exercises, the facilitator gives a multimedia presentation, and partici- pant discussion ensues. These discussions can be facilitated or moderated. Facilitated discus- sions occur in a plenary session or breakout sessions. Breakout group discussions are held before the moderated discussion in which a spokesperson from each group summarizes the key find- ings of the discussion. Evaluators observe and document these sessions. For operations-based exercises, it is important to clearly define and mark exercise areas and exercise materials to avoid confusion. Controllers control exercise flow and provide necessary data and injects to players. Activity and staff not in the exercise areas are simulated by the SimCell staff. Evaluators are pre-positioned at strategic locations to allow observation and documenta- tion of exercise play and player response. Evaluators capture both quantitative and qualitative data using EEGs. EEGs facilitate the exercise evaluation process by clearly delineating exercise objectives and associated core capabilities, capability targets, and critical tasks. Documentation of the exercise play through photos and video recordings can help train personnel, brief senior management, and improve future exercises. Step 3—Wrap-Up Activities Wrap-up activities include debriefings and a “Hot Wash,” which is a debriefing conducted immediately after the exercise. • Debriefings of exercise planning team members gather information about their satisfaction with the exercise, issues, and possible improvements. Participant feedback forms are used

114 A Guide to Emergency Management at State Transportation Agencies to collect participant feedback and develop debriefing notes. The controller and evaluator debriefing provides an opportunity for controllers and evaluators to share their observations and key insights concerning strengths and areas for improvement. • A Hot Wash, a forum for exercise participants to discuss exercise strengths and areas for improvement, is led by an experienced facilitator. Participant feedback forms can be distrib- uted during the Hot Wash. For operations-based exercises, a Hot Wash for each functional area is conducted. Helpful Tips for Exercise Conduct • Establish a contingency process to end the exercise in case of a real-world event. • Document the exercise play through photos and video recordings that can be used to train personnel, brief senior management, and improve future exercises. • Remember to document the debriefings. • Consider providing psychological support during debriefings. Exercise Evaluation Identify core capabilities for each exercise objective to create standardized evaluation mecha- nisms for the exercise, have the ability to track progress across multiple exercises, and meet funding or reporting requirements once exercise objectives have been selected. Next, identify capability targets (performance thresholds) for each of the core capabilities. Information regarding capability targets may be available from results of the THIRA or other risk assessments. Finally, identify the critical tasks needed to accomplish a core capability. Critical tasks are found in the EOP, SOPs, mission area frameworks, or other sources. The four key steps for Exercise Evaluation are described as follows: Step 1—Planning for Exercise Evaluation The first step in exercise evaluation is planning. This should start during the Exercise Design and Development stage. Actions that should be taken include the following: • Select lead evaluator. • Define evaluation team requirements and structure. • Develop EEGs, include objectives, core capabilities, targets, and critical tasks. • Recruit, train, and assign evaluators. • Develop evaluation documentation, including exercise-specific details, evaluator team organi- zation, assignments, locations, evaluator instructions, and evaluation tools. • Conduct a pre-exercise controller evaluator (C/E) briefing to confirm roles, responsibilities, assignments, and any changes. For evaluation of plans, CPG 101, Version 2.0 (page c-4), recommends asking the following questions: • “Did an action, process, decision, or the operational timing identified in the plan make the situation worse or better? • Were new alternate courses of action identified? • Were the requirements of children, individuals with disabilities, and others with access and functional needs fully addressed and integrated into all appropriate aspects of the plan? • What aspects of the action, process, decision, or operational timing make it something to keep in the plan? • What aspects of the action, process, decision, or operational timing make it something to avoid or remove from the plan?

Emergency Management Training and Exercises 115 • What specific changes to plans and procedures, personnel, organizational structures, leadership or management processes, facilities, or equipment can improve operational performance?” Step 2—Observing the Exercise Conduct and Collecting Data For discussion-based exercises, the evaluator or note-taker will record data from participant discussions. For operations-based exercises, the focus is on recording player actions. The data collected during this step will be essential in developing the AAR. Step 3—Analyzing Data and Identifying Strengths and Areas for Improvement Data analysis will determine answers to the following questions: • Were exercise objectives met? • Were the capability targets met? If not, why not? • Were players able to perform core capabilities? Did they execute the critical tasks to meet capability targets? If not, what were the impacts and consequences? • Do current plans, policies, and procedures support critical tasks and capability targets? Were participants familiar with these documents? To determine the root causes of deficiencies, evaluators will closely review critical tasks not completed and targets not met. Step 4—Reporting Exercise Outcomes The following are tips for reporting exercise outcomes: Helpful Tips for Reporting Exercise Outcomes • Select facilitators and evaluators that know and are known by your agency. • Evaluators should evaluate only their own agency and profession and jurisdiction. AAR, IP, and Corrective Actions Lessons learned lead to helpful feedback for the DOT, personnel, and teams. They are reviewed, analyzed, and compiled into an AAR. The AAR provides documentation regarding strengths and weaknesses identified during the event. After an initial analysis step, a qualitative assessment is recommended especially for exercises, incidents, and planned events to develop an initial list of corrective actions. An IP contains a list of corrective actions addressing capability gaps along with responsibility for each corrective action, target date, and tracking mechanism. The IP should include source of funds for corrective actions, and how and when the overseeing agency will be notified upon completion of a corrective action. The IP is based on the AAR and lessons learned along with relevant information from self-assessments, audits, and administra- tive reviews. To develop a draft list of improvement areas and corrective actions, the reviewer asks the following questions: What are the lessons learned for similar problems or scenarios? What changes need to be made to the following to improve performance? – Training, plans, and procedures – Organizational structures, resources, and systems – Management processes (Adapted from HSEEP 2013.)

116 A Guide to Emergency Management at State Transportation Agencies The AAR contains evaluation information, an overview of the performance with a focus on exercise objectives, and the analysis of core capabilities. According to CPG 101, Version 2.0, the AAR should address the following: • Describe the reasons and need to conduct an AAR (e.g., review actions taken, identify equipment shortcomings, improve operational readiness, highlight strengths and initiatives). • Describe the methods and agencies used to organize and conduct a review of the disaster, including how recommendations are documented to improve local readiness (e.g., change plans or procedures, acquire new or replace outdated resources, retrain personnel). • Describe the links and connections between the processes used to critique the response to an emer- gency or disaster and the processes used to document recommendations for the jurisdiction’s exercise program. • Describe how the jurisdiction ensures that the deficiencies and recommendations identified in the AAR are corrected and completed. (CPG 101, Version 2.0, Appendix C, p. c-10.) AAR templates should be reviewed and used to ensure that important information is not left out of the document. The exercise sponsor sends the draft AAR to exercise participants, and elected and appointed officials confirm conclusions and areas for improvement in the AAR. After a draft AAR and IP have been developed, elected officials and key decisionmakers are provided an opportunity to review them and provide comments. An After Action Meeting (AAM) is then held for personnel to review the updated AAR and the draft IP. The AAM facili- tator should lead focused discussions, present the event timeline and responses to assist in event recall, and distribute a feedback form to gather additional comments and input. The final AAR, IP, and Corrective Actions should be disseminated to all training and exercise participants and others affected by them. Improvement planning can be used to support continuous improvement throughout the DOT by taking a consistent approach to related activities, including issue resolution and infor- mation sharing, data collection from and analysis of exercises and events, and longer-term trend analysis. Regarding the maintenance of plans, each component of the plan should be reviewed and revised on a regular basis, and, after specific events, such as key changes in resources, changes in guidance or standards, a major exercise or incident, or a change in officials. Helpful Tips for AAR, IP, and Corrective Actions • Corrective actions should be clear, specific, and actionable. • Corrective actions should be within the DOT’s responsibility. • Ensure support for the IP and corrective actions. • Document the AAR process properly. • Tasked individuals and entities may need to develop implementing documents. • Share improvement recommendations related to NIMS, NIMS plans, and training with the NIMS national coordination process. • 2014 MTI Exercise Handbook (Annex B) contains a sample participant feedback form and sample AAR. • The HSEEP website provides helpful templates, including AAR and IP templates. Training Implementation Solutions NCHRP Synthesis 468 identified and described training implementation challenges, training needs, and solutions. Key challenges for state DOTs were scheduling difficulties and limited budgets. Additional challenges included lack of qualified training staff, personnel turnover, dis- tance issues, senior management issues, inadequate facilities and other resources, insufficient information about available training, and infrequent need for training. Training delivery solu- tions included are in Table 15.

Emergency Management Training and Exercises 117 (continued on next page) Table 15. Training delivery solutions. Advantages —Meetings are brief and held on a regular basis at a location and time convenient to field personnel. —Meetings are also focused and very relevant to field crew. —Hands-on training is possible. Field personnel can practice a procedure or skill. Disadvantages —None. Advantages —High retention of training content. —Cost-effective. Disadvantages —Personnel are not provided the opportunity to practice a skill or process before its real-life application. —Taking the time to train personnel may delay the response effort. —Training personnel in an emergency situation when their level of stress is high may hinder the learning process. Advantages —Opportunity for face-to-face interactions with peers from other response agencies through these exercises is essential preparation for larger and more complex events. —They will also prepare agencies and their field personnel to understand the ICS structure, their roles and responsibilities within the structure, and how they should integrate with personnel from other entities for these events. Disadvantages —Scheduling difficulties may impede the ability of a large percentage of field personnel to attend these sessions. Advantages —Scheduling difficulties may be mitigated by delivering emergency training in conjunction with another related topic. —Intra-agency interaction and communications may be facilitated. Disadvantages —Emergency component may need to be shortened or modified. Advantages —Alleviates the need to schedule the training in advance. —Allows 24-hour access to the material. Some on-demand services offer automated record keeping and trainee progress tracking. Disadvantages —Lack of ability to interact with other students and instructor limits learning. —Student distraction may be more likely —Self-direction is needed. Advantages —Allows trainers to select appropriate training videos, CDs, or DVD training packages that are the best value for their needs. The packages usually focus on a particular topic and contain a variety of tools. —Cost-effective because many trainees may view the content typically for a fixed cost. —Online on-demand training may charge the agency per trainee. —With video teleconferencing (VTC), closed- circuit television (CCTV), or Skype technology, it is possible to present the content at multiple locations. Disadvantages —When VTC, CCTV, or Skype technology is used, technology-related issues can arise, and connectivity and quality of the transmission may be inconsistent. —Training videos and packages on CDs and DVDs are not “on-demand”; the training needs to be scheduled. —Interaction with instructors and other trainees is limited. Field Crew Meetings Just-in-Time Training Interjurisdictional and Interagency Training and Exercises Joint Training Asynchronous Training—Computer-Based Training without Live Instructors Asynchronous Training—Prepackaged DVDs and CDs

118 A Guide to Emergency Management at State Transportation Agencies Advantages —Cost-effective way to leverage limited resources. —Alleviates having to hire additional training staff or consultants. Disadvantages —Content dilution could be possible as additional training tiers are added. Advantages —Both planned events and incidents are good opportunities to practice coordination, communication, resource mobilization, traffic management, and control strategies. Traffic incidents happen daily and provide many opportunities for practice. Disadvantages (Incidents) —There is no guarantee that a series of minor incidents, aside from traffic accidents, will occur prior to a disaster. —Incidents, even minor ones, have more risk associated with them; for instance, a minor traffic accident could become a multicar crash with many fatalities and injuries. Advantages Disadvantages —A large, geographically dispersed audience can be reached. —Allow identification of weaknesses or resource deficiencies in training, plans, procedures, and policies. —Allow the participation and interaction of key personnel in different geographic regions. —Improve individual performance, organizational communication, and coordination. —Dangerous scenarios may be simulated safely. —May or may not be web-based. —Good PC and Internet skills are necessary for learners to gain full advantage of training. —In remote locations or other areas, bad or no Internet access can hinder training. —Unforeseen connection problems may arise during training. If these are on the host’s end, training may be interrupted. —Bandwidth issues may cause delay or disruption. —May lack realism and may not provide a true test of capabilities in an emergency situation. —For synchronous simulations, scheduling can be a problem. Advantages —Can present up-to-date information. —Summarizes materials from various sources. —Can adapt the material to student backgrounds and interests. —Highlights important concepts and materials. —Instructor enthusiasm can motivate students and enhance learning (McKeachie and Svinicki 2013). Disadvantages —Reduced development of problem-solving skills and interaction among students if sufficient interaction opportunities are not provided. —May present scheduling difficulties. —Cost of the training and travel, including time. [Scheduling and travel issues may be alleviated through the use of VTC, Voice over Internet Protocol (VoIP), or similar technology.] Advantages —Cost is lower than classroom training. —Training is standardized. —Training can be provided anywhere with web access. Disadvantages —Training must be scheduled in advance. —Trainees may be distracted. —Ability to monitor student progress may be limited. —Access to a PC and Internet are required. Train-the-Trainer Planned Events and Incidents Computer-Assisted Simulations Classroom Training Online Training with Live Instructors —Familiarity with the Internet and basic PC skills are required. Table 15. (Continued).

Emergency Management Training and Exercises 119 In addition to these methods, blended training allows agencies to combine multiple methods, and choose desired aspects of each method. Cross-training personnel can provide contingencies for situations that result in significant absenteeism. Helpful Tips for Implementation Solutions • Determine who (what positions) needs what type of emergency management training, including NIMS Core Curriculum training. • In general, training should progress from individuals to intra-agency teams to interagency and interjurisdictional exercises. • Establish professional qualifications, certifications, and performance standards for individuals and teams, whether paid or volunteer, with the assistance of the NIC and the state EMA. • Ensure that content and training methods comply with applicable standards and produce required skills and measurable proficiency. • Ensure that all personnel with a direct role in emergency preparedness and emergency management complete the designated FEMA training. • Establish or leverage partnerships with other agencies and organizations to coordinate and deliver NIMS training requirements in conformance with NIMS. • Incorporate NIMS and ICS into all training and exercises. • Identify what additional training resources may be needed in the community to support response and evacuation, shelter-in-place, and quarantine activities. • Strive to make the training interactive. Much learning can occur through instructor–student and student–student interactions. • Make the training relevant and specific to real-world problems. • Provide a chance for learners to reflect on their training. Then, provide opportunities to apply their new knowledge shortly thereafter. • Acknowledge experience and knowledge by providing opportunities for participants to share information and practices. • Maintain comprehensive training records, following EMAP standards and any applicable state or agency policy. • Connect with the NIC for guidance on NIMS and ICS personnel training needs and qualifica- tions for emergency management positions. • The FHWA P2P program offers technical assistance, including training and education, on TIM and planned special event planning, procurement, deployment, and operations. • Memberships in professional organizations can be leveraged to take advantage of their train- ing and certification programs. Organizations include American Public Works Association, American Road and Transportation Builders Association, the American Traffic Safety Services Association, AASHTO’s Transportation Curriculum Coordination Council, and the Inter- national Municipal Signal Association. • Job aids and on-the-job learning can help with training retention and recall. • Technologies, such as VoIP and VTC, can broadcast classroom instruction to other districts or regional offices. • Minor incidents provide an opportunity for personnel to hone their abilities and skills and identify gaps in training. Useful training sources for NIMS and ICS include the following: • FEMA NIMS Training Program • FEMA ICS Training Program and Resource Center • FEMA NRF Resources • FEMA National Training and Education Division • FEMA National Fire Academy • FEMA Center for Domestic Preparedness

120 A Guide to Emergency Management at State Transportation Agencies • FEMA Emergency Management Institute (EMI) • FEMA Independent Study Program • FHWA National Highway Institute • FTA National Transit Institute • FEMA National Training and Education Division • State EMA • State and local police and fire departments • FHWA National Highway Institute (NHI) • Local Technical Assistance Program and Tribal Technical Assistance Program (LTAP and TTAP) centers • Universities and colleges • SHRP 2 National Traffic Incident Management (TIM) Responder Training • U.S. DOT Research and Innovative Technology Administration Transportation Safety Institute There are also many other sources of training on NIMS and ICS and other emergency management topics, including regional coalitions, associations, and member organizations. These sources are noted in Section 2 of this Guide and in Chapter 5 of NCHRP Synthesis 468. State DOT Emergency Management Training and Exercise Implementation Practices During interviews with state DOTs and case example agencies for the NCHRP Project 44-12 (resulting in NCHRP Synthesis 468), the following courses were common elements in many of their training programs: • IS-15.b: Special Events Contingency Planning for Public Safety Agencies • IS-100: Introduction to the Incident Command System • IS-200: ICS for Single Resources and Initial Action Incidents • ICS-300: Intermediate ICS for Expanding Incidents • ICS-400: Advanced ICS • IS-552: The Public Works Role in Emergency Management • IS-554: Emergency Planning for Public Works • IS-556: Damage Assessment for Public Works • IS-558: Public Works and Disaster Recovery • IS-559: Local Damage Assessment • IS-632: Introduction to Debris Operations • IS-700: Introduction to the National Incident Management System • IS-701.a: NIMS Multiagency Coordination System (MACS) • IS-703.a: NIMS Resource Management • IS-706: NIMS Intrastate Mutual Aid—An Introduction • IS-800: Introduction to the National Response Framework • SHRP 2 National Traffic Incident Management (TIM) Responder Training NCHRP Synthesis 468 contains the full training matrices for ADOT’s Emergency Manage- ment Training for its Maintenance and Landscaping/Natural Resources personnel. Portions of the matrices that include NIMS and ICS for various roles and functions are shown as follows. Maintenance Roadway, Signing, and Striping Personnel Highway Operations Worker • Basic Incident Command IS-100 • National Incident Management System IS-700 Tech 1 • Traffic Incident Management • Control of Hazardous Energy

Emergency Management Training and Exercises 121 Supervisor • ICS for Expanding Incidents IS-200 • National Response Framework IS-800 Superintendent • Intermediate Incident Command IS-300 • Advanced Incident Command IS-400 Landscape and Natural Resources Personnel Highway Operations Worker • Basic Incident Command System IS-100 • National Incident Management System IS-700 • Hazard Communication • OSHA/DOT HAZMAT • Traffic Incident Management Tech 1 • Traffic Incident Management • Control of Hazardous Energy • Introduction to Wildland Firefighting Supervisor • ICS for Expanding Incidents IS-200 • National Response Framework IS-800 Superintendent • Intermediate Incident Command IS-300 • Advanced Incident Command IS-400 In addition to this training, ADOT’s Highway Operations Worker undergoes almost 40 addi- tional training courses ranging from fire safety to flagger ATSSA certification to computer secu- rity awareness. Tech 1 takes about 10 additional courses, including advanced work zone traffic control and maintenance communications. Supervisors must also take about 10 additional courses, such as On Boarding New Employees and Managing Resources Effectively. Superin- tendents take six additional courses, including the NHI Maintenance Leadership Academy and Performance Measurement (NCHRP Synthesis 468). The following is a portion of TDOT’s emergency management training based on the DOT’s training needs assessment. • NIMS and ICS training; required courses vary based on worker function • ICS Train-the-Trainer Course • TIM training for all responders • Protect the Queue training for all field employees • Hazardous Materials Awareness training for all field employees • Hazardous Materials for Operational Level Response • Active Shooter Training for all employees • TVA Fixed Nuclear Facilities Emergency Worker Training • Oak Ridge Emergency Worker Training • Storm Spotter Training • Emergency Radio Communications Training • Emergency Management Support Team Training • Damage Assessment Workshop • Basic Public Information • TEMA 101 • Instructor Methodology • Principles of Emergency Management • Exercise Development • Communications Leader Course

122 A Guide to Emergency Management at State Transportation Agencies • Search and Navigation Courses – TEMA Search Operations – GPS Land Navigation Course – Basic Visual Tracking – Managing Search Operations • National Domestic Preparedness Consortium (NDPC)—DHS-funded courses • FEMA National Emergency Training Center (TDOT Case Study.) State DOTs use a number of different methods from field crew meetings to train-the-trainer (TTT) to interagency training and exercises to deliver emergency management training to their field personnel. Examples of these methods include the following: Field Crew Meetings • Caltrans. Tailgate meetings are used to share information and to train field personnel on new procedures, technologies, equipment, and safety issues. The meetings are held every 10 days at every maintenance yard (NCHRP Synthesis 468). • Missouri DOT. Field personnel are trained on specific tasks during field crew meetings (NCHRP Synthesis 468). Cross-Training • Tennessee. TDOT cross-trains four to five additional persons to perform a particular func- tion that has been designated as an “Essential Function” of the DOT. Essential Functions are defined in the TDOT COOP. The training need depends on the gap between the number of people that already have the capacity to perform the function and the number that is required. The actual amount of required training depends on function and system complexity (TDOT Case Study). Joint Training • SHRP 2. SHRP 2’s National Traffic Incident Management Responder Training Course is multidisciplinary and interjurisdictional and has been delivered to many state DOT personnel, along with police, fire, and other responders. The SHRP 2 program has also created a TTT course for incident responders and managers as well as an e-Learning training course being implemented through the NHI (NCHRP Synthesis 468). • Arizona. TIM training is conducted by the Arizona Department of Public Safety using the TTT version of the SHRP 2 National Traffic Incident Management Responder Training course. It contains a strong ICS element and promotes a shared understanding of TIM requirements. SHRP 2’s 2-day TTT course facilitates widespread use of the multidisci- plinary training; the training was shortened to a 4-hour format by the Arizona Department of Public Safety. The training participants include state DOT personnel; federal, state, county, and local emergency response providers; and tow truck operators and contractors (NCHRP Synthesis 468). Interjurisdictional and Interagency Training and Exercises • Arizona. Arizona Division of Emergency Management (ADEM) Training and Exercise Office offers a wide variety of training courses that cover emergency planning, mitigation, awareness, operations, incident command, and domestic preparedness. ADEM also offers emergency response training to ADOT and other agencies for an unlikely accident at the Palo Verde Nuclear Generating Station (NCHRP Synthesis 468).

Emergency Management Training and Exercises 123 • Blue Cascades Exercises. Pacific Northwest Economic Region and the Center for Regional Disaster Resilience bring together public and private partners from the United States and Canada in the Pacific Northwest for a series of scenario-based TTXs (Caltrans is one of the state DOT participants). The goals of the TTXs are to “raise awareness of infrastructure interdependencies and associated vulnerabilities, impacts, and preparedness gaps, identi- fying potential solutions to make needed improvements” (http://www.regionalresilience. org/interdependencies.html). The exercise sequence involved (1) concept identification, (2) workshop, (3) development of materials and scenarios, (4) TTX, (5) After Action Conference, and (6) Final Report. Scenarios have included a major earthquake, flood, pandemic, physical attack, and cyberattack. The exercises result in Regional Action Plans for stakeholders. These plans have included projects and actions in interdependencies, coordi- nation, roles and responsibilities, response, logistics and distribution, information sharing, training and education, public information, and economic continuity and recovery (Caltrans Case Study, NCHRP Report 777). • Tennessee. TDOT has an arrangement with the Tennessee Emergency Management Agency (TEMA) to “exchange” training in which training delivered by TEMA is complimentary to TDOT personnel and vice versa. Interdisciplinary training with the Civil Air Patrol is orga- nized at least once yearly. TEMA’s Technical Hazards Branch provides multiple training and training exercises on radiological emergency response for Tennessee’s two nuclear plants on an annual basis to federal, state, and local responders. Plans are exercised on an annual basis; a federal exercise is also conducted by FEMA and the Nuclear Regulatory Commission yearly at one of the two plants (TDOT Case Study, NCHRP Synthesis 468). • Texas. Texas DOT personnel participate in ICS-300 (Intermediate ICS) and ICS-400 (Advanced ICS) training courses hosted by local fire departments and police departments (NCHRP Synthesis 468). Train-the-Trainer • Arizona. In its first NIMS and ICS rollout, ADOT trained 10 instructors to teach Introduction to ICS and Introduction to NIMS to 4,600 district personnel in a classroom setting (NCHRP Synthesis 468). • Iowa. Iowa DOT used TTT for the IS-100 and IS-200 (ICS for Single Resources) courses. District office and DMV enforcement trainers were trained first; they then trained more than 1,600 personnel over the course of a year (NCHRP Synthesis 468). Computer Simulation • I-95 Corridor Coalition’s three-dimensional, multiplayer computer gaming simulation tech- nology (www.i95vim.com) provides scenario-based, interactive, and real-time incident man- agement training (Virtual Incident Management Training, I-95 Corridor Coalition n.d.). • ON-line eXercise System is a web-based training system for disaster professionals and com- munities. The system can implement TTX, FE, and FSE using the Internet. • FEMA’s EMI offers a series of virtual TTXs focused on disaster training. • The Transportation Emergency Response Application (TERA) is a free, web-based exercise system for the transportation, transit, rail, and airport domains and is compliant with NIMS and ICS and HSEEP. Initially focused on transit scenarios and developed and field-tested under TCRP Project A-36, TERA initial transit scenarios are being modified and expanded with NCHRP funds to include state DOT roles. Additional information regarding TERA and TCRP Project A-36 can be found at www.tera.train-emst.com and in TCRP Web-Only Docu- ment 60/NCHRP Web-Only Document 200, available online at http://onlinepubs.trb.org/ onlinepubs/tcrp/tcrp_w60.pdf (NCHRP Synthesis 468).

124 A Guide to Emergency Management at State Transportation Agencies Classroom Training • Arizona. ADEM and Texas A&M provide emergency management classroom training to ADOT field personnel (NCHRP Synthesis 468). • California. Caltrans provides classroom training, including monthly NIMS, SEMS, ICS, and First Observer training, at its Maintenance Training Academy to new and existing mainte- nance personnel (Caltrans Case Study). • Tennessee. TDOT supervisors are required to take IS-200 (ICS for Single Resources) and IS-800 (Introduction to the NRF) in a classroom setting. Managers are required to take ICS-300 (Intermediate ICS) and ICS-400 (Advanced ICS) delivered through classroom training as well (NCHRP Synthesis 468). Examples of exercise implementation by exercise type are provided as follows: Workshops and Seminars • California. Caltrans uses workshops to deliver all-hazards training and COOP/COG training. Each year, Caltrans holds four All-Hazards Training Workshops (one per quarter) and three COOP/COG Workshops (Caltrans Case Study). • Texas. Texas DOT’s emergency response providers and key district staff members, including district engineers and M&O directors, participate in various workshops and seminars, including an annual hurricane preparedness workshop. The 2013 workshop covered evacuation, reen- try, cleanup, and response techniques. The following were also covered: protocols for the suspension of construction schedules; radio communications and interoperability, debris and environmental contracts; MAP-21; FHWA Emergency Relief and FEMA Public Assis- tance reimbursement; volunteer management; and the Maintenance Management System (NCHRP Synthesis 468). • Vermont. Six discussion-based exercises (seminars or workshops) a year focusing on VTrans specific threats and hazards are attended by VTrans field personnel (NCHRP Synthesis 468). Tabletop Exercise • California. Caltrans participates in the annual CalEMA Golden Guardian Executive Table Top Exercise (TTX) held one month prior to the actual CalEMA Golden Guardian FSE (Caltrans Case Study). • Arizona. ADOT’s districts hold TTXs that relate to ADOT’s statewide exercises (NCHRP Syn- thesis 468). • Tennessee. TDOT conducts TTXs with representatives of tasked organizations to help its EOP. • Multiple DOTs. The I-STEP Highway and Motor Carrier AASHTO Peer Exchange Tabletop Exercise for state DOTs held at the 2012 Transportation Hazards and Security Summit and Peer Exchange highlighted regional prevention, protection, and response practices. The TTX scenario was a terrorist attack against critical infrastructure and an attack against a critical bridge coinciding with a natural disaster (NCHRP Synthesis 468). Drills • California. Caltrans holds drills to evaluate personnel, technologies, and equipment; and for the development of plans and procedures. Caltrans’ personnel undergo monthly drills to ensure that they can mobilize the technologies and equipment. During the drills, any issues with the systems will be flagged. Monthly drills are held on Caltrans Microwave Telephone and Fixed Satellite Telephone and Sat Com Auxiliary Radio Systems, which is a satellite com- munications system owned, managed, and operated by Caltrans for the purpose of emergency communications within Caltrans and with other agencies (Caltrans Case Study).

Emergency Management Training and Exercises 125 Functional Exercises • Iowa. Iowa DOT holds three to four regional TTXs annually (NCHRP Synthesis 468). • Missouri. Half of Missouri DOT’s exercises are FEs; typically, 10% or more of the field personnel are involved in these exercises. Scenarios have included earthquakes; severe weather, including snow, ice, and tornadoes; situations involving nuclear power plants; and terrorism (NCHRP Synthesis 468). Full-Scale Exercises (FSEs) • Arizona. ADOT personnel, including field, emergency preparedness and management, and communications personnel, participated in a 2011 statewide exercise on an improvised explosive device explosion (NCHRP Synthesis 468). • California. Caltrans participates in the statewide Great California Shakeout Interagency Exercise organized by the Earthquake Country Alliance (Caltrans Case Study). Caltrans also participates in the annual CalEMA Golden Guardian FSE. The FSE is preceded by planning meetings and a TTX, a Hot Wash, and an AAR meeting are held afterward (Caltrans Case Study). In addition, Caltrans also incorporates and evaluates technologies, such as Caltrans Microwave Telephone and Fixed Satellite Telephone and Sat Com Auxiliary Radio Systems, into its FSEs. • Texas. Texas DOT organizes and hosts at least one FSE each year, usually focusing on contra- flow evacuations, in which personnel and equipment are mobilized, and the process is timed and evaluated. Additional participants have included the Texas Department of Public Safety, the fire marshal, and local law enforcement agencies (NCHRP Synthesis 468). • Vermont. VTrans field personnel undergo 4- to 5-day FSEs organized by the state EOC. The scenarios are for hurricanes and WMDs (NCHRP Synthesis 468). Training Evaluation U.S. Office of Personnel Management (OPM) regulations require federal agencies to create and demonstrate the value of their training programs to their missions through training evalu- ation. The regulations require the implementation of a training evaluation system that helps the agency determine future investments in training and development. The evaluations should determine whether learning has occurred; whether learning was applicable to job performance or other behaviors affecting results; whether the learning was applied to the employee’s job; and, if so, whether there was positive impact on performance or other job-related behaviors. The 2011 U.S. Office of Personnel Management (OPM) Training Evaluation Field Guide uses the regulations as a foundation for the guide. The 2011 OPM Guide stresses the importance of ensuring that training positively affects agency mission and outcomes through evaluation. The 2011 OPM Guide uses the New World Kirkpatrick Four Levels to offer a structured way for agencies to evaluate their training and training programs. The original Kirkpatrick training evaluation method comprises four levels: • Level 1: Reaction. Trainees provide their reactions to the instructor or facilitator regarding training subject/content, facilities, schedule, and improvements using a feedback form. This is probably the easiest level to implement. • Level 2: Learning. Determines the extent to which trainees have learned the training material. What the trainee knew before the training (pre-test) is compared with what the trainee has learned from the training (post-test). • Level 3: Behavior. Trainees’ application of training to their jobs is evaluated by observation of the trainees during work or during a drill.

126 A Guide to Emergency Management at State Transportation Agencies • Level 4: Results. Demonstrates whether the training is having the desired outcome (e.g., faster response to an incident). It provides information on the effectiveness of a training program. Outcomes need to be measured before and after the training. The new Kirkpatrick method recommends using the four levels in reverse: starting with level 4 and proceeding to level 1. Also, to prove the value of training, it is necessary to conduct both quantitative and qualitative evaluations for each level and to provide evidence of the connection between level 4 and levels 1, 2, and 3. The new Kirkpatrick method also adds the following to each of the levels: • Level 1: Engagement (to what degree participants are involved and interested in the learning intervention) and Relevance (to what degree the content of the learning intervention is appli- cable to the jobs of the participants). • Level 2: Confidence (to what degree training participants feel confident to apply the newly obtained knowledge and skills on the job) and Commitment (to what degree training partici- pants commit that they will apply newly obtained knowledge and skills on the job). • Level 3: Required Drivers (process and systems that reinforce, monitor, encourage, and reward performance of critical behaviors on the job) and On-the-Job Learning (ongoing self- education on the job by the training graduate). • Level 4: Leading Indicators (short-term observations and measurements that suggest that critical behaviors are on track to create a positive impact on the desired results). The new method stresses the importance of monitoring behaviors and required drivers to ensure they are being applied on the job; and, monitoring leading indicators to determine whether correct behaviors have been chosen. The 2011 OPM Guide notes the distinction between effective training and training effec- tiveness. Effective training fulfills levels 1 and 2 expectations. Training effectiveness relates to levels 3 and 4, application of the training to the employee’s job, which produces results that contribute to the agency’s mission. State transportation agencies should refer to the 2011 OPM Guide as a resource when devel- oping training evaluations. It contains case studies and sample tools, including participant surveys, appropriate for each of the four levels and observation checklists for levels 2 and 3. In addition, the 2011 OPM Guide provides a brief summary of additional evaluation tools that may be of use.