National Academies Press: OpenBook

Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach (2013)

Chapter: Module 4 - Engage in Continuous Improvement via Benchmarking

« Previous: Module 3 - Improve Image Management to Become an Employer of Choice
Page 130
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 130
Page 131
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 131
Page 132
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 132
Page 133
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 133
Page 134
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 134
Page 135
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 135
Page 136
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 136
Page 137
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 137
Page 138
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 138
Page 139
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 139
Page 140
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 140
Page 141
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 141
Page 142
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 142
Page 143
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 143
Page 144
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 144
Page 145
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 145
Page 146
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 146
Page 147
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 147
Page 148
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 148
Page 149
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 149
Page 150
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 150
Page 151
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 151
Page 152
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 152
Page 153
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 153
Page 154
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 154
Page 155
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 155
Page 156
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 156
Page 157
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 157
Page 158
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 158
Page 159
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 159
Page 160
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 160
Page 161
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 161
Page 162
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 162
Page 163
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 163
Page 164
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 164
Page 165
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 165
Page 166
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 166
Page 167
Suggested Citation:"Module 4 - Engage in Continuous Improvement via Benchmarking ." National Academies of Sciences, Engineering, and Medicine. 2013. Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach. Washington, DC: The National Academies Press. doi: 10.17226/22489.
×
Page 167

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Building a Susta Workforce in the Transportation In A Systems App inable Public dustry— roach Module 4. Engage in Continuous Improvement via Benchmarking

4-1 Module 4 Contents Module 4: Engage in Continuous Improvement via Benchmarking 4.1 The Benchmarking Process 4-4 4.2 Phase 1: Planning 4-5 4.2.1 Identify Unit of Analysis for Benchmarking Study 4-6 4.2.2 Identify Comparative Organizations 4-9 4.2.3 Select Performance Measures 4-11 4.2.4 Identify Data Collection Methods 4-12 4.2.5 Overall Benchmarking Planning Phase 4-12 4.3 Phase 2: Analysis 4-14 4.3.1 Collect Data Internally and Externally 4-15 4.3.2 Measure and Compare Current Performance Gap 4-16 4.3.3 Identify Effective Practices to Close the Gap 4-17 4.3.4 Overall Benchmarking Analysis Phase 4-20 4.4 Phase 3: Integration 4-20 4.4.1 Communicate Benchmarking Findings to Gain Acceptance 4-20 4.4.2 Develop an Action Plan 4-22 4.4.3 Overall Benchmarking Integration Phase 4-23 4.5 Phase 4: Action 4-23 4.5.1 Implement Specific Actions and Monitor Progress 4-23 4.5.2 Overall Benchmarking Action Phase 4-24 4.6 Phase 5: Maturity 4-25 4.6.1 Close Performance Gap 4-25 4.6.2 Integrate Practices Fully into Processes 4-26 4.6.3 Overall Benchmarking Maturity Phase 4-27 4.7 Combined Process Map 4-27 4.8 Additional Tools to Aid in Benchmarking 4-27 4.8.1 Phase 1 (Planning) Tools 4-27 4.8.2 Phase 2 (Analysis) Tools 4-31 4.8.3 Phase 3 (Integration) Tools 4-32 4.8.4 Phase 4 (Action) Tools 4-34 4.8.5 Phase 5 (Maturity) Tools 4-34 M o d u l e 4 Engage in Continuous Improvement via Benchmarking

4-2 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach This module presents information on a systematic process—benchmarking—for public trans- portation organizations to use to gather examples of effective workforce development prac- tices used by similar organizations. This process of data gathering may also be used to compare workforce development practices across the public transportation industry and with non-peer industries. By approaching benchmarking in the detailed way outlined in this module, public transportation organizations will be able to gather meaningful data from other organizations and identify features and nuances of practices that have led to success for other organizations. Table 4-1 indicates the location of practical content in this module. This module includes: • Discussion of what benchmarking is. • Information on when a benchmarking study may be needed. • An overview of the benchmarking process. • Step-by-step information for conducting a benchmarking study. • Case studies to show how each phase can be implemented. • Tools that can be used throughout benchmarking to make the data gathering processes easier to implement. While best practices and effective strategies for recruitment, retention, training and develop- ment, and professional capacity building have been presented in previous modules, public trans- portation organizations are often evolving and dealing with change, so even a few months after Key Content Exhibit Location Process Maps Practical Tools Benchmarking Phase 1: Planning Ex. 4-10 Ex. 4-7 (Sample Criteria for Partners) Ex. 4-8 (Types of Partners ) Ex. 4-9 (Data Collection Methods) Ex. 4-20 (Partner Selection Tool) Ex. 4-21 (Metric Identification) Benchmarking Phase 2: Analysis Ex. 4-13 see Module 2 (Metrics Scorecards) Ex. 4-22 (Interview Guidelines) Ex. 4-23 (Gap Analysis Table) Benchmarking Phase 3: Integration Ex. 4-15 Ex. 4-24 (Example Communication Plan) Ex. 4-25 (Data Fields for Strategic Action Plan) Ex. 4-26 (Example Action Plan) Benchmarking Phase 4: Action Ex. 4-17 Ex. 4-27 (Monitoring Chart) Ex. 4-28 (Stakeholder Scanning) Benchmarking Phase 5: Maturity Ex. 4-19 Ex. 4-29 (Focus Group Protocol) Ex. 4-30 (Checklist of Risks/Consequences) Table 4-1. Content location of Module 4.

Module 4: engage in Continuous Improvement via Benchmarking 4-3 implementing a program, things are rarely the same as when the decision was originally made to implement the program. Addi- tionally, the environment that surrounds public transportation (e.g., service delivery needs, the labor market, technological advances) is not static. This means that to remain competi- tive and establish a sustainable workforce, transit organizations need to be flexible and proactive. Therefore, it is important that transit agencies are able to continually evaluate their practices to ensure successful outcomes. To do this, organizations can study other successful organizations to identify what makes them successful. Implementing a benchmarking process enables a public transportation organization to study other like organiza- tions and gather insights from successful human resource man- agement and workforce development practices. This allows a transit agency to evaluate practices that are already in place to ensure continuous improvement within its organization. Defining Benchmarking According to Bogan and English (1994, p.1), organizations will “perennially reinvent the wheel” if they fail to systematically analyze the business processes, workforce practices, and strate- gies that have made other organizations successful. The process of identifying the innovative or effective practices of other orga- nizations is referred to as benchmarking. For benchmarking to be effective, public transportation organizations should engage in a continuous improvement process that includes evaluating the current operational and administrative practices and bench- marking workforce initiatives of public transportation organi- zations that have a similar performance profile. It is important to distinguish between the term benchmarking, which is often used in the context of conducting a benchmarking study, and the term benchmarks. A benchmark is the operating statistic or measurement used to compare a public transportation organi- zation’s performance on a particular dimension (e.g., retention of staff) with that of another organization or the public trans- portation industry. Benchmarks indicate the extent to which an organization is deficient or superior in a particular process area. A benchmarking study helps point organizations to the causes or drivers of deficiencies beyond simply showing where gaps exist (Bogan and English, 1994). The phrase to benchmark is sometimes used to refer to the process by which an organization compares processes, functions, or standards to those of another organization based on data col- lected through a benchmarking study. These distinctions, as well as other definitions that will be beneficial for understanding and implementing a benchmarking study, are provided in Exhibit 4-1. Is a Benchmarking Study Needed? Once the purpose of a benchmarking study is understood, a decision can be made as to whether a benchmarking study is needed within a transit agency. Self-reflection questions can be Benchmarking Highlights ¾Why Benchmark? • For continuous improvement to identify effective transit practices. • To remain competitive in the industry. • To identify strategies that can be easily translated into effective practices through systematic data collection. • To make investment decisions based on tested approaches. Image Management Benchmarking ¾When Should Benchmarking Be Used? • Any time there is a need to identify innovative strategies. • As part of an evaluation to see how the outcomes of a practice used by one transit organization measure up to the outcomes realized by another. • When the dynamics of the public transportation industry are changing and new workforce develop- ment practices are needed. ¾Who Should Use Benchmarking? • All public-transportation–related organizations, including transit systems, transit contractors, and transit vendors, can use benchmarking. ¾What Is the Most Important Message of Benchmarking? • Proper preparation for a transit benchmarking study is crucial. For example, partner selection in benchmarking may affect how useful the informa- tion collected is to the sponsor organization.

4-4 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach used to help decide if a benchmarking study should be conducted. These questions are presented in Exhibit 4-2. If the answer to one or more of these self-reflection questions is yes, a benchmark- ing study would likely be beneficial to your transit organization. The remainder of this module describes the five phases of the benchmarking process. 4.1 The Benchmarking Process While many models have been presented to describe the benchmarking process, the model employed by Xerox Corporation in the 1970s is arguably one of the most comprehensive and timeless. In describing this model, Camp (1989) explains that the benchmarking process consists of five phases: planning, analysis, integration, action, and maturity. For benchmarking efforts to Benchmarking Terms Benchmarking: The process of gathering and evaluating practices from another organization to compare to or refine one’s own set of practices. Benchmark: The operating statistic or measurement used to compare an organization’s performance on a dimension with that of another organization or the industry. To benchmark: The act or process by which an organization compares processes, functions, or standards to those of another organization. Benchmarking partner: The department, organization, or industry being used as the comparison in a benchmarking study. Sponsor: The individual, department, or organization conducting the benchmarking study. Inputs or Enablers: Methods or behaviors that facilitate the implementation of a best-in-class practice. Examples are resources such as people, technology, and financial assets. Terms Used in This Guidebook Organizational processes: Specific transit organizational processes that are the focus of this guidebook are retention, recruitment, training and development, and professional capacity building. Strategies: Specific initiatives that fall under each of the organizational processes. Practices or programs: Specific initiatives that public transportation organizations adopt to respond to a particular need or challenge (e.g., public relations campaign to change the community’s view of transit’s role and contribution to quality of life). Organizational outcomes: Results achieved or desired that are relevant to the public transportation organization (e.g., hiring new workers, training programs, employee turnover). Services: Those services offered by the transit agency (e.g., rail, fixed-route bus, paratransit). Exhibit 4-1. Glossary of terms. Question 1: Does our organization need to engage in strategic thinking to update organizational processes? For example, is our transit agency still using antiquated methods to reach potential candidates for jobs? Question 2: Could our public transportation organization’s approach to workforce development be improved, possibly from updating existing strategies or implementing new strategies? Question 3: Are there human resource management approaches used by transit industry peers that are not implemented or fully utilized in our organization (e.g., using interactive forums to provide workers a voice and encourage retention)? Question 4: Could our transit organization benefit from a better understanding of peer agencies, the dynamics of the public transportation industry, and how we compare to others in the industry? Question 5: Do resource constraints require our transit organization to think creatively about recruiting, developing, and retaining quality workers? Exhibit 4-2. Questions to determine necessity of benchmarking.

Module 4: Engage in Continuous Improvement via Benchmarking 4-5 be successful, the organization is advised to view each of these phases as critical to completing a benchmarking study. These specific phases and the 12 adapted steps that make up the phases are described in more detail in the following sections. The five phases of the benchmarking process, as well as the steps involved in each phase, are identified in Exhibit 4-3. This exhibit displays the information that is provided in the subsequent sections of this module and can be used as a road map to follow when implementing a bench- marking study. 4.2 Phase 1: Planning The planning phase of the benchmarking process involves the initial preparations for begin- ning a transit benchmarking study. Benchmarking can focus on various activities, such as all of the processes that make up workforce development for a transit organization, a particular human resource management function, or a specific transit service delivery mode. Therefore, the focus of the benchmarking study should be determined during the planning phase. Exhibit 4-4 provides an overview of the important questions that will be answered during Phase 1. The decisions made during this phase set the stage for all of the data that will be collected during the benchmarking study: what will be collected, from whom, and how the data will be gathered. This step helps to ensure that the data collected will be valuable to the transit benchmarking sponsor and that the data will help the sponsor improve desired areas. The purpose of this step is to ensure that the benchmarking process is relevant to and useful for the transit benchmarking sponsor. The steps that make up the planning phase of a benchmarking study are: • Identify unit of analysis for benchmarking study, • Identify comparative/potential partner organizations, Exhibit 4-3. Overview of five major benchmarking phases and related steps.

4-6 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach • Select performance measures, and • Identify data collection methods. Each step is discussed in the following sections. 4.2.1 Identify Unit of Analysis for Benchmarking Study A key first step in planning a benchmarking study is to identify the unit of analysis or level at which the benchmarking study will be conducted. For example, it is important to consider whether there is interest in studying generalized programs in place across the transit industry, an entire system of workforce practices within a transit organization, or the practices within a particular functional area of a transit organization (e.g., human resource management, civil rights, organizational development). Defining a specific unit of analysis allows the sponsor transit organization to collect the appro- priate data regarding specific processes and supporting practices that contribute to the desired output. It is important to note that while processes discussed in this integrated guidebook mainly pertain to workforce development, processes related to operational functioning, such as service design, vehicle maintenance, or emergency response, could also be the focus of a benchmarking study. Certainly, it can be argued that such processes could affect the perceptions of passengers and community stakeholders and have implications for building a sustainable workforce by influencing recruitment efforts within the transit service area. Given that processes are driven by day-to-day practices, benchmarking studies typically focus on identifying the components and steps involved in an individual transit organization’s pro- cesses. Through a series of deductive steps, a process is determined to be responsible, at least in part, for exemplary performance (i.e., desired outcome) of the public transportation organiza- tion. A process involves repeated steps that are performed within a particular sequence, translat- ing input into output to generate value for the consumer (D’Amelio, 1995). The way in which this organizational process is performed or the method used (the how of a process) is often referred to as a practice. A benchmarking study should clarify the practice(s) used to perform the work process in order to answer the question: How is this process conducted? (D’Amelio, 1995). In the planning phase of the benchmarking study, the transit sponsor should outline ques- tions to clarify, such as, How is success of this transit practice measured? This question refers to the performance metrics used by the transit organization. Another question to consider is, What resources and behaviors, including the people, technology, and other assets, enable the conduct of this work practice? These are referred to as the enablers or inputs. To gather this information, a sponsor agency targets the practice owner for participation in the benchmarking study. The practice owner is the transit employee or group of employees responsible for implementing the steps involved in a particular workforce process. This information gathering is one of the first steps of organizational process mapping. The first phase of the benchmarking process—planning—will help answer the following important questions: • What are we going to benchmark? • Who should we compare ourselves against? (Who are the benchmarking partners?) • What methods will we use to conduct the benchmarking study? Exhibit 4-4. Important questions to be answered in Phase 1.

Module 4: engage in Continuous Improvement via Benchmarking 4-7 Exhibit 4-5 illustrates the relationship between practices, processes, enablers, and outcomes. To understand the systematic steps involved in a process, a transit benchmarking sponsor should plan to conduct process mapping. The result of process mapping is typically a workflow diagram that helps to clarify the practices or steps involved in a process or series of parallel pro- cesses. Ahoy (1999) describes the steps to develop a process mapping flowchart: 1. Determine the boundaries. This refers to where the process begins and ends and which other processes feed into the process. 2. List the steps. The process map developer should determine the desired level of specificity in the flowchart. If a benchmarking sponsor is seeking to mimic the process to achieve a specific outcome, typically the process mapping will detail every decision point and finite action. Each step in the process mapping should begin with a verb to illustrate an action to be taken. 3. Sequence the steps. Once the steps have been documented, the process map developer typi- cally moves the steps into their proper sequential location in the process flowchart and includes arrows or lines to show the relationship and sequence between steps in the process. 4. System model. A complete flowchart would illustrate how a series of processes or parallel processes work together to achieve an outcome, with specific symbols for different types of steps (see Exhibit 4-5). Thus, the final chart may include multiple flowcharts to illustrate timelines and sequencing. Contextual factors that should be included in the flowchart are the inputs (people, machines, technology, and other materials), outputs (desired outcome or results), controls (rules followed by the best-in-class partners), and feedback (information obtained via data collection). 5. Check and finalize the flowchart. This includes ensuring that all of the information in this chart, such as dates and titles, is accurate. This final development step may also include col- lecting additional data (e.g., follow-up interviews) to determine not only whether the pro- cess steps have been accurately documented but also whether the process is being conducted correctly. The flowchart developer should document any recommended deviations so that process improvements can be noted. Guidance to create a process map is provided in Exhibit 4-6. The first column is a visual depiction of the types of information included in a process map. The second column presents an example process map depicting the recruitment practice of creating an online, video-based, realistic job preview. Exhibit 4-5. Overview of the organizational process.

4-8 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach It is important that the data collector conducting the benchmarking study be intimately famil- iar with the practices, processes, outcomes, and related enablers within his or her own transit organization to guide the type of data collected and help define the questions that need to be asked of the benchmarking partner to determine how the processes might be transferable to the sponsor transit organization. Prioritize outcomes. While a transit benchmarking sponsor may be interested in study- ing the processes that drive a number of successful outcomes, it is important that the sponsor prioritize the outcomes for focus. The transit sponsor needs to focus its interests to conduct a thorough study of a particular outcome. This focus allows the sponsor transit organization to effectively prepare itself in terms of structure, culture, and employee engagement for trans- formation that will likely occur as a result of adopting new practices and processes. Focus- ing on multiple areas of interest can distract the transit organization in its study by straining resources, confusing data collection, and segregating the organization in terms of the changes its employees will be willing to accept and implement. Thus, a sponsor transit organization should identify priorities in terms of which metrics and desired outcomes are most important and target these first. Elements of a Process Map Defined Example Process Map for Recruitment Strategy (Implement RJPs) Source: Ahoy, C., 1999. Exhibit 4-6. Overview of the benchmarking process.

Module 4: engage in Continuous Improvement via Benchmarking 4-9 To summarize, following are steps to identify the unit of analysis of a benchmarking study. These steps are presented in sequential order and can be followed to efficiently determine the focus of a benchmarking study. • Determine transit functional areas or processes of focus. • Outline questions to clarify (e.g., How is the success of this organizational process measured?). • Develop process map(s) of in-house practices to fully understand how the process(es) being benchmarked will fit in. • Ensure that the data collector is familiar with the practices, processes, and enablers to help define necessary questions to be asked of benchmarking partners. • Prioritize outcomes to ensure that resources are not strained and desired results are reached. 4.2.2 Identify Comparative Organizations An important component of designing a benchmarking study is selecting the organization(s) to use for comparison. In transit organizations, this is a particularly important issue since there are many factors that can influence performance. These can include the number of transit employ- ees, the size of a transit agency’s vehicle fleet, the service area (e.g., urban, small urban, rural), and the type of transit service provided (e.g., rail, fixed-route bus, paratransit/demand-response service), among other factors. The choice of entities to benchmark against can be driven by a variety of criteria, including a desire to compare against a similar type of transit organization, the similarity in demographics of the population served, the size and geography of the transit service area, and the availability of willing benchmarking partners. Frequently an organization will jump to benchmarking with another organization due to the perceived innovativeness or success of the potential benchmarking partner’s practices with little regard for the environ- ment in which the potential benchmarking partner operates. While much can be gained from studying innovative practices, there may also be barriers to implementing these same practices, particularly for more resource-strained or smaller transit agencies. For example, a transit agency in the upper Midwest may face very different challenges in recruiting a diverse workforce than a transit agency in the Northeast, where diversity may be a defining characteristic of the local community. Further, a large transit agency may have the resources to maintain a fully staffed human resources department, whereas the sponsor organization may have to pursue recruit- ment through a supervisor or manager who is tasked with other, non-HR duties. This speaks again to the value of defining not only the unit of analysis but also to being strategic when iden- tifying a benchmarking partner organization. It is important that the transit sponsor organiza- tion list desired target benchmarking partners and then create a clear outline of the similarities and differences of the profiles of those potential benchmarking partners. In addition, factors such as population served, transit services offered, operating budgets, transit agency size, and functions carried out in-house versus functions contracted out should be considered in creating these profiles. These profiles also help ensure that data gathered in the benchmarking study can be effectively translated into information and practices that have utility for the sponsor transit organization. Further, if a transit organization decides to benchmark with a transit agency that is very different from itself, profile mapping can help to inform the types of questions the sponsor transit organization should ask of a benchmarking partner organization(s). These profile ques- tions will also help determine the action(s) the sponsor needs to take to modify a benchmarking partner’s practice to make it work within its own organization. Examples of criteria by which profiles could be created are contained in Exhibit 4-7. By creating a profile of comparative organizations in advance, the sponsor transit organization can also begin to refine its list of potential benchmarking partners. The organizations deemed as benchmarking partners are those willing to participate in the study. They are referred to as partners because it is important for the participants in a benchmarking study to see their role

4-10 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach as valuable and to experience a benefit of participation, such as learning about other industry practices. In fact, as an incentive to participation, after study completion it is recommended that a sponsor transit organization share some of the valuable data gathered by offering preliminary reports or informational sessions to benchmarking partners. While benchmarking typically compares a sponsor transit organization against industry peers, some transit agencies may employ contractors or have administrative or operational units that function independently of each other. Thus, internal benchmarking (e.g., comparing the human resource practices of different internal units or comparing internal human resources practices with those of a contractor) can unearth new information. Further, benchmarking can go beyond public transportation industry peers, allowing access to a larger universe of leading companies and innovative practices from non-transit industries. Various types of benchmarking partners, as well as an example of each type of benchmarking study, are provided in Exhibit 4-8. In summary, the steps to follow in identifying appropriate benchmarking partners are: • Strategically identify potential benchmarking partners by creating profiles of comparative organizations that may be targeted for participation in the study. Many features of a transit agency (e.g., size, population served, geographic location, transit service delivery mode, avail- able financial and staff resources) can influence the way the transit organization addresses workforce development. • Select the type of partners to target in a benchmarking study (see Exhibit 4-8 for partner types). • Transit rail • Bus rapid transit • Bus fixed route • Bus deviated fixed route • General public demand response • ADA paratransit • Vehicles operated in maximum service by mode • Total number of transit employees • Service area population density services delivered internally versus services contracted out • Functions performed in- house versus functions outsourced • Annual total passenger trips by mode • Annual total vehicle revenue miles by mode • Annual total vehicle revenue hours by mode • Cost per revenue vehicle mile/cost per revenue vehicle hour • Cost per passenger trip • Annual total operating expense • Percentage of budget dedicated to workforce development • Geographic region • Size of service area in square miles Exhibit 4-7. Sample criteria for creating profile of comparison organizations. Type of Partner Definition Example Internal Compares best practices within separate departments or units of an organization Comparing a retention strategy in one division of a large transit organization with the retention strategies used in other divisions to compare their performance Competitive Compares best practices with direct competitors Comparing a public transit provider with a private, for-profit provider Industry Comparisons made to peers in the same industry Comparing a transit organization’s retention to the retention of another transit agency in a similar environment Functional Compares business functions between dissimilar industries Comparing training and development programs between a transit organization and an express package delivery service company Generic Compares similar processes in significantly different types of organizations Comparing a training and development practice in a transit organization to training and development practices in the military Exhibit 4-8. Types of benchmarking partners.

Module 4: engage in Continuous Improvement via Benchmarking 4-11 Sample case study – selecting a benchmarking partner. A mid-sized urban transit bus sys- tem in the Midwest that provides internally delivered fixed-route and ADA paratransit services has been experiencing difficulties recruiting mechanics over the past 2 years. It has not had a problem getting appropriately skilled applicants to fill management and technical positions, and always has a large pool of qualified applicants for bus operator vacancies. However, staff- ing its in-house bus maintenance function with mechanics that have the knowledge, skills, and abilities to perform required preventive and corrective maintenance activities has become a challenge. Its maintenance recruitment efforts have fallen well short of the mark, and mainte- nance positions are often vacant for a considerable period before they are filled. At times, the transit agency had to hire less-than-desirable applicants simply to keep from falling behind on its bus maintenance activities and continue to have the required number of vehicles available for daily pullout. The human resource and maintenance managers jointly decided that it would be advantageous to engage in a benchmarking partnership with a peer transit agency to identify new and more successful mechanic recruiting strategies. To identify a transit organization that might make a good partner in a benchmarking study, the agency first determined the criteria to use in choosing a transit system: • Is located in the Midwest—based on the need to choose an organization with similar demo- graphics and regional culture. • Provides in-house delivery of bus fixed-route services and ADA paratransit. • Operates approximately the same number of vehicles in maximum service by mode. • Has a similar total number of employees. • Primarily performs its preventive and corrective maintenance in-house. • Has a similar number of annual vehicle revenue miles and vehicle revenue hours. After establishing these criteria, the human resource manager used the following sources to identify transit systems that met the criteria: • National Transit Database (NTD). • FTA regional office. • APTA. Using information from these sources, the transit agency was able to identify three transit sys- tems. The human resource manager and the maintenance manager then conducted conference calls with each of the transit systems. Based on these discussions and the level of success that each transit system was experiencing in recruiting mechanics, the managers selected the transit system that clearly represented the best opportunity to identify practices with transferability. A benchmarking partnership was negotiated and entered into, and performance measures that would be included in the benchmarking study were jointly selected. 4.2.3 Select Performance Measures When undertaking a benchmarking study, performance measures or metrics can be used to compare organizations and the outcomes of their processes. This step involves selecting the specific metrics that will be used in the benchmarking study. Metrics for various organizational processes are provided in the scorecards in Module 2. As an example, a transit organization may want to examine a metric such as the time to fill an open job position. For agencies trying to reduce the cost of filling job vacancies, this would be important to include in a benchmark- ing study since this metric could help to determine the effectiveness of a transit organization’s recruiting efforts.

4-12 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach The recruitment scorecard provides anchors to evaluate organizational processes on this time to fill an open position metric. The scorecard also provides the format used to report the metric. For example, this metric is reported in terms of the average time to fill positions. Thus, perfor- mance on this metric would be measured by calculating the average time required to fill all open positions within the target period. To summarize, the steps in selecting performance measures for a benchmarking study are: • Examine potential metrics, such as those provided in the scorecards in Module 2, and • Determine metrics to include in the benchmarking study. 4.2.4 Identify Data Collection Methods A comprehensive benchmarking study should include a detailed data collection plan. This plan is a document that describes existing data and records the sponsor transit organization intends to obtain, the original data to be collected, and the timeline, methods, and contacts involved in collecting the data. The sponsor transit agency should thoroughly research data on any benchmarking partner that is publicly available prior to requesting data from the orga- nization. By reviewing existing data first, the sponsor agency will have a better sense of what information is available, what is still needed, and who the process owners are in the partner organization. This helps to reduce the demands on the benchmarking partners, to ensure that the sponsors are well prepared, and to streamline data collection in terms of time and resource requirements. While it is helpful to identify the metrics used by benchmarking partner organizations to make noteworthy comparisons, the focus of the benchmarking study should be on identifying the effective practices that allow an organization to meet specific targets and become an indus- try leader. Thus, it is recommended that a benchmarking sponsor use multiple data collection methods and sources to collect data. The purpose is to ensure that information collected is com- prehensive and deemed accurate because it is verifiable across various sources. Additionally, the means of data collection should not be based solely on convenience but rather on the purpose of the data collection and type of data desired. The table in Exhibit 4-9, adapted from D’Amelio (1995), provides an overview of different types of data collection methods that may be used, the advantages and disadvantages of each, and the resource considerations. This exhibit also includes example sources for collecting data. As a review, the planning phase should conclude with a detailed data collection plan that lists all the methods of data collection that will be used. Using multiple data collection methods ensures that comprehensive information is collected. In addition, it is important to research data that are publicly available on any benchmarking partner prior to requesting data from the organization. This helps to ensure efficiency in collecting data and identifying key persons to contact within the partner organization (i.e., the process owners). Further, it reduces demand on benchmarking partners. 4.2.5 Overall Benchmarking Planning Phase Following each of the steps described within this phase will aid in planning a successful bench- marking study. In addition to creating a process map for each process or series of practices to be studied, it is valuable to articulate a process for the conduct of the benchmarking study itself. The process map displayed in Exhibit 4-10 illustrates a model for planning a benchmarking effort using the process and tools provided in this section.

Module 4: engage in Continuous Improvement via Benchmarking 4-13 METHODS Existing Data Review Survey Interview Focus Group Site Visit Definition Analysis and interpretation of information that exists in-house or in public domain Questions sent to partner organization (multiple choice, open-ended, forced choice, scaled) A one-on-one meeting with a partner conducted either face-to-face or via telephone Multiple participants with similar profiles (job titles, rank) who are convened to respond to questions from a facilitator An on-premises meeting at the partner’s facility; combines interview with observation When to Use Before engaging partner organization Need to gather same info from multiple sources; need representative sample to verify data To gather data at a detailed level; to probe stakeholders on particular topics of interest Means for gathering data from multiple sources at once; can be used to reach consensus when there are discrepancies in opinion If observing practices would be valuable (e.g., production line); may be used to help identify enablers Advantage Being prepared/ focused in data collection Permits extensive data gathering; analysis performed by computer; easy to gather large sample and amount of data; phone interviews inexpensive and shorter, easier to complete than interviews Provides flexibility by allowing for probes/follow-up; allows interviewee to directly question an individual; helps to obtain buy-in from key stakeholders Can encourage creative thinking and brainstorming; interaction of participants may reveal new concepts or ideas; can use to collect data from multiple partner organizations, which can be mutually beneficial for partners and facilitator Can observe to verify practices, processes, enablers, and metrics Disadvantage Finding the appropriate information can be time-consuming. Response rates may be low (tough to engage participants); may have confusion about terminology; creative ideas will not be captured; difficult to probe for depth; may be difficult to reach participants via phone for phone surveys; phone surveys are time- consuming because multiple persons have to be called. Can be time consuming; interviewees may be reluctant to dedicate an hour to an hour and a half (the typical length of an interview); may need to redirect interviewee to keep him/her on topic. May be nonproductive to gain multiple perspectives, particularly when gathering specific factual steps or details about a particular process used Requires travel and coordinating schedules (may affect multiple persons at site); typically requires more time than other methods Example Sources Internet; internal human resource reports; financial reports; sustainability reports; journal (trade, peer reviewed); news, magazine articles Web-based; paper-pencil; e- mailed; phone In person; phone- facilitated (use Internet meeting software—e.g., WebEx/LiveMeeting) At client site or conference facility near client In person Expertise Required Thorough researcher ability; knowledge of sources Analyst; web survey expert; item development expertise; survey administrator; phone surveyors need interview skills. Knowledge of Internet meeting software; knowledge of interview protocol development; skill in conducting interviews and encouraging participation/open responding; need a note- taker and interviewer Facilitator skills require a unique skill set (different from interview skills)— require ability to manage multiple participants and encourage participation from multiple personality types at once; need a note taker and facilitator. Listening skills; technical expertise of processes observed; ability to ask questions and take notes Cost/Time Needed Low to no cost; time consuming Development can be costly. Low cost unless travel required; need to account for interviewer practice time and protocol development time Can be costly, especially if arranging travel for participants to meet centrally; extensive preparation required Travel expenses; extensive preparation needed to arrange logistics Exhibit 4-9. Possible data collection methods.

4-14 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach 4.3 Phase 2: Analysis The analysis phase includes initiating data collection and assessing findings. To adequately analyze how practices and processes from partner organizations may be used to enhance a spon- sor transit organization’s processes and practices, it is essential to not only collect data from the benchmarking partner organization, but also from within the sponsor transit agency itself to know what processes and enablers currently exist. Exhibit 4-11 provides an overview of the important questions to be answered in this phase. It is important to understand not only what best practices are being implemented within the partner organization(s), but also where the greatest performance gaps exist between the partner organization and the sponsor transit organization and which practices and processes will be most effective in addressing those discrepancies. The analysis phase involves the following steps: • Collect data internally and externally. • Measure and compare current performance gap. • Identify effective practices to close the gap. These steps are described in the following sections. Start Process Develop Process Map for process to be benchmarked Identify leading non- competitors with similar processes Decide on type of partner Identify industry leaders in process to be benchmarked Competitive/Industry Note: KPI = Key performance indicator. Functional/Generic Phase 1: Planning Brainstorm areas to be benchmarked Identify unit of analysis for benchmarking study Contact desired partners to request participation Are key outcomes covered? Request partner KPIs, metrics, and measures for relevant processes Develop or select additional metrics A A Yes No B Determine best remaining potential partners Are more partners needed? No Yes Rate partners using decision tool Select metrics using decision tool Determine appropriate data collection methods Exhibit 4-10. Benchmarking planning process map.

Module 4: engage in Continuous Improvement via Benchmarking 4-15 4.3.1 Collect Data Internally and Externally As previously mentioned, it is important that the data collector of the benchmarking study first be well educated on the needs, practices, processes, metrics, and enablers used within the sponsor transit organization to provide context to the study and be aware of the level of receptiv- ity the organization may have to certain new processes and practices. Unless the data collector is a hired contractor, it is likely the data collector is employed by the sponsor transit organization, with which he or she should be familiar. Speaking with internal employees prior to collecting data externally from benchmarking partners is the best way to gather information. Once the data collector is well versed in the practices within the sponsor transit organization, the data collector should then proceed to research information available in the public domain on a benchmarking partner. It is both frustrating to a partner and inefficient to ask questions that might have been easily answered by publicly available data. The NTD is one source of data that is publicly available to collectors looking for transit agency information. Membership information from APTA and CTAA can also be useful resources for transit data. For larger transit agencies, the FTA regional offices could be a source of transit-specific statistics. For smaller transit agen- cies, state DOTs often publish information on their subgrantees that can be useful. Corporate sustainability reports and company websites often provide a wealth of information on initiatives that pertain to image management and workforce development. When the data collector is ready to begin collecting data from the benchmarking partner organization(s), he or she should develop structured protocols that outline key questions for discussion. These protocols should define the questions that will inform the process mapping and help identify the variables that contribute to the translation of processes into outcomes. The protocols should be structured according to their purpose (e.g., open-ended questions are typically best for focus groups, while multiple/forced-choice questions are often used in sur- veys). One of the key challenges in collecting data from partner organizations is identifying the appropriate participants that are most knowledgeable about the practices and processes being benchmarked. Ideally, the participants should be the process owners—the employees respon- sible for implementing each aspect of the practice However, it may be beneficial to initiate data collection by hosting an on-site or phone-facilitated focus group with senior leadership of the partner organization. This strategy can serve as a method for the initial collection of data from a high-level perspective, with the potential for obtaining buy-in for the benchmarking study from the top. These broad focus groups may then be followed by more in-depth interviews, surveys, site visits, or some combination of data gathering strategies. Benchmarking partner participants are often more willing to provide information if they know their senior management supports the project. Prior to participation, it is important that consent to release information is obtained from all partners and that the data collector is clear about the intent and use of the data collected. The following questions will be answered in the analysis phase (Phase 2) of benchmarking: • What does the benchmarking partner’s performance look like for the practices identified for inclusion in the benchmarking study? • How does the sponsor transit organization differ from other organizations identified as successful in workforce development? • Are there effective practices or features of practices others use that can be implemented by the sponsor transit organization to bring about improvements in performance? Exhibit 4-11. Important questions to be answered in Phase 2.

4-16 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach Furthermore, prior to sharing any data, clearances for confidential information should be obtained from the partner organization. If there is any question regarding the appropriateness of data or release of information, this should be discussed with legal counsel at the partner orga- nization and the sponsor organization to ensure that the sponsor organization does not violate any ethics codes or confidentiality requirements. Understand current performance. It is important to remember that data collection on its own does not produce change. Without analysis, data does not independently suggest one course of action or another. Transit organizations engaged in benchmarking must allocate staff and resources toward analyzing and interpreting the data in light of various contextual fac- tors, including both the sponsor and partner organization’s goals and constraints. For example, transit organizations with limited budgets may have less to spend on recruitment and retention programs, limiting their effectiveness in these areas. Analyzing the results in context will enable a more accurate and valuable assessment of any gaps in performance between the sponsor and partner organizations. As previously discussed, the data collector should have an understanding of the current per- formance of the sponsor transit organization prior to data collection. Following data collection, the performance of the benchmarking partner along specified metrics should be clearer (see Module 2 for example workforce development metrics). Conversion formulas may be required to establish proportional metrics that can be easily used to make comparisons on metrics across organizations. For example, if one transit organization with 100 employees has a retention rate of 92%, and another transit organization reports it also loses about eight employees a year, a comparison percentage should be computed based on total employees. Further, it is important to consider from which functions and demographic groups these losses are taking place. If most of the employees that leave were new hires as opposed to retirees, then the impact of retention processes and practices may have increased significance; however, the impact of institutional knowledge loss from retirees must also be weighted when interpreting results. To summarize, the key steps in benchmarking data collection, both internal and external, are: • Ensure that the data collector understands the processes and practices within the sponsoring transit agency, • Gather information from internal transit employees who are responsible for the targeted pro- cesses or practices, • Research publically available information about the benchmarking partner that is relevant to the current benchmarking study, and • Gather data from the benchmarking partner using data collection methods identified in the planning phase. 4.3.2 Measure and Compare Current Performance Gap Next, comparative analysis should be used to analyze the performance gaps and indicate where and why gaps exist, as well as to identify the magnitude of the gaps between the partner and sponsor organizations. To assess the gaps, a combination of qualitative and quantitative analy- sis may be used, depending on the type of data collected. The focus of analysis is typically on determining what the partner does differently from the sponsor organization, especially when external operations by partner organizations are more effective (Camp, 1989). Understand enablers and causes of difference. In analyzing gaps, it is important to under- stand the inputs or enablers that may contribute to how the practices and processes are per- formed. These contextual factors may indicate whether practices can be wholly adopted by the sponsor transit organization or must be modified with the possibility of only partial benefits being realized. For example, if a partner organization has an entire HR department that is

Module 4: engage in Continuous Improvement via Benchmarking 4-17 dedicated to the employment and development of staff, while the sponsor transit organization does not, and establishing a human resource department is not feasible given resource con- straints and other enablers, it may be necessary to spend funds to train a few staff members on the critical components of HR and realize only partial benefits of this change. More discussion on types of enablers is provided in the section on planning, and Exhibit 4-5 provides an illustra- tion of how enablers relate to practices and processes. Positive, negative, and neutral comparisons. According to Camp (1989), there are three types of performance gaps: negative gaps, operations at parity, and positive gaps. Negative gaps suggest that the practices of a benchmarking partner organization are more sophisticated and productive than those of the sponsor transit organization, while positive gaps indicate areas where the sponsor transit organization is advanced in the performance of specific practices beyond the benchmarking partner organization. It is often presumed that a benchmarking study is only conducted to address negative gaps; however, the development of a business case for a new initiative (e.g., recruitment campaign to target youth) may elicit a benchmarking study that results in the identification of positive gaps. Parity, also referred to as a neutral gap, indi- cates practices for which there are no notable performance differences between the sponsor and partner organizations. Lapide (2003) notes that sponsor organizations often make the mistake of solely focusing their efforts and action plans on the negative gaps, whereas he believes all gaps provide useful information that the sponsor organization should use to make effective changes. He argues that the distinction between positive, neutral, and negative should be used to prioritize the order with which practice and process improvements are addressed rather than be used to disregard specific processes. For example, he suggests that negative gaps should be categorized as needing immediate focus. For negative gaps, it is important to understand the contextual factors and enablers that may account, in part, for these gaps. In some cases, there may be a tradeoff between making process improvements to reduce the gap and avoiding other losses that could result from these process improvements. For example, by investing heavily in an employee referral program when a sig- nificant percentage of your transit workforce is either eligible or soon to be eligible for retire- ment, the transit organization could end up with new hires that are themselves older and likely ready to leave the organization sooner than is desirable. An even potentially more detrimental and common error is the large investments organizations make in recruiting new staff when high potentials go unnoticed and underdeveloped within the existing organization. In sum, it is critical that the sponsor transit organization identify not only where negative gaps exist but also weigh the cost and benefits of addressing those gaps. For less substantial negative gaps and neutral gaps, Lapide (2003) suggests the gaps be given a “raise the bar” status, where the focus is on setting higher goals and standards to continue what works well. He also acknowledges that some neutral and some positive gaps may need to be on “monitor and revisit” status because in some cases it may be difficult to explain why certain posi- tive gaps exist. If the sponsor transit organization is unable to articulate what it is doing that is allowing it to perform so well in a given area, then it needs to monitor those areas. The sponsor transit organization should recognize that the positive gap could quickly change to negative if it does not know what factors are allowing for positive performance, thus rendering it unable to replicate those factors. 4.3.3 Identify Effective Practices to Close the Gap There is no definitive list of best practices that are applicable to all transit organizations. Each transit agency must assess which strategies and practices it can employ effectively given the specific market in which it operates, its business model, and the resources it has available.

4-18 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach The effectiveness of a specific practice may depend on numerous other related organizational attributes and enablers. As such, it is advisable to avoid generalizing that a specific practice is best in all contexts. Specifically, there is a wide range of strategies that can be used to improve human resource practices in transit agencies. One transit human resource concern that may need to be addressed is the time needed to fill open positions by job function. A transit agency may find that certain job functions critical to mission achievement, such as rail operator, bus operator/driver, and mechanic, take longer to fill than other job functions. Developing strategies to address this issue then becomes a priority. These strategies might include aggressively targeting applicant pools that have experience in similar structured job settings, such as individuals leaving the military, those who have entered retirement, recent graduates of technical schools, college students, school bus drivers, and long-haul truck drivers that may want to reduce time away from home. Another pos- sibility is to enhance the flexibility of the job requirements by offering part-time employment, with or without benefits, that fits the needs of individuals who may have other commitments or do not want to work on a full-time basis. To overcome this issue of time to fill a position, another strategy is to examine the recruitment process and find ways to compress the time each step takes, from identifying the vacancy to bringing a new employee on board. The washout rate of rail operators or bus operators/drivers during the new-hire training period may present a problem that can be solved by effective human resource practices. If the washout rate is unacceptably high, this could potentially indicate a less-than-successful recruit- ment and selection process; thus, strategies would need to be developed to improve the quality of candidates sought for these positions. These strategies might include looking at the profile of previous applicants who have proven to be successful employees and targeting recruitment efforts accordingly. This quality of applicant concern might also be addressed through identify- ing key indicators of potential for success in the position and building those indicators into the selection process. This could be as simple as looking for factors in previous employment history that indicate success potential, or could involve revising the interview and selection process by building in questions, exercises, or test instruments to identify high-quality applicants. Additional human resource strategies that could be used to decrease the time needed to fill open positions include improving community outreach programs and increasing the percentage of employees promoted from within the organization. Strategies to improve community outreach include partnering with more community stakeholders, such as technical schools and commu- nity colleges, and developing more effective campaigns to market the organization. Increasing cross-positional training programs and providing leadership skill development programs for employees are strategies to increase the percentage of employees promoted from within. There is great variability in the types of workforce development practices, not to mention in the impact the structure of each transit organization has on those practices and their effective- ness. This variability further emphasizes the importance of comparing the sponsor and partner organizations to determine any unique qualities of each that may affect the success of imple- menting practices. Anticipating nuances in advance and planning well to conduct benchmarking help ensure that identified effective practices translate into best practices for the sponsor transit organization. Tools to aid in comparing the sponsor organization and benchmarking partner organization are provided in the analysis phase portion of this module. Project future organizational performance levels. It is important for sponsor organiza- tions to project future performance levels based on whether the adoption or modification of practices is expected to widen, narrow, or have minimal impact on the performance gap identi- fied between the sponsor organization and benchmarking partners (Camp, 1989). In projecting future performance levels, the sponsor public transportation organization must understand how

Module 4: engage in Continuous Improvement via Benchmarking 4-19 practices will be received internally, the adaptations and resources required to adopt new prac- tices, anticipated changes in market demands, and emerging external workforce factors (e.g., the number of students entering certain fields of study). Making these projections requires estimat- ing the direction and magnitude of the sponsor transit agency’s expected performance changes as well as estimating changes likely to occur over time in the public transportation industry. To represent the change from the current performance gap to the projected performance gap, it is beneficial to create what is commonly referred to as a Z chart. The Z chart simply shows how performance levels from the sponsor transit organization will likely change over time relative to the projections shared from the benchmarking partner organization and with respect to overall public transportation industry projections. Thus, the three components of a Z chart include the projected future trends from both the partner and sponsor organizations related to implementa- tion of a specific practice as well as the expected trend for the industry as a whole. An example of a Z chart can be seen in Exhibit 4-12, with each of these three components included. The Z chart displays the expected performance gap between the sponsor and partner organization as well as the public transportation industry. This Z chart also serves as the basis for the case study described in the following. Case study 1 – interpreting a Z chart to assess impact of RJPs to recruit bus drivers. In the overly simplified Z chart provided in Exhibit 4-12, the projected trend, when averaged across the industry, is that the majority of drivers (close to 90%) will accept job offers from public trans- portation organizations as a result of the guidance implemented at a national level from this guidebook; specifically when RJPs become commonplace in the industry. However, the spon- sor and benchmarking partner organizations appear to be outliers from this trend due to the unique recruitment challenges they face as small systems serving remote areas. In this example, the sponsor organization projects that roughly 70% of the individuals offered bus driving jobs in the winter months will accept those jobs once an RJP is implemented due to challenges recruit- ing in a rural area. While this percentage of acceptances is expected to increase slightly in the early spring, it will likely decrease as summer approaches and other short-term employment opportunities become available. However, the sponsor organization estimates that when the RJP is implemented in a similar manner to the benchmarking partner, the effects for the spon- sor will be greater than the partner believes it will experience. One challenge anticipated for the partner organization is that a local network of schools has agreed to partner and increase their wages for drivers, thereby reducing the appeal of working for the partner organization. Interestingly, the sponsor organization should take note that, while it still projects to exceed the partner in overall acceptances, in summer months the partner anticipates a slight jump in its Performance Gap Pe rc en ta ge o f B us D riv er s A cc ep tin g Jo b O ffe rs Exhibit 4-12. Example Z chart for bus driver job acceptances.

4-20 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach job offer acceptances. This projection is due to the anticipated effects of a recruitment program the partner will launch in the winter and spring to actively recruit for summer help by target- ing bus drivers in local school districts, thus targeting job candidates for whom job fit would be high. Thus, by examining the projected trends for both sponsor and partner organizations over distinct periods of time alongside the future industry trend, the sponsor transit agency could evaluate where improvements are possible. Further, by presenting projected industry trends on the Z chart, sponsor transit organizations might have more realistic expectations about what performance should be expected. Types of actions. Following the use of a Z chart such as this, Camp (1989) suggests that researchers determine the reasons for the gap, both current and projected, by distinguishing between strategic and tactical actions required to close it. Strategic actions are those that require commitment of significant organization resources over a long period of time, whereas tactical actions are the more detailed, specific steps involved in completing strategic actions. Thus, strategic actions tend to define what needs to be done, whereas tactical actions describe how it will be done. 4.3.4 Overall Benchmarking Analysis Phase Following each of the steps described in this phase will aid a sponsor organization in planning a successful benchmarking study. The process map displayed in Exhibit 4-13 illustrates a model for the analysis phase of the benchmarking process. 4.4 Phase 3: Integration The integration phase of the benchmarking study refers to establishing targets or goals for transit organizational transformation. Exhibit 4-14 provides an overview of the important ques- tions to be answered during this phase. It is recommended that the sponsor transit organization chart a very specific course and set of steps. These steps should include the timeline for adopting and incorporating new practices. The two primary steps to integration are: • Communicate benchmarking findings to gain acceptance, and • Develop an action plan. These two steps are described in the following sections. 4.4.1 Communicate Benchmarking Findings to Gain Acceptance This step is critical to successful integration of new practices. To gain acceptance and imple- ment change, it is important that all employees understand the rationale for the changes, have Exhibit 4-13. Bench- marking analysis process map. The third phase of the benchmarking process, integration, will help the transit organization answer the following questions: • What is the best means to make employees aware of benchmarking findings and help encourage buy-in to planned changes in the organization’s workforce processes and practices? • How will new or updated processes be implemented in the transit organization? What are the specific implementation steps, who will be responsible for implementation, and who will be affected by the new or updated processes? Exhibit 4-14. Important questions to be answered in Phase 3.

Module 4: engage in Continuous Improvement via Benchmarking 4-21 specific knowledge of how the changes should be implemented, and are given a voice in the process. Thus, acceptance of change must be embraced throughout the public transportation organization, from frontline employees up through senior management. The first step for creat- ing organizational change in a transit organization involves upper management communicating information about the change and the positive impact this change will have on both organiza- tional and individual success. It is preferable to first initiate this communication process in per- son rather than in writing. This in-person communication can be done through management, staff, team, or safety meetings, or through some other established organizational practice. These meetings allow for dialog on the change and building of trust so that employees at all levels of the organization can work together to increase the probability of success. If the transit agency is unionized, union leadership should be brought into the change process as quickly as possible to get their input and buy-in. In fact, it may prove beneficial to involve union leadership in the initial planning phase of the benchmarking study so that prospective plans for change are anticipated up front and any challenges mitigated from the start of the study. Support from the top is imperative to help employees accept and adopt new practices. The best way for management to demonstrate its commitment to new strategies is by visibly role model- ing the desired behavior and providing open and honest answers to employee questions. Transit management must demonstrate their changed behavior if frontline employees are expected to change their job performance. In transit organizations with an us-versus-them culture, success- ful change is often difficult since it rises or falls on top-down authority. On the other hand, in transit organizations characterized by high levels of trust and communication, change has a much greater chance of being successfully carried out. For example, if management expresses the intention to improve employee engagement and retention by adopting a blame-free cul- ture during discussions on reducing on-time performance numbers or failure to meet vehicle preventive maintenance schedules, then management must respond by taking corrective rather than punitive actions when problems are reported. Likewise, offering frontline staff a voice in the decision-making process helps them believe their contributions to the overall outcome are valued. Thus, employees are more likely to make the necessary changes to achieve the desired out- come. In transit organizations, it is critical to never lose sight of the fact that frontline employees are ultimately the ones who will determine the success of new strategies and practices. To ensure that this important communication occurs, it is recommended that the transit agency develop a detailed communications plan. This plan should identify: • The audience to whom the message will be sent, • The content of the message, • The purpose of the message, • The medium by which the message will be delivered (e.g., organizational meetings, written documents, and e-mail), • The timing of the message, and • The criteria used to determine receptivity to the message. One of the most effective ways to communicate benchmarking findings and encourage accep- tance is by demonstrating the impact the practices and processes have had for the partner organi- zation. In fact, it is much easier to obtain buy-in to the benchmarking findings if the purpose for conducting the benchmarking study is presented to employees prior to initiating the study (in the planning phase). This helps reduce skepticism and prepare employees for upcoming changes by demonstrating that the changes will be rooted in comprehensive research. Furthermore, the transit sponsor organization should begin communications by articulating the positive gaps (i.e., what they are doing well) and the factors that have contributed to these gaps. Negative gaps should then be presented within the context of the positive gaps. This will aid in highlighting positive transit organizational performance and assist in leveraging that positive performance

4-22 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach to improve areas where there is a negative gap between the sponsor organization and the bench- marking partner. This approach also shows transit employees that their positive performance is appreciated, thus encouraging a strong sense of ownership in the process of reducing neutral or negative gaps. Providing additional validation, such as transit case studies that support the success of specific strategies, can help bring clarity to the employee communication and buy-in process. Finally, sponsor transit organizations are encouraged to develop an internal assessment, such as an employee survey, to determine the degree to which employees are aware of the new strategies and intend to implement new practices. The assessment itself is often a successful com- munications tool in that it further demonstrates that employee opinions are valued. Establish functional goals. Transit management needs to establish long-term targets and business plans that will support new organizational strategies. These targets should be functional goals that have clear timelines, specific milestones, and a way to measure results. These goals should be based on new strategies developed from benchmarking findings, a thorough review of assessment data from employee perspectives, and an outline of factors to consider when imple- menting practices. These goals should then be clearly communicated throughout the transit organization using the communication approach previously discussed. 4.4.2 Develop an Action Plan The process of developing an action plan begins with the construction of operating-principle statements (Camp, 1989). These statements are then integrated into performance goals. The sponsor transit organization should outline specific strategies and tactical decisions to initi- ate a series of actions that help implement new practices based on information garnered from benchmarking findings. The performance targets established for these actions should become an essential part of the transit organization’s daily activities and integrated into short- and long- term goals and objectives. An important component to developing an action plan is to identify the factors that may serve as facilitators or barriers to successful implementation of benchmarked practices. The sponsor transit organization can then leverage this information to determine which factors should be maximized and which require additional intervention. According to Camp (1989), an action plan should also include the impact the benchmarked practice is anticipated to make, the time- frame for implementing the new practice, the timeframe for a change in output to be recognized, resources needed for implementation, extent to which the effect of the practice is expected to be positive, and the degree of effort and management control required to implement the practice. Case study 2 – developing an action plan for operations supervisor training. Based on a benchmarking study with a partner transit organization, a sponsor transit agency discovers that its accident handling and investigation training and development program for operational supervisors is characterized by a negative gap and requires improvement. The action plan to address this negative gap might include: • The anticipated impact an approved accident handling and investigation training program will have on operations supervisor performance and accompanying transit liability. • How long it will take to develop the new training curriculum and retrain each transit opera- tions supervisor based on that new curriculum. • The length of time it will take to determine the impact of the new training in terms of improved accident-related documentation and reduction of transit liability based on enhanced opera- tional supervisor accident handling and investigation skills. • The budget required to pay for the new training program based on trainer and trainee hours dedicated to the effort, as well as the possible requirement for supervisory overtime to ensure con- tinued service coverage during all hours of operation due to pulling supervisors in for training.

Module 4: Engage in Continuous Improvement via Benchmarking 4-23 • The level of improved operations supervisor performance that the retraining effort is expected to generate and how that improved performance can be measured. • The assignment of responsibility to oversee the implementation of the new supervisory train- ing program and the level of coordination that will be required between training staff and operations management to carry out the training program. 4.4.3 Overall Benchmarking Integration Phase The process map presented in Exhibit 4-15 illustrates a model for the integration phase of the benchmarking process. 4.5 Phase 4: Action While the action plan establishes the foundation for the steps and resources that will be required to implement practices, this phase refers to the process of taking those steps and acquir- ing necessary resources. Exhibit 4-16 provides an overview of the important questions to be answered during this phase. The first step involved in the action phase is implementing the effective practices identified via benchmarking. Camp (1989) recommends that the employees who actually perform the work should be engaged in the action planning process to help determine the best way to encourage implementation of work processes. As previously discussed, this means getting transit employees engaged and ensuring that they have a voice in the change process should be identified as part of the communication plan. 4.5.1 Implement Specific Actions and Monitor Progress This phase includes translating the benchmarking findings into language transit employees will understand. While it may not be difficult to translate a best practice into the specific steps involved in the practice and the roles and responsibilities of those performing it, it is important to keep in mind the number of different factors (barriers and enablers) that may affect how well practices are implemented. According to Camp (1989), sponsor organizations—those driving the benchmarking study—should engage in a comprehensive analysis as part of the action phase. This analysis includes articulating ways to obtain acceptance of practices from multiple parties. This phase also includes anticipating the possible objections that transit workers may have to implementing the new practices and the assistance (e.g., consultants) the sponsor tran- sit organization may need to implement the practices. Furthermore, the location and timing of implementation should be well articulated so that employees at all levels of the organization are on the same page when the new practices are implemented. To identify potential stumbling blocks to implementation, the sponsor transit organization should consider pilot-testing prac- tices prior to full implementation. Exhibit 4-15. Bench- marking integration process map. During the fourth phase of the benchmarking process, the action phase, the following important questions will be answered: • How is the implementation of new or updated transit human resource practices progressing? • Do any of the implemented metrics need to be recalibrated or adjusted? Exhibit 4-16. Important questions to be answered in Phase 4.

4-24 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach Another key element of this phase is the frequent measurement of performance and change in outcomes to ensure that the practices are being implemented true to what was intended. To help track implementation, the sponsor transit organization should maintain a continu- ous reporting system and share the implementation progress throughout the organization. To monitor progress regarding specific actions, it is important to look at both task and behavioral changes resulting from implementation. While implementing the tasks properly is essential to achieving the desired outcome, attitudinal support from transit employees is vital to ensure that the tasks are performed at the level they should be and with the integrity intended. Case study 2 (continued) – implementing the action plan for operations supervisor train- ing. Continuing from the action plan case study presented in Phase 3, this portion of the case study addresses how the sponsor transit organization might carry out the strategies addressed in its action plan. Strategic steps include: • Conducting meetings with union leadership or frontline employee groups to explain the purpose of the new operations supervisor training effort and the positive results antici- pated. These meetings would include the opportunity for union or employee input into the effort. • Conducting meetings with operations management and all operations supervisors to explain the reasons the new training on accident handling and investigation is needed and how this training will positively affect overall operational and individual supervisory success. These meetings will allow operations leadership to discuss critical training curriculum content and approaches to training delivery with training staff. • Developing a schedule for carrying out the operations supervisor training campaign and shar- ing it with all affected and interested employees. • Organizing a small number of operations supervisors, putting them through the training program, and measuring the effectiveness of the training through post-training simulation of accident handling and investigation activities. Based on this pilot, the training curriculum should be redesigned as necessary to maximize its effectiveness and impact. • Training all operations supervisors per the previously published schedule and integrating the training program into the new-hire orientation of operations supervisors. • Tracking post-training performance of operations supervisor accident handling and inves- tigation to ensure that the training is having the desired positive affect. This tracking might include a review of accident reports, accident investigation reports, and insurance claims, and the success of transit agency defense against legal actions and liability claims. Recalibrate benchmarks. One of the primary reasons that benchmarking should be continuous is that the timeliness of the practices identified may be affected as behaviors, attitudes, and marketplace demand shift. It is important that the sponsor transit organiza- tion does not continue to target outdated benchmarking findings. If the sponsor organization maintains an ongoing benchmarking approach, practices that have lost their value can easily be spotted and metrics can be recalibrated, if needed. The metrics scorecards shown previ- ously will aid in this endeavor. By recalibrating metrics when needed (a process described in Module 2), the sponsor transit organization stays on the leading edge of changes in the transit industry, and the organization can be in ready mode to quickly anticipate and adapt to necessary changes. 4.5.2 Overall Benchmarking Action Phase The process map presented in Exhibit 4-17 illustrates the action phase of the benchmarking process.

Module 4: Engage in Continuous Improvement via Benchmarking 4-25 4.6 Phase 5: Maturity This phase in the benchmarking process refers to the point at which benchmarking becomes institutionalized within the sponsor transit organization and is viewed as a critical component to the transit management process. Exhibit 4-18 provides an overview of the important questions to be answered during this phase. Maturity also refers to the phase when change is starting to happen and desired outcomes begin to manifest themselves. The two key steps to maturity are: 1. Close performance gap, and 2. Integrate practices fully into processes. These two steps are described in detail in the following sections. 4.6.1 Close Performance Gap Negative performance gaps are overcome once the sponsor transit organization begins to experience parity or positive performance gaps compared to other transit industry leaders. A strong indicator of whether strides have been made in achieving desired outcomes is when the Exhibit 4-17. Benchmarking action process map. With the final phase of benchmarking, maturity, the following questions will be answered: • Do human resource management performance gaps still exist between your transit organization and partner organization(s)? • Has the practice benchmarked become a normal part of functioning within your transit organization, or is there still reluctance to change? • Are there areas outside of the workforce development arena, such as administration, operations, or maintenance, where the benchmarking process has the potential to enhance transit mission achievement? Exhibit 4-18. Important questions to be answered in Phase 5.

4-26 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach sponsor transit agency begins receiving requests to participate in benchmarking studies or is recognized with transit industry awards. The sponsor transit organization can communicate its success by creating a system for sharing the knowledge gained with others in the transit industry. Communicating success gives the sponsor transit organization the opportunity to acknowledge the leadership position it has obtained in the transit industry and encourages transit employees to become more engaged in the organization. 4.6.2 Integrate Practices Fully into Processes For the sponsor organization to become a leader in the industry, best practices identified via benchmarking must be fully incorporated into business processes, and desired outcomes must be realized (Camp, 1989). In the transit industry, this includes tailoring best practices based on the demographics of the service area, transit operating characteristics, and the culture of the sponsor transit organization. For example, one transit agency may find success in using a certain type of community outreach program to recruit employees, whereas the sponsor transit organization may not be able to use that exact same program for a variety of reasons. These reasons might include the urban or rural nature of the service area, the demographics of the service area, receptivity of the local community to the outreach message, the modes of service operated by the transit agency, the nature of labor management relations, and the pay and benefits the transit agency can offer. Even so, it may still be possible for the sponsor transit organization to incorporate some elements of the successful community outreach program identified during benchmarking. These elements might need to be modified but could still have a significant positive effect on improving transit recruitment efforts. Further, once practices have been successfully implemented within the workforce develop- ment area and the value of benchmarking demonstrated, the transit organization might seek to conduct benchmarking to make comparative assessments of other functional areas within the agency such as administration, operations, and maintenance. As part of institutionalizing benchmarking, those that use the results can become educated on the process for conducting benchmarking, continuously assess processes internally, and make improvements to practices. Further, when benchmarking is conducted at all levels within the organization and not solely by trained facilitators, maturity is achieved. Ultimately, by fully integrating practices into processes, the transit organization demonstrates the value of benchmarking to its employees, and bench- marking becomes an essential business practice. Some administrative, operations, and maintenance processes and practices that lend them- selves to benchmarking are: • Grants management; • Vehicle procurement; • Technology procurement; • Risk management strategies and processes; • Approaches to insurance coverage; • Compliance with the ADA; • Transit marketing initiatives; • Operational service design; • Safety, security, and emergency response policy and procedures development; • Customer service policy and protocol development; • Scheduling and dispatching systems; • Vehicle maintenance management; • Preventive and corrective maintenance procedures and protocol development; • Hazardous material management; • Shop safety management procedures and protocols; and • Contractor management and monitoring.

Module 4: engage in Continuous Improvement via Benchmarking 4-27 4.6.3 Overall Benchmarking Maturity Phase The process map in Exhibit 4-19 illustrates the basic steps during the maturity phase of benchmarking. 4.7 Combined Process Map The five benchmarking phases described in this module combine to form one comprehensive approach to benchmarking. The combination of these phases is illustrated in the process map in Figure 4-1. 4.8 Additional Tools to Aid in Benchmarking This section provides a number of additional practical tools to assist a public transportation organization in going through the five major phases of conducting a benchmarking study. 4.8.1 Phase 1 (Planning) Tools Two tools are provided to assist public transportation organizations in accomplishing the steps in the planning phase of the benchmarking process. The first tool, presented in Exhibit 4-20, will aid in selecting benchmarking partners. This Benchmarking Partner Selection Tool offers a simple method to rate potential bench- marking partners on factors that affect their desirability as a partner organization. The tool can be adjusted to add additional factors that the sponsor transit agency would like to assess. Rating scales (low, medium, high) or numerical values for each category may be used to rate partners. The partners can then be ranked and sorted based on the priorities of a given study. The sug- gested items listed in the tool are only a sample of potential factors to consider. Each transit organization should choose those most important to it when selecting a benchmarking partner or partners. There are a number of other factors that apply regardless of whether a public transporta- tion organization is benchmarking an industry peer, another organization that is functionally similar, or an organization that simply shares a similar approach to workforce development. For example, a sponsor transit agency should consider the overall similarity of the processes used by its organization and the partner organization, whether the partner is perceived as an industry leader with respect to the studied practice, and whether the partner is achieving the outcomes the sponsor transit agency desires to achieve. Although all of these factors (and potentially others) should be considered, the choice of partner will also depend on the willingness of the partner to participate. Thus, the sponsor transit organization should consider ways to ensure that the benchmarking arrangement is mutually beneficial. Benchmarking Partner Selection Tool Adaptation Strategy for Smaller Transit Agencies For ease of implementation, smaller transit agencies may want to use only a limited number of partner selection criteria identified in the Benchmarking Partner Selection Tool. The following criteria are of particular importance to a smaller transit agency: • Transit mode similarity. If a smaller transit agency operates only general public demand- response service, it may not want to choose as a benchmarking partner a transit agency that relies significantly on fixed-route or deviated fixed-route service delivery models. Phase 5: Maturity Close performance gap Practices fully integrated into process End Process D Exhibit 4-19. Bench- marking maturity process map.

Start Process Develop Process Map for process to be benchmarked Identify leading non- competitors with similar processes Decide on type of partner Identify industry leaders in process to be benchmarked Competitive/Industry Functional/Generic Phase 1: Planning Brainstorm areas to be benchmarked Identify unit of analysis for benchmarking study Phase 2: Analysis Contact desired partners to request participation Are key outcomes covered? Request partner KPIs, metrics, and measures for relevant processes Develop or select additional metrics A A Yes No Develop data collection protocols B B Determine best remaining potential partners Are more partners needed? Review own organization’s internal practices No Yes Rate partners using decision tool Select metrics using decision tool Determine appropriate data collection methods Identify partner process owners Collect data using protocols Analyze partner performance Measure and compare performance gap using tool Identify best practices to close gap C

Phase 3: Integration Phase 4: Action Develop communication plan Communicate benchmarking findings and encourage acceptance Establish functional goals Develop action plan Implement specific actions in plan Monitor progress Recalibrate benchmarks Measures still relevant? No D Phase 5: Maturity Close performance gap Practices fully integrated into process End Process Yes D C Figure 4-1. Combined process map.

4-30 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach Potential Partner Industry Similarity Transit Mode Similarity Size/Scale Similarity Exhibits Process to Benchmark Industry Leader in Process Achieves Target Outcomes Willingness to Participate Example: Partner 1 High High Medium Yes Yes Yes Medium Exhibit 4-20. Benchmarking Partner Selection Tool. Recruitment Metrics Retention Metrics Training and Development Metrics ___ Time to fill position ___ New hire turnover within the first year ___ New hire turnover during training ___ New hire performance ratings ___ Recruiting cost ratio ___ Offer-to-acceptance ratio ___ Employee voluntary turnover rate ___ Cost to fill open positions ___ Diversity turnover (race/gender) ___ Employee engagement ___ Impact of turnover on employee knowledge ___ Pre- and post-training knowledge/skill testing ___ Employee performance post- training ___ Percent of employees rating training as job-relevant ___ Impact of training on operations, safety, and customer service ___ Trainee satisfaction with training ___ Recency of training materials Professional Capacity-Building Metrics Global Metrics ___ Timeliness ___ Percentage of employees who participate in employee development opportunities ___ Internal employee advancement ___ Employee involvement in organization’s decision making ___ Percentage of positions with up-to-date job descriptions ___ Stakeholder buy-in ___ Time to implement ___ Cost to implement ___ Full return on investment ___ Sustainability Exhibit 4-21. Metric Identification Tool. • Size/scale similarity. The size of the benchmarking partner organization and the size of its service area, along with the scale of its operations in terms of days and hours of service, could be an important factor in selecting an appropriate benchmarking partner based on similarity. • Exhibits process to benchmark. Obviously it is important that the partner selected have a very clear process to examine and replicate. • Achieves target outcomes. It is important that the practice the partner is using has been successful. However, lack of outstanding results may not preclude the ability to tailor and re- engineer the process under examination to make it work. Another tool to aid in the benchmarking planning phase is the Metric Identification Tool presented in Exhibit 4-21. This tool can assist a transit agency in selecting appropriate metrics to include in a benchmarking study. It provides a concise look at metrics identified as valuable to the organizational processes included in this guidebook. Further descriptions of each of these metrics are provided in Module 2.

Module 4: engage in Continuous Improvement via Benchmarking 4-31 To use this tool, transit organizations should first identify the type of organizational process for which a benchmarking study will be conducted, such as for recruitment of applicants. Once the type of organizational process is known, an appropriate metric or metrics can be identified and selected from this tool. 4.8.2 Phase 2 (Analysis) Tools A tool to assist in the analysis phase is the set of metric scorecards presented in Module 2 of this guidebook. These scorecards can be used to gather data from internal subject matter experts regarding the practices identified for benchmarking. Specifically, the sponsoring transit organi- zation could convene expert groups to rate existing practices using the appropriate scorecard. A second valuable tool for the analysis phase is the Interview Guidelines Tool presented in Exhibit 4-22. This tool provides useful information to aid individuals in conducting a successful benchmarking interview. Thirdly, the Gap Analysis Tool presented in Exhibit 4-23 was developed to help a sponsor organization analyze gaps between its performance and a benchmarking partner’s perfor- mance. Important inputs and enablers that influence a gap and potential practices to close a gap can be listed. An example of how comparisons would be made is included in the first row of the tool. Gap Analysis Tool Adaptation Strategy for Smaller Transit Agencies Smaller transit agencies may not have a way to calculate specific percentage breakdowns between their performance and partner performance on any given issue. They should, however, identify some absolute value by which to compare themselves to the partner organization. This absolute value should be such that it allows the sponsor transit organization to focus attention on whether its overall performance on a given issue is negative or positive compared with the partner. If its performance on the issue under examination is negative, it then should identify the reasons why partner performance has been more positive and, when possible, incorporate a tailored version of partner strategies into its own internal processes. Avoid talking too much. • The benchmarking partner should do most of the talking. Interviewers are to guide the conversation and then listen, observe, and gather relevant data. Avoid evaluative expressions. • Avoid frowning or other expressions that imply evaluation since these can make the benchmarking partner hesitant to talk openly with you. Maintain eye contact. • Eye contact will let the benchmarking partner know that you are interested in what he or she has to say. Maintain a pleasant demeanor. • Regardless of your opinion of the benchmarking partner or the partner’s responses, maintain a friendly smile. Avoid intimidating the partner. Restrict personal judgment. • The benchmarking partners have been selected because of their successful performance in a specific area. Gather data from the benchmarking partner and avoid letting personal opinions or knowledge of the benchmarking sponsor’s practices influence the conversation or data collected. Exhibit 4-22. Interview Guidelines Tool.

4-32 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach 4.8.3 Phase 3 (Integration) Tools One tool provided here to help with the integration phase is an example communication plan. Plans like these are used for external communications, such as gathering and exchanging infor- mation with the benchmarking partner organization(s), as well as for internal communications, such as sharing information with employees about the benchmarking process, performance goals, and newly identified practices. It is important that distinct communication plans be cre- ated for both internal and external communications. Communication should be used through- out the life cycle of new program or practice adoption to facilitate successful implementation. This includes introducing the need for the new program or practice prior to implementing it as well as allowing time for employee input, from initial implementation to final debriefing on the results of the program. This will help employees feel engaged and valued in the process. A com- prehensive communication plan can mean the difference between program success and failure and can have residual effects on workforce development (e.g., retention, image management). The following categories are typically addressed in a detailed communication plan: • Direction of focus. Internal or external communications. • Timeline. The general timeframe when the communication will be released. • Medium/media. The mechanism through which the communication should travel (e.g., e-mail; phone). • Target audience. The receiver of the message sent. • Communication source. The group or persons designated to implement the communication. • Purpose. The reason for sending the communication. • Content. A broad summary of what will be included in the communication. • Status update. What needs to be prepared or reviewed in order to implement the commu- nication (e.g., memo); where communication materials are in the development and review process. Exhibit 4-24 provides an example communication plan. Another tool to aid in the integration phase is presented in Exhibit 4-25. This exhibit outlines some of the factors that should be addressed in an action plan. Metric Sponsor Performance Partner Performance Direction of Gap (Positive, Neutral, Negative) Degree of Gap (% of Partner Performance) Important Inputs and Enablers Potential Practices to Close Gap Example: Percent of jobs filled within 3 months 45% 76% Negative 59% Best-in- class benefit package Restructure benefit package, highlight quality-of-life benefits Exhibit 4-23. Gap Analysis Tool.

Timeline Purpose Medium/ Media Target Audience Communication Source Content Status Update First week of December 2013 Inform leadership of upcoming changes; establish credibility and top leadership support for changes; obtain buy- in prior to initiation of a program Written memo sent via interoffice mail Middle leadership down to supervisors HR drafts, executive director/CEO sends Details about needed changes in personnel hiring practices, including rationale for changes and opportunity for leaders to voice concerns Correspondence drafted by HR; waiting for exec. director approval Mid- December 2013 Notify employees of need for changes; gain buy-in by informing prior to initiation of new program E-mail memo All employees HR drafts, executive/CEO signs and sends in collaboration with middle leaders and supervisors Introduction to changes in personnel hiring practices identifying the pros and potential growing pains (and pains will be addressed); outline process for submitting concerns Correspondence released within 2 weeks of release of initial e-mail Mid- January 2013 (optimal release; may change if feedback leads to mods) Description of the new program Broadcast e- mail Identify features of new program and when full implementation will occur; Describe how employee concerns were addressed TBD Phone call (OPTIONAL) Initial communications should always come from the top to obtain buy-in. All intended communications should be listed in the plan. “Optional” indicates communication may not occur unless needed. By describing how concerns were addressed, employees are more likely to feel valued and support program throughout life cycle. “TBD” means “to be determined” since how program will roll out may depend on reaction to initial correspondence. Initial communications should allow ample time to voice concerns. The perception of being heard encourages buy-in. Exhibit 4-24. Example communication plan (direction of focus: internal).

4-34 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach Section 1 Overview of Strategic Workforce Initiative or Action Plan Highlights (key points) of action plan Rationale for new practice Section 2 Target Audience(s) Segment of organization/workforce affected by practice Estimated time to implement Return on investment Section 3 Implementation Plan Action lead(s)/process owners Steps to implement Results anticipated Section 4 Communication Plan Communication/outreach strategies Section 5 Useful Internal and External Resources (includes personnel, budget, and equipment; both available and needed resources) To implement practice To sustain practice Section 6 Examples of Effective Programs Examples of successful programs (specific transit organizations that have demonstrated success in the program/practice) Exhibit 4-25. Overview of proposed data fields to be used in each strategic workforce action plan. Exhibit 4-26 is an example of the implementation part of an action plan for instituting a mentoring program—a workforce practice that helps to recruit, retain, and develop employees. 4.8.4 Phase 4 (Action) Tools One important part of the action phase is to continuously monitor the practices that are imple- mented based on the benchmarking study and recalibrate any metrics as needed to ensure that implemented practices are still relevant and beneficial to the organization. A tool to assist in this effort is presented in Exhibit 4-27. An example program has been entered into this exhibit for illus- tration purposes only. In this example, the program has been rated every fourth month using the Recruitment Metrics Scorecard found in Module 2. The rating is then compared to previous ratings to ensure that the practice is not decreasing in effectiveness or usefulness to the transit organization. This tool would be completed for each implemented practice that is being monitored. Given that resistance from key stakeholders can be a significant impediment to success in the implementation of an action plan, the tool provided in Exhibit 4-28 can help public trans- portation organizations better anticipate and circumvent resistance. This tool is most effective when completed prior to implementation of the action plan. The purpose of the tool is to help anticipate and address challenges, thus mitigating negative reactions from stakeholders who might otherwise lobby to interrupt efforts to change. In the example provided within this tool, the transit organization may be planning to implement a new community recruitment initiative, and another local transit organization could be concerned about losing its customer base and workers as a result of this new campaign. 4.8.5 Phase 5 (Maturity) Tools One means of determining whether benchmarking has permeated the entire organization is through conducting employee focus groups. These focus groups should be organized according to specified demographic characteristics that will enable candid disclosure of experiences and perceptions. For example, typically it is best to avoid conducting focus groups with supervisors

Steps to Implement 1. Assemble Committee. Assemble committee, including HR manager, recruitment staff, and senior leadership, which will be led by the mentoring program coordinator. 2. Design Mentoring Program. Design the mentoring program by laying out the program objectives, coordinators, and size and scope of program. 3. Determine Mentorship Match Criteria. Determine match criteria that will designate how mentees/protégés and mentors are placed into relationships. A formal mentoring program, as recommended, includes mentees being assigned a specific mentor based on predetermined criteria. Example criteria for matching are: • Competency matching. Mentees are matched based on their weak areas. Mentors are selected based on their strengths or level of proficiency on competencies such that mentors can help a mentee fill in competency gaps. • Job type matching. Mentees are matched with mentors who have at least 5 years of experience in the same job type and who have demonstrated excellent performance of the job tasks. • Demographic characteristics. Mentees may be matched with mentors across jobs or lines of business based on sharing common characteristics with the mentor, such as age, race, or gender. • Combination approach. Matches occur based on a combination of factors. Typically the factors are prioritized so that the first cut for matching occurs along the most important dimension, and then within that dimension, additional factors are considered prior to making a match. • Allow input on specific matching by mentors. The senior-level staff often see relationships that would be most conducive to mentoring based on criteria that are very qualitative or soft. 4. Market the Mentoring Program. Conduct briefing sessions where employees learn about the program, including its benefits and how the restructured program differs from past mentor programs. 5. Recruit Mentors Through Various Media Channels. Information should include benefits for the mentor, program expectations, and time commitments. Interested employees should be asked to complete an application. Mentors should be prescreened to ensure that they have good working relationships with their colleagues and have no discretions on their performance record that might result in a mentee being put at risk should the behavior re-emerge. 6. Invite Mentees to Participate. This should occur at the same time as mentor recruitment. However, it is recommended that once a mentoring program is established, mentees be assigned a mentor within 1 month of employment to help with the onboarding process. The mentee should also be informed of the benefits and expectations. Exhibit 4-26. Example action plan (institute a mentoring program). Practice Score: Month 1 Score: Month 4 Change in Score: Score: Month 8 Change in Score: Score: Month 12 Change in Score: Highlight flexible benefits when recruiting employees 60 75 +15 78 3 70 -8 Exhibit 4-27. Monitoring chart: continuous tracking of implemented practices. Key Stakeholder Name Affiliation of Stakeholder Stakeholder Level of Influence Impact of Action on Stakeholder Stakeholder Priorities Stakeholder Interest in Project Best Means of Stakeholder Engagement Joe Smith, CEO of Peopleserve Local paratransit agency Moderate Low Maintain customer base High Schedule in- person lunch meeting to negotiate win–win in advance Exhibit 4-28. Stakeholder scanning.

4-36 Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach Introduction [Explain the purpose for conducting the focus group. Discuss the value of employee input on new programs. Keep in mind that all forms of data collection can be useful because the process of gathering data in itself suggests organizational values. Demonstrate that the organization values its employees by how the intent of this focus group is decribed. Discuss here how comments will be handled, such as whether confidentiality will be maintained.] Ground Rules [Establishing ground rules at the start of a focus group can help to create a supportive, open environment whereby participants feel comfortable sharing candidly. These ground rules are an easy way for the facilitator to direct participants back on track if the focus group begins to derail. Facilitators may also find it valuable to ask participants to suggest ground rules. By asking participants to suggest additional ground rules, participants begin to own and honor these rules throughout the focus group.] • Only one individual speaks at a time. • Value all viewpoints; disagreement is acceptable as long as presented diplomatically. • Respect the role of the facilitator; the facilitator may have to interrupt and redirect the conversation to ensure ample time to proceed through the questions. • Provide concise, on-point answers to questions (i.e., answers relevant to the question asked). Time: 45–60 mins [It is difficult to maintain adequate participation after 60 minutes.] Example Questions [Protocols should include primary questions and probe questions. Typically, no more than 10 questions should be used in the focus group protocol. The probe questions are only asked if the facilitator is having a difficult time getting participants to expand on their ideas. The facilitator should become very familiar with the protocol in order to know when probes are needed and how to effectively transition between what a participant says and the next protocol question.] Q1. Please share your experience with the new succession planning program [insert precise name of program here] we put into place this year. • What is your level of knowledge of the program? • What is your level of involvement in the program? (e.g., successor evaluation) Q2. Please describe the types of activities that you know to be a part of the new succession planning program. • To what extent were you asked to participate in those activities? • What role do you anticipate having in the succession planning program? Q3. Please describe the extent to which you feel supported in participating in the new succession planning program. • To what extent is your supervisor aware and supportive of your participation? • To what extent are your colleagues supportive? Q4: What aspects of the new succession planning program have been most effective? Q5: What aspects of the new succession planning program have been the least effective? • What have been some of the consequences of the new program implementation? Q6: What improvements would you suggest to make the succession planning program more effective? • What do you see as additional resource or personnel needs? Exhibit 4-29. Focus group protocol (employee evaluation of new succession planning program). and their subordinates in the same group. The differences in rank can inhibit free responding. However, it is recommended that focus groups be conducted with a selection of participants from across different job types and different perspectives within the organization to gain a rep- resentative view of the employee reaction to a new program/practice. If possible, it is beneficial for focus groups to be conducted by a trained, neutral (preferably third-party) facilitator. The optimal number of focus group participants per group is between six and eight. Prior to con- ducting the focus group, the facilitator should develop a structured protocol to ensure that the discussion results in meaningful dialogue. The protocol should call for open-ended questioning (as opposed to dichotomous, yes/no questions) to encourage detailed responses. Exhibit 4-29 provides an example of how a focus group protocol should be organized, along with some sample questions that a public transportation organization may wish to use to gather input on the implementation of a new program/practice.

Module 4: engage in Continuous Improvement via Benchmarking 4-37 Another means of assessing maturity is by closely evaluating the risks or consequences of implementation. Exhibit 4-30 provides an example of a checklist that can be used to determine how well the implementation has been executed and its effects. Each public transportation orga- nization should develop its own checklist prior to implementation based on the consequences that will have the greatest impact on its organization. • Leadership support/buy-in wavers often − Indicator: New business case for program requested every 1 to 2 months. No Yes • Cost estimate exceeds budget − Indicator: After 1 month, the costs are close to those anticipated at month 4. No Yes • Return-on-investment lead time is greater than anticipated − Indicator: Results anticipated at year 1 are not likely to be realized until year 2 at earliest. No Yes • State and federal regulations are resulting in performance delays − Indicator: No Yes • Technology requirement exceeds resource availability. − Indicator: No Yes • Contractor/vendor timelines are exceeding predetermined due dates. − Indicator: No Yes • Resistance of employees to adopting new practices. − Indicator: No Yes Exhibit 4-30. Example checklist of risks/consequences.

Next: Appendix A - References »
Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s Transit Cooperative Research Program (TCRP) Report 162: Building a Sustainable Workforce in the Public Transportation Industry—A Systems Approach provides a guidebook that addresses contemporary issues in workforce development, retention, and attraction, and public transportation image management.

The guidebook provides practical tools to transit agencies on a variety of workforce issues including workforce strategies that enhance organizational processes, performance metrics to evaluate the impact of workforce strategies, image management techniques that improve perceptions of the public transportation industry, and benchmarking processes that allow for continuous organizational improvement.

The guidebook is separated into modules that may be used independently or together in the form of the fully integrated guidebook.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!