Data to Support Transportation Agency Business Needs: A Self-Assessment Guide (2015)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

10 Overview This chapter outlines the tasks and activities associated with preparing for an agency data self- assessment and focusing the effort to best meet agency needs. Key preparation activities include the following: 1. Identify a champion to lead the self-assessment effort 2. Develop an inventory of data programs or management areas 3. Identify goals and candidate focus areas for the self-assessment 4. Obtain management support for the effort 5. Assemble a planning team and finalize the scope 6. Identify assessment team participants 7. Select and prepare facilitators Step 1: Identify a Lead New initiatives need good leaders. A single champion should be identified as the main point of contact responsible for leading, coordinating, and conducting the data self-assessment process and ensuring the exercise is worthwhile for the agency. The champion can begin by determining a proposed focus for the data self-assessment and gaining the endorsement and support to move forward from an executive sponsor. Once this is accomplished, this champion can establish a planning team to lead and oversee the effort. An additional staff support team of 1 to 2 individuals should also be identified at this point so they can participate from the start. These individuals will help with the assessment logistics and with compilation and presentation of results. Responsibilities for the assessment champion include • Reviewing this Guide and creating a list of potential focus areas for the assessment based on agency needs and priorities. • Preparing a briefing presentation describing the assessment—what is involved and how it would benefit the agency. • Setting up an executive briefing and enlisting support from an executive sponsor. The cham- pion should emphasize the relationship of the process to strategic goals and business needs and let the sponsor know that the process can be implemented at different scales or using a phased approach and that it need not necessarily represent a major effort. • Establishing a planning team to guide the effort. • Securing and preparing 1 to 2 facilitators to conduct the assessment. • Convening and chairing planning team meetings. • Attending assessment sessions. C H A P T E R 3 Phase 1: Preparing for the Assessment

Phase 1: Preparing for the Assessment 11 • Arranging for staff support to assist with meeting arrangements, meeting invitations, note taking and recording of meeting proceedings, documenting results, and preparing the action plan developed during the Improve and Monitor phase of the assessment process. • Sharing the results of the process and action plan strategies with senior management and the business units that will be involved in implementing improvement strategies and actions. • Monitoring implementation. • Scheduling and organizing follow-up assessments. Champions should have strong leadership and organizational skills, established relationships with the executive team, knowledge of agency data resources, and an appreciation of the impor- tance to the agency of data stewardship and management. Step 2: Develop an Inventory of Data Programs A data program (or “data management area”) inventory provides a useful resource for the remaining Prepare phase steps. A data program inventory provides a “map of the territory” that can be used to identify specific data management functions or business units that might be covered in the data self-assessment process. The inventory need not be comprehensive, but should include the high-value data categories and the primary business units with responsibili- ties for data planning, data collection, data quality assurance, and data delivery/reporting for these high-value categories. It is important to include databases that are centrally managed as well as databases managed and operated by individual business areas, districts, or regions if those databases are considered of high value to the agency. The data inventory can serve as a master list of candidate areas to include in the assessment during Step 3. A data inventory initially can be prepared in skeleton form and filled in over time. A data program inventory is not the same as an inventory of IT applications and centrally managed databases. It is also not the same as a data dictionary or data catalog. These types of data inventories are useful, but can be time-consuming to compile and are, therefore, viewed in this Guide as improvements that the agency may wish to pursue, rather than as a necessary activity conducted as part of the Prepare phase. Appendix B provides templates for compiling a data program inventory. Step 3: Identify Goals and Candidate Focus Areas As described in Chapter 1, the data self-assessment includes two different types of tools, each of which can be applied to different business units. The champion’s first task is to review the options carefully and, based on clearly defined goals for what the agency wants to achieve from the assessment, develop a proposed approach to applying the assessment tools in the organization. This proposal can be refined in later steps, but it provides a solid point of departure for the effort. Specific Pain Points and Opportunities One way to identify goals for the assessment is to make a list of current visible problem areas related to data. An assessment could document these issues systematically and provide an oppor- tunity to develop balanced plans to address them. Worksheet 1 provides a format for considering possible pain points—specific examples of each type of “data pain” can be documented in the second column. Blank rows are provided at the end for additional entries. The champion can complete this worksheet based on knowledge of the agency and/or discussions with key managers and staff members.

12 Data to Support Transportation Agency Business Needs: A Self-Assessment Guide A second activity that could help to establish focus areas is to identify upcoming initiatives that could benefit from the data assessment. The assessment process can piggyback on or feed into existing data and IT improvement initiatives. Worksheet 2 provides a format for identifying these opportunities. Assessment Goals and Tool Selection Identified pain points and opportunities can be used to formulate goals for the assessment and identify which of the assessment tools to apply to achieve these goals. Worksheet 3 lists possible goals and maps these goals to appropriate assessment tools. Selecting an Assessment Option The exercises in Worksheets 1 through 3 provide good preparation for selecting an assessment option. Four basic options are listed below—in order of level of effort required and degree of comprehensiveness. The first two options involve application of the data management assess- ment only; the third option involves application of the data value assessment only; and the fourth option involves application of both tools: • Option 1: Data Management Assessment for agency-wide practices • Option 2: Data Management Assessment for one or more priority data categories Data Pain Points Specific Examples Difficulty compiling the data needed for a major planning effort Difficulty gathering and integra ng data needed to produce agency performance reports Inability to comply with current or emerging external repor ng requirements Emerging agency policy ini a ves or priori es that require new or different informa on Reported data quality problems, including accuracy, currency, completeness, and reliability Percep on that the agency is behind its peers with respect to data management prac ces or data availability Recognized data problems expressed by users; people aren't ge
ng what they need when they need it, or it is taking too much work to get the data into a useful form Risk of data loss associated with informally or unmanaged databases Lack of documenta on leading to poten al misuse of data Loss of key staff with specialized knowledge of key data sets A large perceived mismatch between money spent on data collec on and the value being realized (e.g., pavement data being collected but not used for making any decisions) High-value databases are owned and operated by individual business areas (silos) and are not easily integrated, shared, or accessed Data is not being analyzed or used to provide actionable informa on Worksheet 1. Known data pain points.

Phase 1: Preparing for the Assessment 13 • Option 3: Data Value Assessment for one or more priority business areas • Option 4: Data Value Assessment for one or more priority business areas combined with data management assessment for related data categories—may also include an agency-wide data management assessment. Each of these four options is described further below. Each option is illustrated using a ver- sion of the Assess block of Figure 1 that highlights applicable portions of the assessment pro- cess. For example, given that Option 1 entails an agency-wide Data Management Assessment Type of Iniave Notes  Major System Implementaon or Replacement (e.g., financial, ERP, asset management, traffic monitoring, crash data)  Data Business Planning or Governance Iniave  Data Warehouse/Data Integraon  Performance Measurement or Management Iniave  Asset Management Iniave  Reorganizaon of IT Services  Reassignment or Reconfiguraon of GIS Services and Resources  Enterprise Architecture  Other: Worksheet 2. Upcoming initiatives that a data assessment could inform. Assessment Goal Data Value Data Management (Agency-Wide) Data Management (Program Specific)  Get an agency-wide view of how to strengthen data stewardship, data integra on, and coordina on across business units.   Look at one or more major data management areas (e.g., crash, inventory, and financial) and iden fy gaps, risks, and opportuni es to improve efficiencies.   Look at one or more business func ons (e.g., planning, asset management, and project development) and understand how well exis ng data is working to meet business needs.   Beer understand employee percep ons of data availability, quality, and usability from a data user perspec ve.   Track progress in improving data management against user percep ons of data value over me.    Worksheet 3. Assessment goals and associated tools.

14 Data to Support Transportation Agency Business Needs: A Self-Assessment Guide only, the data value and data category-specific data management elements of the Assess phase are “greyed out” in Figure 2. Option 1. Agency-Wide Data Management Assessment (single assessment team) This first option (illustrated in Figure 2) may require the least effort for most agencies. Using a single assessment team, agencies can improve their agency-wide data management capabilities and assess whether they are managing data as a strategic asset. If an enterprise data governance group has been established, they would be the ideal team for this assessment. If there is no enterprise data governance group, then a team can be composed so as to include representatives of units that provide data services to multiple business units. These would include a GIS group that manages agency-wide spatial data sets, an enterprise data warehouse team, and an enterprise business intelligence or reporting team or a group responsible for managing data collection ini- tiatives serving multiple business units. Option 2. Data Management Assessment for One or More Data Categories This option is illustrated in Figure 3. This option can be selected when an agency wants to assess data management practices within one or more specific priority data area(s) (e.g., road inventory, traffic, safety, and real-time operations data). If this option is selected, champions will need to work with their planning team to determine what specific data management areas to focus the data self- assessment effort on (e.g., traffic, safety, pavement, bridge, maintenance, and operations). Option 3. Data Value Assessment This third option, illustrated in Figure 4, can be used to focus on how users of data perceive its value—for one or more agency functional areas (e.g., providing traveler information, Figure 3. Option 2: data management focus for specific data category. Assess Data Value - Business area assessments Data Management - Agency-wide assessment - Data-specific assessments Figure 2. Option 1: agency-wide data management focus. Assess Data Value Business area assessments Data Management Agency wide assessment Data specific assessments

Phase 1: Preparing for the Assessment 15 project scoping, and maintenance budgeting). This option enables agencies to assess whether or not they have the right or “good enough” data to make decisions, meet business needs, and address reporting requirements. Option 3 can help agencies determine whether or not they are getting full value from the data they have and which supporting data programs need improvement. If this option is selected, champions will need to decide what the specific focus and goals of the assessment will be. Data value assessments are designed to address how data is meeting needs within a specific business functional area. However, such assessments can be tailored to include multiple business units as needed to address a given policy or strategic focus area (e.g., safety, infrastructure preservation, or customer service), a special initiative (e.g., open data), or a compliance need (e.g., MAP-21). Another approach is to choose a type of data (e.g., traffic or road inventory) and involve the major consumers of this type of data to assess its value. Option 4: Combined Data Value/Data Management Maturity Assessment This option, illustrated in Figure 5, can be used to examine a cluster of business areas and their supporting data programs from both user and management perspectives. This option may include doing the agency-wide data management assessment in addition to a set of more focused data management assessments. For example • Agency Functions: Maintenance Budgeting and Maintenance Work Tracking • Supporting Data Categories: Road Inventory, Maintenance Work Orders, Maintenance Level of Service, and Maintenance Budgets and Expenditures. This option requires the most effort and provides a more comprehensive view of both data management and data quality perspectives. Figure 4. Option 3: data value only. Assess Data Value - Business area assessments Data Management - Agency-wide assessment - Data-specific assessments Figure 5. Option 4: combined data value and management assessment for selected agency business areas and supporting data. Assess Data Value - Business area assessments Data Management - Agency-wide assessment - Data-specific assessments

16 Data to Support Transportation Agency Business Needs: A Self-Assessment Guide Identifying Candidate Focus Areas If Option 2, 3, or 4 is selected, you will need to identify specific focus areas for the data value and/or data management assessments. Selected focus areas should be manageable. It is impor- tant to establish realistic expectations for the assessment process and limit the scope to what can reasonably be accomplished given competing work priorities. It is also advisable to identify several different candidate areas and finalize the selections pending further discussions with the agency leadership and the planning team. Worksheet 4 provides a format for recording your recommended assessment option and candidate focus areas. Some agencies may have already undertaken data self-assessments as part of comprehen- sive highway safety planning, including the Crash Data Improvement Program (CDIP) and the Roadway Data Improvement Program (RDIP). Agencies may find this Guide and the accompanying tools useful to provide a higher level assessment of agency data programs while relying on existing safety assessment tools for more detailed assessments and diagnos- tics to identify safety-related data management improvements. Alternatively, agencies could use the completed safety assessment to inform portions of the assessment tools included in this Guide. Step 4: Obtain Management Support Before proceeding with the self-assessment, it is important to identify an executive sponsor and ensure that the effort has strong management support and is considered a priority activity. Management support is necessary to make the necessary staff resources available to participate in the assessment. If management is not interested in seeing and acting on the results of the assess- ment, the opportunity to use the effort to make a positive and lasting effect on the organization will be diminished. Management support can be secured through a combination of one-on-one conversations and more formal briefings that describe the purpose and expected outcomes of the assessment, the timeframe, and the expected level of effort that will be required. Managers need to have confidence that the process will produce objective and balanced results, rather than a long list of new data to collect. Ideally, a briefing would include members of the senior management team—particularly those whose responsibilities include data programs or business areas that have been identified as assessment candidates. Managers would be asked to endorse proceeding with the effort and weigh in on the selection of focus areas and planning team members. Such managers can also be asked to send an email (drafted by the champion) to prospective plan- ning team members requesting them to participate in the effort and attend an initial meeting. Opon Candidate Focus Areas  1. Data Management Assessment: Agency-Wide NA  2. Data Management Assessment: Selected Programs Use Worksheet 5 to idenfy potenal data programs  roW esU tnemssessA eulaV ataD .3 ksheet 6 to idenfy potenal business funcons  4.Comprehensive Assessment—both data value and Data Management Use Worksheets 5 and 6 to idenfy data programs and business funcons Worksheet 4. Data self-assessment option.

Phase 1: Preparing for the Assessment 17 Step 5: Assemble the Planning Team and Finalize Scope The next step is to assemble a planning team of 3 to 6 individuals to help set direction, review findings and results, and oversee action planning. The planning team can refine assessment goals, identify priorities and specific issues or questions to be addressed, and set the scope and timeframe for the assessment process. They can help the champion identify individuals to include in the Assess phase and serve as a sounding board for validating assessment results and action plan recommendations. Ideally, the members of the planning team could allocate resources to data improvement strategy implementation. Members of the planning team could include a mix of • Business area managers or supervisors in the areas picked for the assessment focus • Data program managers or supervisors • IT managers or supervisors • Principal data system or application owners or stewards Candidate Programs  GIS/LRS—management of base maps, road centerlines, linear referencing system (LRS), area boundaries, and feature locaons  Road Inventory—management of roadway characteriscs (may be combined with GIS/LRS)  HPMS—management of data required for federal HPMS reporng (may be combined with above elements)  Traffic Monitoring—management of traffic count, AADT, and vehic le miles traveled (VMT) data  System Operaons—management of informaon about travel me, delay, reliability, and incidents (may be combined with Traffic Monitoring)  Travel Demand—management of travel demand model inputs and outputs  Acve Transportaon—management of informaon about pedestrian and bicycle travel and pedestrian and bicycle facilies  Freight—management of informaon about freight transportaon supply and demand  Crash—management of crash and fatality data  Pavement—management of pavement inventory and condion data  Bridge—management of bridge inventory and condion data  Financial—management of financial data (e.g., revenues, obligations, budgets, expenditures, vouchers, and payments)  Capital Program—management of informaon about scope, schedule, budget, and funding allocaons for capital projects (e.g., S/TIP data)  Construcon Informaon—management of design and as-built plans and related documents  Facilies—management of facility inventory and condion informaon (e.g., buildings, park-and-ride lots, rest areas, airports, ferry terminals, and rail staons)  Maintenance—management of informaon about planned and completed maintenance acvies and associated equipment, labor, and materials  ____________________________________________________________  ____________________________________________________________  ____________________________________________________________  ____________________________________________________________  ____________________________________________________________  ____________________________________________________________ Worksheet 5. Select candidate DOT data programs for data management assessment.

18 Data to Support Transportation Agency Business Needs: A Self-Assessment Guide Note: A group with a similar membership composition may already exist (e.g., a data gover- nance group or an IT investment review board). If this group is willing to take on the responsi- bilities of the planning team, it is not necessary to form a new group for the assessment. In fact, it is preferable to piggyback on established decision-making groups. Once the members of the planning team are identified and agree to participate, their first order of business is to review and refine the assessment objectives and prioritize the candidate Candidate DOT Business Funcons Planning and Programming  Agency Performance Management  Long-Range/Corridor Planning  Freight Planning  Safety Planning  Pavement Management  Bridge Management  Asset Management  Program Development and Management (STIP)  Bicycle/Pedestrian Planning  ___________________________________ Maintenance and Opera ons  Maintenance Budge ng  Maintenance Management  Equipment Management  Facili es Management  Traffic Opera ons Management  ___________________________________ Project Development & Delivery  Project Scoping  Right-of-Way and U li es  Environmental  Survey and Roadway Design  Traffic Engineering  Pavement Design  Bridge Design  Construc on Management  Materials/Tes ng  Federal Repor ng  ___________________________________ Modal Opera ons  Avia on  Transit/Rail  Marine  ___________________________________  ___________________________________ Other  Public Affairs and Communica on  Driver Licensing and Vehicle Registra on  ____________________________________ Worksheet 6. Select candidate DOT business functions for data value assessment.

Phase 1: Preparing for the Assessment 19 business areas and/or data programs from among the candidates identified by the champion. Several factors may be considered in this process: • Alignment with Agency Priority—the degree to which the data program or business area sup- ports a priority area for the agency and the degree to which data improvements are critical for making progress toward established objectives • Impact—the degree to which an assessment would be likely to bear fruit (e.g., support actions to improve agency efficiency and effectiveness) • Willingness and ability of business units/data program managers to participate (given work- load and other initiatives) • Stability of current business processes and systems—if the unit or its main data systems are undergoing transition, it may not be the best time for an assessment • Staff experience—if there has been recent staff turnover and current staff is very new, it may be difficult for them to provide a valid assessment of current practice or needs If the goal is to get a broad view across the agency covering several different business units and/or data programs, one option is to develop a staggered schedule for the assessments—covering three to four assessments each quarter. Either the champion or members of the planning team can then secure agreement to participate from the managers of each of the selected groups and schedule them for a period that fits with their other activities. Decisions about how many areas to assess should be informed by an estimate of time and resource requirements for each assessment. Worksheet 7 can be used to develop these estimates (this does not consider startup time for the planning team or the facilitators). Estimated hours shown are for each assessment conducted, with the exception of the last one—the summary presentation can cover results from multiple assessments. This worksheet assumes that a sup- port staff person helps with scheduling meetings and conference rooms, preparing handouts, compiling questionnaire results, and preparing summary presentations. Members of each assessment team would be expected to allocate between 1 and 2 days of time, spread across three workshop sessions. The facilitator and support staff would spend an addi- tional 2 to 3 days preparing and synthesizing results between workshops. Further information on the activities listed can be found in Chapter 4 of this Guide. Acvity Staff Involvement # Hours Meeng Room? PHASE 2: Assess Assessment Preparaon Meeng Champion + Staff 4-8 No Workshop 1—Assessment Full Assessment Team 4-8 Yes Preparaon Meeng for Workshop 2 Facilitator, Champion + Staff 2-4 Yes Workshop 2—Gaps and Candidate Acons Full Assessment Team 2-4 Yes Summary of Assessment Results Facilitator + Staff 2-4 No PHASE 3: Implement and Monitor Workshop 3—Acon Planning Full Assessment Team 2-4 Yes Acon Synthesis Facilitator, Champion + Staff 2-4 No Results Presentaon to Planning Team and Execuve Sponsor (covering results from mulple assessment teams) Champion and Facilitator 1-2 Yes Worksheet 7. Assessment time and resource requirements.

20 Data to Support Transportation Agency Business Needs: A Self-Assessment Guide Step 6: Identify Assessment Participants Once business areas and/or data programs have been identified and scheduled, assessment participants can be identified. Each group should have a minimum of 4 and a maximum of 15 participants. Fewer than 4 participants may not yield a sufficiently balanced perspective, but more than 15 participants may be unwieldy to manage and it may be difficult to achieve a consensus set of assessment ratings. The following guidelines can be used to select participants: • Data Value Assessment: The objective is to obtain a balanced set of perspectives about data availability, quality, and usability to support a particular business function. Therefore, there should be representation from – Senior staff who request information (decision makers) and more junior staff involved in hands-on data gathering and preparation activities (data analysts) – Staff responsible for different aspects of the business area that depend on (or could benefit from) quality data – Staff in both central office and field offices (where applicable) – IT and data program managers—their participation will enable them to learn more about how data is being used and answer any questions on quality and availability • Data Management Assessment—Program Specific: The objective is to obtain an objective perspective about the extent to which different data management practices are being carried out. There should be representation from – Managers responsible for overseeing the data program – Data program staff responsible for hands-on data management tasks (e.g., quality assur- ance, data loading, report development, and data request fulfillment) – IT or GIS group staff who support data management tasks (as appropriate) – Key customers of the data program (as appropriate) • Data Management Assessment—Agency-wide: Participants should be similar to those for the program-specific data management assessment—with managers, staff and customers of agency- wide functions (where they exist) for data governance, metadata management, data architec- ture, data security, data warehousing, data integration, reporting/business intelligence, and GIS. Step 7: Select and Prepare Facilitators This step—selecting the right people to facilitate the assessment and ensuring that they are well prepared—is critical to the success and value of the assessment. The following are essential characteristics of a good facilitator: General Facilitation Skills • Someone who participants will trust and view as neutral, with no specific agenda other than to achieve an objective assessment; • Ability to work through an agenda within the allotted time, and ability to keep the discussion focused on the topic at hand and diplomatically steer the discussion back if it gets off track; • Ability to ensure that perspectives are heard from all of the participants—not just the senior managers or the most naturally vocal people in the group; • Ability to ensure that all participants have a consistent understanding of what each person is saying—by asking speakers to provide clarification or examples and by restating comments in different words and asking for confirmation; • Ability to guide a group to a consensus opinion—by crystalizing and restating differences in perspectives and clarifying assessment criteria definitions so the group can reach agreement.

Phase 1: Preparing for the Assessment 21 Substantive Knowledge • Understanding of agency business processes and data—enough so that participants do not need to spend a lot of time explaining the basics of their program or business function to the facilitator. • Sufficient experience with data management practices and concepts to be able to understand the terminology used in the assessment and explain it in terms that participants will under- stand (this is particularly important for data management assessments, but helpful for data value assessments as well). Ideally, agencies would identify two facilitators to share the load, provide more scheduling flexibility, and ensure that a backup facilitator is available if one leaves the agency. Once facilitators are identified, they should review the material in this Guide carefully and ensure that they understand and can clarify all of the terminology and distinguish across the different assessment rating scales (i.e., maturity levels for the data management assessment; good-fair-poor ratings for the data value assessment).