Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
15  Roadmap This chapter explains how the proposed data structure fits into transit agenciesâ data work- flows. Currently, most transit agencies receive ITS data from private vendors. The level of aggre- gation and format of this data varies. The data structure and related tools are intended to support the management of data both for transit agencies that receive very detailed data (for example, heartbeat vehicle location data, individual timestamped boarding records) as well as agencies that receive more processed data (such as data assigned to a trip and stop or aggregated at a station by time period). 3.1 Event vs. Summary Data Files The core of the proposed structure consists of two sets of files. Throughout this report, they are referred to as Event and Summary. Event data refers specifically to: ⢠Timestamped vehicle locations and speeds. ⢠Timestamped boardings and alightings. ⢠Other timestamped passenger-related events, such as door openings and ramp deployments. ⢠Timestamped fare transactions. Summary data refers to: ⢠Vehicle arrival and departure times by trip and stop for each service date. ⢠Total boardings and alightings by trip and stop for each service date. ⢠Information about other events (e.g., door opened, ramp deployed) by vehicle trip and stop for each service date. ⢠Fare transactions by vehicle trip and stop for each service date. ⢠Vehicle trips performed for each service date. ⢠Summarized transactions, entries, and exits by stop or station and time period (for events not associated with a vehicle). While event data may have information about the stop and trip it is associated with, it is orga- nized in a way that is distinct from the summary data (note: throughout this report, the term trip refers to a trip made by a transit vehicle). In the event data, there can be multiple timestamped events associated with a single stop, trip, and service date. In contrast, each record in a summary dataset refers to (1) a specific trip, stop, and service date, (2) a specific trip and service date, or (3) a stop or station and time period. Another distinction between the two data file types is that Event Data Files can store ITS device-specific data (e.g., data from a specific fare gate or vehicle location device). In contrast, summary data is assigned to a stop or station. There is one excep- tion: the data structure allows users to record boardings and alightings across two separate door groupings (front and back or left and right) in the summary stop-level data. C H A P T E R 3
16 Improving Access and Management of Public Transit ITS Data In addition to the Event Data Files and Summary Data Files, the data structure also relies on some additional Supporting Data Files that store information on vehicles, devices, and operators and on GTFS. 3.2 Data Flow Figure 3 depicts the proposed data flow. Data originates from ITS hardware and software, which are most commonly provided by vendors. The vendors share data (Vendor Outputs in the first blue box) with transit agencies. Vendor Outputs may consist of summary data (such as total boardings at a stop) or may include detailed time-based event data (such as individual timestamped boardings). Some transit agencies may also collect time-based event data from some systems (e.g., AVL) but not others (e.g., AFC) or from some modes (e.g., bus) but not others (e.g., light rail). Given this variation, transit agencies may follow a combination of two different paths through this data flow, dependent on what type of data they receive as Vendor Outputs: ⢠Path if using Event Data Files: The first path, depicted with the solid blue arrow is designed for an agency that receives time-based event data from their vendor(s) and desires to maintain that data in a common format. In this case, the vendor-specific outputs are transformed into the Event Data Files. Event Data Files consist of timestamped event data. In these files, each row refers to a specific event and there may be multiple events associated with a single stop visit. Alternatively, agencies may choose to maintain existing data formats tailored to agency needs and then use agency-specific tools to transform these files into the Summary Data Files, which they then would use to evaluate the KPIs. For transit agencies that opt to use the Event Data Files, the Data Transfer Tool transforms the Event Data Files into the Summary Data Files. Before the transfer, the data flow includes the application of the Format Validation Tool and Data Quality Tool to ensure that the data provided is in the correct format and that unique identifiers across internal and external reference files match. ⢠Path if using only Summary Data Files: The second path, shown with the dotted blue arrow, can be used by transit agencies that do not receive time-based event data or do not want to transform their Vendor Outputs into the Event Data Files. In that case, the Vendor Outputs are transformed into the Summary Data Files. Transit agencies may opt to use some but not all Event Data Files. In this case, they would follow the first path for some data types and use the second path for other data types. Regardless of Figure 3. Data flow. Note: Box colors in the diagram reflect different data file types. The colors are used consistently throughout this report.
Roadmap 17  the initial path, after the Summary Data Files are produced, the For- mat Validation and Data Quality Tools are applied to the Summary Data Files to ensure formats are correct, IDs are unique and consistent, and questionable and missing data is flagged. The Summary Data Files organize the data in such a way that it can support evaluation of KPIs. The analysis tools are then applied to analyze the data to generate the KPI Reports. 3.2.1 Special Note on Train-Car-Specific Aggregation The Summary Data Files allow transit agencies to record boardings and alightings across two doors (or sets of doors) per vehicle (including for an entire train consist, or group of vehicles that form the rail unit). Specifically, boardings and alightings are recorded in two fields. One field tallies boardings and alightings for the single front-most or right- most door, and a second field tallies boardings and alightings for the rest of the doors on the vehicle. For transit agencies that are interested in performing analysis compar- ing boardings and alightings within vehicles (including train consists) with many doors, such as platform crowding analysis in rail stations, a mid-level of aggregation is needed. These transit agencies have the option of creating a file that summarizes passenger events by stop visit and by individual train car rather than for the entire train consist. This file could be generated directly from Vendor Outputs or through the aggregation of the Event Data Files. Instructions for how to incorporate this option into the data structure are provided in Chapter 6 on Best Practices. 3.3 Integration with GTFS The data structure provides consistency and interoperability with GTFS by: ⢠Encouraging the use of GTFS identifiers (such as stop_id, stop_sequence, and trip_id), and ⢠Incorporating GTFS schedule information into the data structure to make connections between observed data and the scheduled system. Transit agencies use GTFS in different ways, but there are some common GTFS use practices that can pose problems for this data structure. Most importantly, in this data structure, identi- fiers must correspond to an individual trip and trip-stop for each service date uniquely. Depending on an agencyâs practices, GTFS identifiers may not be unique across modes or agencies for each service date within GTFS (e.g., ITS data sourced from multiple agencies that operate in a single transit service area). In addition, these identifiers may not persist through multiple GTFS pub- lications even if they represent the same or similar stops, trips, or routes. This could prevent the evaluation of KPIs over time across multiple iterations of GTFS. As a result, some transit agencies may need to develop unique identifiers specific for this data structure, which are distinct from GTFS identifiers, to maintain historical consistency. Section 6.3 explains how transit agencies can adapt their GTFS practices to use GTFS identifiers with this data structure. Section 6.5 provides guidance for how transit agencies can maintain information on unscheduled trips, trips that detoured from schedule, or scheduled trips that were not made. Note on Implementation: Transit agencies may process Vendor Outputs into the Event or Summary Data Files themselves, or they may request that vendors do the processing. Third parties may also develop tools for this processing; however, Vendor Outputs typically are specific to each agency, which may limit the potential for shared tools. In future procurement processes, transit agencies could require the proposed file formats from vendors. Over time, vendors may adapt their tools and methods to produce data in the proposed formats.
