Review of the Federal Railroad Administration’s Research and Development Program (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1 Summary The Transportation Research Board (TRB) formed the Committee for a Review of the Federal Railroad Administration’s (FRA’s) R&D Program at the request of FRA’s Office of Research, Development, and Technology (RD&T) for strategic feedback on the program. RD&T asked that par- ticular attention be given to strategic planning, means for setting priorities, stakeholder engagement and communications, and project and program evaluations. This is the seventh TRB review of the RD&T program and the first since 2015. The RD&T contract research portfolio is programmed and administered by four research divisions covering the major railroad engineering disciplines: track and structures, human factors, rolling stock, and train control and com- munications. The committee divided into four subcommittees consisting of subject matter experts to review the work of the four divisions by consider- ing the safety needs they address, querying their leadership and staff about methods used for selecting research topics and managing projects, consulting with stakeholders familiar with each division’s work, and sampling a handful of projects. The committee reconvened as a whole to consider the results of the division reviews and examine RD&T’s support functions. RESEARCH DIVISION REVIEWS Alignment of Research with Incident Causes On the whole, the four research divisions do a good job programming re- search that addresses railroad safety problems and causal factors identified

2 REVIEW OF FRA’S R&D PROGRAM from FRA incident data. A theme that characterizes the Track and Structures Division’s (Track Division’s) portfolio is innovation in track inspection ca- pabilities to detect track component defects, the top cause of track-related derailments. Railroad worker fatigue, a well-established cause of derailments and evident in the number of incidents attributed to excess speed and switch run-throughs, is a focus area of the Human Factors Division. A large por- tion of the Rolling Stock Division’s portfolio addresses wheel, bearing, and other truck and suspension system failures that are major causes of train derailments attributed to locomotive and railcar equipment and compo- nents. While much of the research in the Train Control and Communications (TCC) Division’s portfolio supports the statutorily required implementation of positive train control (PTC), it devotes the balance of its portfolio to research aimed at reducing the large number of reported grade-crossing and trespasser fatalities. Other Considerations When Programming Research In addition to prioritizing and programming research informed by incident data, each of the four research divisions takes other considerations into account, including the interests and needs of industry, other FRA offices, Congress, and other stakeholders. They do so in a generally balanced way, which can be challenging given their discretionary budget constraints. All four divisions make a concerted effort to engage with the railroad in- dustry, consulting regularly about research needs and partnering in the con- duct of research. By connecting with industry, the divisions hope to ensure that their research projects meet practical needs and produce usable results that increase safety, while also minimizing the potential for duplication of effort and augmenting program budgets to leverage industry contributions. Although the level of industry engagement is not consistent across the four divisions—and some additional engagement and communications activity may be desirable for reasons observed below—the committee’s impression is that industry outreach and collaboration is a strong suit of the RD&T divisions. It is notable that the Human Factors Division has developed a strategic plan for the Cab Technology Integration Laboratory (CTIL) to place greater emphasis on industry consultation, collaboration, and com- munication, which was identified as a key need in the previous TRB review. The four research divisions also view other FRA offices as customers of their research, while also responding to legislative direction. The Track Divi- sion has a direct role in supporting the regulatory and enforcement needs of the Office of Railroad Safety (RRS) through its projects on inspection technol- ogies, and the Rolling Stock Division conducts research to inform rulemaking on tank cars, the transport of hazardous materials, and other related topics. Congress has directed research in the Rolling Stock Division to improve

SUMMARY 3 the safety of passenger railcars and in the Human Factors Division through dedicated funding for the Short Line Safety Institute (SLSI), which seeks to measure and improve the safety culture of short line and regional railroads. External Views To obtain insight on how others view the relevance, impact, and reach of the four divisions’ work, the subcommittees consulted with researchers who have worked on projects, freight and passenger railroad industry represen- tatives, and FRA officials from other offices. The commentary was largely positive, as the work supported by the divisions was frequently described as high quality, relevant to industry needs, and likely to produce safety benefits. For example, the Rolling Stock Division’s wheel research program was lauded for having made valuable contributions to understanding wheel failure mechanisms; the TCC Division’s research on PTC was described as addressing industry needs to expand the capabilities and improve the over- all performance of this technology; and the Human Factors Division’s work on suicide and trespasser countermeasures was characterized as timely and helpful for addressing these vexing safety problems. Among the concerns that surfaced in discussions with industry, as noted in observations below, were criticisms about the slow and inconsis- tent process for publishing research, the sometimes limited communication of ongoing work and results, and the slow pace of some work, which in certain cases was attributed to the practice of dividing, or phasing, topics into multiple projects spread over several years. The phasing of projects into 1- and 2-year performance periods was also viewed by some academic researchers as problematic for attracting doctoral students, which is desir- able for building the railroad research and engineering talent pool. Some researchers also commented on the need for more opportunities to work on basic research and propose research on cutting-edge or novel solutions to safety problems. Division-Level Observations and Advice Track and Structures Division The committee observes the efforts made by the Track Division to identify research needs based on a combination of safety data, industry consulta- tion, and collaboration. The division appears to place a heavy emphasis on proposals that involve partnerships and collaborations with railroads, suppliers, RRS staff, and other potential users of the results. The willing- ness of stakeholders to participate in this way increases the likelihood that the research will address practical problems and lead to innovations that

4 REVIEW OF FRA’S R&D PROGRAM will have application in the field. Advance planning to assure widespread deployment capability is also evident in the selection of projects and by the frequent partnering of research contractors with industry. Even though the division plans for deployment, an observed need is for even more front-end technology transfer planning to avoid deployment obstacles, such as those that have arisen from assignments of user and intellectual property rights. In accordance with these observations, the Track Division should en- sure that technology transfer planning is sufficiently thorough and anticipa- tory at the outset of projects to take into account and manage the challenges that can hinder desired levels of deployment of research products and services, including the management of intellectual property such as user and data rights. Human Factors Division The committee observes that several of the programs and projects of the Human Factors Division are coordinated effectively through partnerships with and implementations by railroads toward achieving desired safety ben- efits, including the trespassing and suicide countermeasures program and the work of the SLSI. The strategic plan for CTIL, which calls for increased collaboration with industry, should help ensure that this world-class human factors laboratory is used to the maximum extent to address critical safety issues such as worker fatigue, effective application of automation, and im- provements in human–technology interfaces and integration. An observed need is for the division to match its high-quality work with a similar high-quality capability to communicate and disseminate the results and to tap a larger audience for its work, including the U.S. rail transit and highway sectors. Some longer-duration projects might also be desirable to attract doctoral students to projects, such as those performed using CTIL. To expand its reach and influence, the Human Factors Division should (1) become more involved in the design, development, and programming of research projects across FRA’s RD&T portfolio; (2) engage more frequently with other U.S. Department of Transportation (U.S. DOT) modal adminis- trations and other relevant federal agencies to identify opportunities for its work to inform, and be informed by, their human factors–related challenges and research; and (3) collaborate on the design and conduct of relevant research and the dissemination of those research results and products that have broad, multi-modal application. Rolling Stock Division The committee observes that the Rolling Stock Division, perhaps more than the other divisions, covers a wide safety domain that demands consideration

SUMMARY 5 for and balance of multiple interests when programming research. It also observes that because of this wide span of interest, industry consultation, communication, and collaboration are critical to ensuring that the division’s research capacity is used to its greatest advantage. With these observations in mind, the committee recommends that to better ensure that its projects not only align with the most important safety problems but also focus on research needs that are best suited to the unique strengths of a government contract research program, the Rolling Stock Di- vision should make industry consultations and collaborations a core feature of all research that will ultimately require industry acceptance and applica- tion of the results. In having so many diverse research responsibilities, such an alignment and focus is essential to ensuring the effective and judicious use of the division’s limited research budget. Moreover, the division should prioritize projects based on safety data; these priorities should be used to determine the number of active projects commensurate with the division’s funding level and allocated personnel. Train Control and Communications Division The committee observes that the work by the TCC Division to improve the operational capabilities and performance of PTC aligns with its safety mission. It also observes that the division’s focus on automation, intelligent transportation systems, communications, and sensor technology recognizes the importance of human factors to train operations and grade-crossing safety. Having such a wide breadth of responsibility—from furthering PTC to grade-crossing safety—the division has demonstrated creativity in harnessing these technologies to address human performance and behavior issues that can affect safety. An observed need is for the TCC Division to ensure that the results of its research are published and disseminated faster, which is especially important for ensuring the timely application of the division’s advanced technology–oriented work. To this end, the TCC Division should make a concerted effort to ensure that research results are made available to indus- try and other users as quickly as possible to contribute to the advancement of PTC and other systems being deployed in the field. REVIEW OF R&D SUPPORT FUNCTIONS Asked to review the support provided by RD&T for strategic planning, communications, and evaluation, the committee met with RD&T’s senior management and division chiefs to discuss these functions and reviewed rel- evant planning and budgetary documents. In inquiring about the status of RD&T’s strategic planning, the committee was told that a late-stage draft

6 REVIEW OF FRA’S R&D PROGRAM of the FY 2018–2022 strategic plan was pending internal review, and thus might not be available during the study period. On the basis of these brief- ings and limited document reviews, as well as insights gleaned about the effectiveness of these support functions from the division reviews, the com- mittee offers the following observations and advice with respect to RD&T’s strategic planning, communications, and evaluation support functions. Observations and Advice Strategic Planning Absent the availability of an up-to-date strategic plan during the study committee’s deliberations, the committee examined RD&T’s FY 2013–2017 strategic plan, assuming that RD&T may be using it as a model for the pending strategic plan and that it continues to exert influence over the exist- ing program. As a model for future strategic planning, the committee finds both strengths and weaknesses in the FY 2013–2017 plan. The articulation of stakeholder communication and partnering strategies are strengths. The plan identifies key stakeholders who should be engaged, both inside (e.g., RRS) and outside (e.g., passenger and freight railroads, labor, and suppliers) of FRA. In addition, the plan contains well-reasoned strategies for identify- ing research needs and prioritizing projects, such as the importance of using incident data and consulting with RRS when identifying safety research needs and developing project concepts. The FY 2013–2017 strategic plan is weaker in some other respects. Notably, it lacks explanations for procurement and evaluation strategies and thorough planning for technology transfer of research products and services. For instance, the plan does not explain why or when Broad Agency Announcements should be used as the means for procurement or when phasing of contract research into a series of smaller projects is warranted. The plan does not address advance planning for technology transfer, includ- ing means for addressing important issues such as the management of user and intellectual property rights. More generally, the plan does not offer compelling articulation of the role and value of FRA’s RD&T, including the range of outcomes intended, which presumably consists of not only tech- nology development to enhance safety but also information for decisions, operational solutions to problems, knowledge to support future research, technology transfer, and the development of a skilled pool of railroad safety researchers. Inasmuch as the strategies espoused are likely to involve interests out- side RD&T, including other FRA offices, the railroads and their suppliers, labor, and the research community, the opportunity for them to have input

SUMMARY 7 into the plan at a formative stage is essential, but not evident in the FY 2013–2017 strategic plan. RD&T should engage in ongoing strategic planning that not only articulates agency priorities, strategies for pursuing them, and justifica- tions for its programs and budgets, but also clearly defines the support functions that are its responsibility and explains how those functions will be implemented. The plan should provide strategies for nurturing new technologies and techniques, such as the application of machine learn- ing to railroad applications. The plan should also provide direction for assessing the impact of the research program over time, such as for the period covered by the most recent strategic plan, and it should articulate strategies for future impact assessments. Because many of the strategies in the plan are likely to involve interests outside RD&T, including other FRA offices, the railroads and their suppliers, labor, and the research com- munity, such stakeholders should be formally invited to offer input early in the planning process. The practice of phasing research, while conferring several benefits such as providing checkpoints on the satisfactory conduct and utility of the work, can lead to delays in the conduct of research and a larger administra- tive and oversight burden that can slow the development and deployment of important safety products and knowledge. RD&T should make more judicious use of phased project procurement to ensure that valuable research results from well-scoped, low-risk projects are not delayed as a result of multiple procurements and that the number of projects under contract does not create oversight and administrative burdens that risk delays across the portfolio. Note: While the study committee’s report was in peer review, RD&T released its Research, Development, and Technology Strategic Plan for 2020–2024 (Fed- eral Railroad Administration Office of Research, Development, and Technology 2020b). Unfortunately, the release occurred too late in the study process for the committee to thoroughly deliberate over and critique the document. One general observation that can be made, however, is that the strategies identified in the plan are similar to those of the earlier plan, with the exception of a new strategy for addressing safety risks in rural areas. A notable improvement is the inclusion of benchmark safety outcomes or goals as well as other metrics to evaluate a project’s success, such as reduce the cost per mile of recording track conditions. Apart from these observations, however, the updated plan does not appear to address the points raised above in the recommendation. It also bears noting that for 3 years RD&T has not had the benefit of this refreshed strategic plan.

8 REVIEW OF FRA’S R&D PROGRAM Communications Both in planning documents and from discussions with senior manage- ment, stakeholder communications was described as integral to the pro- gram because of the benefits for priority setting, project management, and dissemination and communication of research results. Providing multiple channels for the broad array of stakeholders to communicate their research needs was characterized as critical to ensuring that the right problems are being targeted for research and that appropriate research mechanisms and procurement methods are being used. Communications was also described as being incorporated into the entire project life cycle to ensure that the work is being done well and that the results will be used in the field to yield positive safety impacts. The committee was told that the RD&T division staff have been di- rected to evaluate the progress of the research and usefulness of the results on an ongoing basis by partnering with stakeholders in the conduct of the work and by making a coordinated effort to disseminate the results to the railroad community. The committee’s review of the work of the four divi- sions suggested that this direction is being followed, but gaps surfaced in certain aspects of communications, particularly in the area of dissemina- tion, such as hastening publication review and enhancing the office’s elec- tronic library (FRA’s eLibrary), which may warrant attention and support from senior management. RD&T senior management described the approach to communica- tions within FRA and with the railroad industry through examples of the individual divisions engaging with railroads and suppliers. RD&T has conducted periodic informal meetings for years with colleagues in RRS, but since 2017 these meetings have been scheduled on a regular basis to discuss safety and research needs. The subcommittees found many examples of engagement with the railroad industry when reviewing the work of the four divisions. The examples cited in the report show a wide variety of approaches being employed, including RD&T staff participating in As- sociation of American Railroads and American Public Transportation As- sociation research and technical committee meetings and staff presentations at industry conferences. In the absence of an updated strategic plan, however, it proved difficult for the committee to assess the role that the RD&T management team plays in supporting communications. The senior management team could be proactive in trying to identify where additional communications efforts are desirable and provide the needed guidance and resources to support and sustain them. RD&T should consult the four research divisions about their commu- nications needs and challenges and look for opportunities where support

SUMMARY 9 by the management team would be advantageous, such as in providing resources for staff travel and participation in industry conferences and for convening annual, multi-day stakeholder workshops for sharing research results, identifying and prioritizing research needs, and exploring opportu- nities for collaboration in the conduct of research and in the demonstration and deployment of research results. Evaluation RD&T senior leadership and its divisions depend on close connections with stakeholders to ensure that the work is relevant, remains on course, and has impact when completed. The committee agrees that stakeholder connections are a necessary means for evaluating relevance and impact in large part because they ground the research program in reality and thereby provide a form of built-in evaluation. The four research divisions appear to ably develop and sustain such connections, which has yielded many de- ployable products for improving railroad safety. However, there are many reasons for engaging in formal and deliberate evaluations and for not relying solely on this built-in process for ensuring program effectiveness. One practical reason is that FRA research seeks to do more than develop deployable safety-enhancing products, services, and operational concepts. While responsibility for undertaking program-wide evaluations clearly lies with RD&T divisions, the management team can also support the divi- sions in conducting their own program and project evaluations. The FY 2013–2017 strategic plan is short on details describing how RD&T senior management engages in and supports program and project evaluations. The committee learned that the management team has been building internal capabilities for program evaluation to ensure that projects have the highest probability of delivering benefits, but few additional details were provided on the status and accomplishments of this initiative. A general sense of the committee is that the management team is interested in conducting more informative evaluations as well as providing the divisions with more guidance and capacity for conducting their own evaluations, but it lacks the needed expertise in the field of research performance management and program evaluation. RD&T should work to develop a more comprehensive approach to program and project evaluation, including the development of a common evaluation framework that can be used by the four divisions to assess outputs, outcomes, and ultimate safety impacts of their work. FRA should adopt best practices that become apparent through periodic benchmark- ing exercises with other agencies in U.S. DOT and other federal research agencies about their research program evaluation methods and support functions, including the Federal Highway Administration.