Letter Report on the Freight Analysis Framework (2004)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

February 9, 2004 Mr. Bruce Lambert Office of Freight Management and Operations Federal Highway Administration 400 Seventh Street, SW Washington, DC 20590 Dear Mr. Lambert: We are pleased to transmit this letter report of the Transportation Research Board (TRB) Committee on the Future of the Federal Highway Administration’s (FHWA’s) Freight Analysis Framework (FAF). The committee membership is shown in Enclosure A. The committee, convened at the request of FHWA’s Office of Freight Management and Operations (the Freight Office), was tasked with holding a workshop involving key stakeholders to discuss issues and options for future enhancements of the FAF. The committee’s report on this workshop was to (a) summarize the views of various stakeholder groups on the issues and options and (b) comment on major data gaps, frequency of data and FAF updates, and technical issues concerning linking the FAF with other FHWA and U.S. Department of Transportation (USDOT) models and applications, to the extent that these latter items were addressed during the workshop. The committee held its workshop on October 21–22, 2003, at the National Academies facilities in Washington, D.C. Further details are provided later in this letter. Following the data-gathering sessions of the workshop, the committee met in closed session to deliberate on its findings and recommendations and begin preparation of this report, which was completed by correspondence among members. In developing these findings and recommendations, the committee drew on the information gathered at the workshop, information provided by other individuals and organizations with an interest in freight issues, a draft overview of the FAF provided by the Freight Office (FHWA 2003), and the experience and expertise of individual members. Reference was also made to a recent TRB report, A Concept for a National Freight Data Program (2003b). The committee would like to thank all those who contributed to this project through their participation in the workshop and their responses to follow-up questions. The committee also gratefully acknowledges the assistance of those who provided information about various aspects of the FAF through correspondence and telephone calls. In summary, the committee found that the FAF and accompanying maps of freight flows have been effective in raising awareness of freight issues among policy makers at the highest levels within USDOT, state departments of transportation (DOTs), and the U.S. Congress. The FAF has also demonstrated clearly the potential value of combining data from different sources to create a national multimodal freight database and linking this database to economic forecasts. In the absence of a more comprehensive source of multimodal freight flow data, particularly one encompassing truck flows, some users are looking to the FAF to provide the data and accompanying forecasts they need to inform project-specific analyses, planning, and investment decisions. However, the FAF was originally developed for

2 internal FHWA use in examining national policy issues and is, in general, an unsatisfactory and inappropriate tool for a broader spectrum of users and applications. Furthermore, design and data limitations make the FAF an unsuitable foundation upon which to develop a database of the kind users require, namely, a comprehensive national multimodal freight database providing transparent data that can be readily linked to state and local data sets to inform analyses at various levels of geographic detail. The committee recommends that the Freight Office, in cooperation with other groups within FHWA, the Bureau of Transportation Statistics (BTS), and other USDOT modal administrations, focus its resources on developing a national freight data program targeting the needs of transportation analysts and planners. This initiative would draw on lessons learned in the FAF project, as well as on a previously proposed concept for a national freight data program. Further work on the FAF itself should be restricted to short-term documentation and clarification efforts and some limited updating, to the extent that the latter activity can usefully elevate awareness of freight issues in the context of the reauthorization of the Transportation Equity Act for the 21st Century (TEA-21). The remainder of this report commences with some brief background on the FAF. An overview of the committee’s workshop is followed by a discussion of FAF use. The committee’s findings are presented in the sections on FAF accomplishments and on issues and options for the future. The report concludes with the committee’s recommendations concerning modest, short-term improvements to the FAF and longer-term efforts to respond to the increased user demand for freight data—a demand stimulated in part by the FAF. BACKGROUND The FAF project was initiated by the Freight Office in 1999 with the goal of creating a tool for internal use by FHWA staff in examining national policy scenarios within the context of the pending reauthorization of TEA-21. The project involved three major technical steps: developing the physical FAF network, developing domestic and international freight flows and linking them to the FAF network, and developing forecasts for 2010 and 2020. The FAF road network draws on state-specific databases and data from federal road inventories that contain, or can be linked to, Highway Performance Monitoring System (HPMS) data.1 The FAF information on freight flows is based on freight transportation data from both public and private sources, notably the 1993 Commodity Flow Survey (CFS), a public data set, and Reebie Associates’ proprietary Transearch data set.2, 3 Because of data gaps, some of the FAF freight flows were synthesized by using models. The FAF estimates of commodity volume and value for 2010 and 2020 are based on proprietary economic forecasts developed by DRI-WEFA, Inc. (now Global Insight, Inc.). Agreements with Reebie Associates and DRI-WEFA, Inc., allowed FHWA to use 1 The HPMS is a national-level highway information system, developed and implemented by FHWA and the states, that includes data on the extent, condition, performance, use, and operating characteristics of the nation’s highways. 2 The Transearch data set uses both public and private data sources to generate information on county- to-county freight volumes by commodity and mode. 3 Other data sources used in constructing the FAF include the BTS Transborder Freight Statistics, the Surface Transportation Board (STB) Rail Waybill Sample, data from the Maritime Administration and the U.S. Army Corps of Engineers on waterborne freight, private port directories, and trade association production and shipment reports.

3 proprietary data and forecasts in developing the FAF as an analytical tool for application in USDOT headquarters’ studies conducted in-house or fully funded by the department. The FAF describes domestic and international freight movements within the United States, by commodity and mode,4 on a network of FAF transportation facilities for 1998 (base case), 2010, and 2020. The FAF database contains freight flow information at the county-to-county level, and this information is accessible to federal government users. However, because of the proprietary nature of some of the constituent data sets, publicly available data from the FAF are aggregated to the state-to-state level. The FAF database has been used to generate a variety of freight flow maps, which are available on the FAF website (www.ops.fhwa.dot.gov/freight/adfrmwrk/). The maps for states and major cities show flows by truck, rail, and waterway, as appropriate. The FAF map for each port or border crossing shows the inland movement of international freight by truck in 1998. The FAF was developed by a team composed of Freight Office staff and private-sector consultants. FAF contracts for the period from 1999 through 2003 totaled approximately $5 million and covered the purchase of private-sector data and forecasts, as well as consultants’ services in developing analytical methods, the FAF data sets, the freight flow maps, and the like.5 FAF WORKSHOP On October 21–22, 2003, the committee hosted a workshop on the FAF, which was attended by approximately 40 participants from federal government agencies, state DOTs, associations, academia, and private-sector organizations, including consulting companies. The committee’s objectives in hosting the workshop were to learn about applications of the FAF and to discuss with users their views on the FAF’s strengths and weaknesses, as well as how they would like to see the FAF develop in the future. After an introductory overview of the FAF from Freight Office staff, three panel discussions were held with FAF users, as follows: Panel 1. Federal Government Users, Panel 2. State and Metropolitan Planning Organization (MPO) Users,6 and Panel 3. Private-Sector, Research/Academic, and Other Users. After very brief presentations from each of the panelists, the panel sessions were opened to general discussion involving the committee members, panelists, and other meeting guests. Lists of panelists and of the guests who participated in the group discussions are provided in Enclosure B. The following section on use of the FAF highlights key points arising from the three panel discussions. The committee’s findings, presented in the subsequent sections on FAF 4 The FAF covers shipments by highway, railroad, water, and air modes. 5 Time spent on the FAF project by Freight Office staff is not itemized. 6 Panel 2 was originally intended to address FAF usage by state DOTs and MPOs. However, as discussed in the section of this letter on users and applications, the committee found that MPOs have made only limited use of the FAF. Therefore, Panel 2 involved state representatives only.

4 accomplishments and on issues and options for the future, draw extensively on the workshop presentations and discussions. USE OF THE FAF The committee’s observations on use of the FAF, which are based on information gathered in planning and conducting the workshop, are presented in this section. An overview of users and applications is followed by comments on the FAF’s strengths and weaknesses. The section concludes with a summary of general comments about the FAF by workshop participants, including attempts to respond to the question, What is the FAF? Users and Applications Although the FAF was originally envisaged as an analytical tool for internal FHWA use, presentations by members of the FAF team stimulated outside interest, and use of the FAF has spread beyond the agency. In October 2002, a press release from the Secretary of Transportation announced the release of the FAF, stating that “by using this tool, state and local government and the private sector can determine which transportation corridors are or will become heavily congested in the future and better plan solutions to help alleviate these bottlenecks in the intermodal transportation network.” 7 As part of the FAF outreach effort, the Freight Office mailed 1,300 FAF CDs to mid-level managers and planners interested in freight issues. In preparing for the workshop, the committee was largely dependent on the Freight Office for help in identifying users of the FAF. However, individual committee members and TRB staff also made inquiries of individuals and organizations known to be interested in freight issues to determine whether they had used, or considered using, the FAF. Insofar as trends can be identified, it appears that, to date, the FAF has been used primarily by analysts in federal government agencies and state DOTs and by consultants investigating freight-related questions for state DOTs and private-sector clients. MPOs reported that, because the FAF does not provide the level of geographic detail required for local projects, they have not used it except as a communication tool for highlighting major freight issues. FAF use among academic researchers appears very limited, but the committee did not attempt to identify the reasons behind this observation. Examples of applications of the FAF described by participants in the three workshop panel discussions are given below. Panel 1 Federal government users reported to the committee that they have used the FAF as a freight flow database for analyses and benchmarking and as a tool to support examination of different policy scenarios. Database applications include the development of a food/ingredient supply chain model by the U.S. Department of Agriculture (USDA) to study possible supply disruptions following emergencies and a market analysis by the Maritime Administration to examine trade flows along the East Coast and opportunities to move cargoes onto coastal vessels. Scenario 7 Transportation Secretary Mineta Announces New Database to Help Public, Private Sectors Analyze and Plan for Growing Freight Movement (www.ops.fhwa.dot.gov/freight/pp/fhw4102.htm).

5 analyses using the FAF include efforts by FHWA to examine issues relating to truck size and weight and the associated potential for modal diversion. Panel 2 State representatives reported to the committee that they have used the FAF—notably the maps of freight flows—as a starting point in discussing freight issues with high-level decision makers. The FAF has proved to be an effective tool for such communication and education purposes, as illustrated by the extensive use of the FAF maps by speakers at the annual meeting of the American Association of State Highway and Transportation Officials. The FAF is far less helpful for state DOT analysts working on specific projects, such as corridor studies and traffic forecasts, because it does not provide the necessary level of geographic detail and cannot be linked to other data sets containing supplementary information. For example, analysts and planners may need to determine whether the addition of a full interchange on a key truck corridor will relieve parallel routes of truck infiltration, or whether the development of a grade-separated rail crossing will increase both road and rail productivity in an urban area. Such determinations require detailed information about items such as the volume of freight moved, the origin and destination of the shipment, the vehicle type and configuration, routing, and time of day. The FAF does not fulfill these detailed information requirements. Moreover, it does not provide a mechanism by which users seeking to overcome some of the FAF’s data limitations can link state, metropolitan, and local data sets to the bigger picture of national freight movements provided by the FAF. Panel 3 Users outside of the federal government and state DOTs reported on a variety of analytical and modeling efforts using the FAF. For example, the Port of Pittsburgh is examining possible economic and social advantages that could accrue from greater use of the inland waterway system to move freight. Several of the studies described to the committee combined the FAF with freight and economic data from other sources. For example, an examination of freight flows in Appalachia for the Appalachian Regional Commission has used the FAF motor carrier data in combination with data on rail movements from other sources. While some users were able to correlate FAF data satisfactorily with other data sets, other users reported unacceptable discrepancies and a sense of frustration that the lack of information about data, forecasts, and methodologies used in the FAF precludes efforts to resolve these difficulties. As in the case of state DOT projects, the lack of geographic detail available in the FAF, particularly for nonfederal users who cannot access the county-to-county flows, was identified as an important limitation. Several users highlighted the value of the FAF economic forecasts in identifying possible future freight transportation challenges. Strengths The features discussed in this section—the FAF maps and forecasts, and the convenience of the FAF as an analysis tool—were identified by some workshop participants as strengths of the FAF. The committee agrees that, in principle, these features are potentially useful components of a freight analysis tool, but has some concerns about their implementation in the FAF for the reasons given below.

6 Maps Workshop participants generally agreed that the FAF maps, such as the depiction of rail flows to and from Illinois illustrated in Figure 1, have been the key element in making the FAF an effective communication tool. One participant commented that the FAF would not have been successful as a communication tool without the maps, and another noted that a picture can be powerful in making a case and explaining the context, even if detailed data are absent. In addition to being used in presentations, the FAF maps have been used to illustrate reports, such as a study of North American port and intermodal systems for the U.S. Chamber of Commerce (National Chamber Foundation 2003). The committee agrees that maps of freight flows at the national level constitute a valuable illustrative tool. However, care is needed in interpreting maps of this type, as illustrated by the three maps shown in Figure 2, all of which depict international truck flows through Detroit border crossings for either 1998 (Figure 2a) or 1999 (Figures 2b and 2c). Comparison of these three maps demonstrates clearly that different underlying data sets and different mapping techniques can result in different perceptions of corridor lengths and associated demands on the transportation system. The FAF freight flows shown in Figure 2a are based on freight transportation data from both public and private sources and, because of data gaps, include some flows synthesized by using models. As already noted, detailed information on the data used to generate freight flows in FAF is not publicly available. Figures 2b and 2c were generated by using data from the Ontario Commercial Vehicle Survey (CVS), a roadside carrier survey that forms part of the larger Canadian National Roadside Survey (NRS). For the 1999 Ontario CVS, data on items such as origin and destination for both trip and commodity, axle widths, commodity, carrier, driver, highways used, stops along the route, and international border crossings were gathered for a random sample of trucks intercepted at 136 roadside data collection sites. The survey sample contains 45,650 records, of which 17,600 capture travel to the United States. Validation of the 1999 CVS data against information from the International Registration Plan, individual carriers, and corporate carrier registrations indicated errors in the survey estimates of carrier activity to be on the order of 3 to 4 percent. Figures 2a and 2b are both “step-type” maps illustrating steps or ranges of flow increments. With only four levels of network flows depicted, the degree of resolution in these maps is generally insufficient to show many of the variations in flow levels captured in the underlying data, particularly at the state level. Figure 2c is a bandwidth map illustrating three levels of network flows. Such bandwidth maps are often used to illustrate changes in flow levels and directional information. In developing the FAF maps, efforts were made to demonstrate the connectivity among different regions at the state level. As a result, the lines representing major flows emanating from transportation hubs or border crossings may be misleading. For example, Figure 2a suggests that international truck flows westward from Detroit are as heavy in Iowa as they are between Detroit and Chicago—contrary to evidence from the 1999 Canadian NRS and surveys by Michigan DOT showing that the majority of such flows go no further west than Chicago. Figure 2b, which uses the same four levels of network flows as Figure 2a, also fails to illustrate the drop in international truck flows in Iowa compared with those between Detroit and Chicago. However, the bandwidth map (Figure 2c) shows this drop clearly. Thus, even though Figures 2b and 2c were constructed on the basis of identical data, they show corridor lengths and associated demands on the transportation system that appear different, at least superficially.

FIGURE 1 Illinois rail flow map. Source: FAF website (http://www.ops.fhwa.dot.gov/freight/adfrmwrk/)

FIGURE 2a International truck flows through Detroit border crossings: FAF map. Source: FAF website (http://www.ops.fhwa.dot.gov/freight/adfrmwrk/)

FIGURE 2b International truck flows through Detroit border crossings: Ontario Commercial Vehicle Survey (CVS) step map. Source: Ontario Ministry of Transportation, Toronto, Canada

FIGURE 2c International truck flows through Detroit border crossings: Ontario CVS bandwidth map. Source: Ontario Ministry of Transportation, Toronto, Canada

11 Figures 2a and 2b, both of which are step-type maps, give different pictures of the major international truck freight corridors emanating from Detroit. In the absence of detailed information on how the FAF map was derived, the committee is unable to draw any definitive conclusions about the reasons for these differences. However, it notes that for some states, such as New York, Pennsylvania, South Carolina, Georgia, Ohio, Texas, and Idaho, Figures 2a and 2b convey different messages about trade with eastern Canada, particularly at the corridor level. Because the principal function of the FAF maps is to illustrate the importance of freight to policy makers, such maps must convey accurately information on trade corridors and associated demands on the transportation system. While the committee is not in a position to state which of the three maps in Figure 2 provides the “correct” picture of international truck flows through Detroit border crossings, it urges caution in interpreting such maps, given the differences that can result from the use of different data sets and different mapping techniques. Forecasts Some users reported that they have used the FAF because it incorporates economic forecasts for 2010 and 2020 and thus indicates how changing trade patterns may influence future freight transportation requirements. Clearly, there is value in linking trade and transportation, and the inclusion of economic forecasts is a potential strength of an analysis tool such as the FAF. Nevertheless, some specific features of the forecasts included in the current FAF give cause for concern. Because of budget and schedule constraints, relatively coarse geographic models were used to prepare the FAF forecasts. In addition, the forecasts were prepared before September 11, 2001, and so do not take account of business trends or rates of change in behavior that were not anticipated prior to that date. For example, the rate of growth in gross domestic product slowed considerably after September 11, 2001, compared with levels in the mid to late 1990s. In addition, the more rapid than expected shift in production to China has resulted in an increase in transpacific shipments arriving at California ports and a corresponding decrease in maquiladora production along the U.S.–Mexico border and associated freight flows. These trends are not reflected in the FAF forecasts. A Convenient Tool? Some users commented to the committee that the FAF, despite its data limitations, may be a convenient tool for those who have neither the time nor the knowledge to combine data from different sources. For example, while individual states may have better data on some features of the highway system than are included in the FAF, using the FAF database avoids the need to examine and combine data sets from disparate sources. The committee recognizes the potential value of a single comprehensive data source, particularly for those needing to solve practical problems with limited time and resources. However, it is concerned that the limitations of the FAF are not made clear to users and potential users. Thus, the FAF may appear, at least superficially, to be a convenient analysis tool and may be used for applications for which it is inappropriate. The reasons for the committee’s concern are explained further in the following section on weaknesses of the FAF.

12 Weaknesses Lack of Transparency As noted in the overviews of the workshop panel discussions, many FAF users are frustrated by the FAF’s lack of transparency. In addition to public data sets, the FAF incorporates proprietary private-sector data and economic forecasts. Consequently, some details of the data, forecasts, and associated methodologies are subject to disclosure constraints designed to protect commercial confidentiality. The FAF’s lack of transparency is particularly problematic because of the different types of data used. The FAF incorporates “real” data from surveys, administrative records, and the like; some of these data come from public data sets, while others come from proprietary private-sector sources. In addition, to fill data gaps, the FAF incorporates synthesized data derived from models. However, users are given no information about data reliability. More specifically, they are not told which data are real and which were synthesized. Clearly, the distinction between real and synthesized data is important for informed analyses and decision making. While data derived from surveys are estimates of varying reliability and administrative records may be unreliable because of problems with reporting procedures, such data are nonetheless real. In contrast to synthesized data, they do not depend on the assumptions built into the analytical models used to generate them and how well these models replicate reality. Data Integration Issues While they recognize that a national tool such as the FAF cannot realistically meet all data needs for state and regional analyses, state users expressed a sense of frustration that they are unable to link the FAF to their own more specialized data sets. In most urban areas, the majority of commercial vehicle miles of travel are accounted for by local truck trips. Therefore, regional, state, and local planners investigating options for mitigating congestion and improving the overall efficiency of their transportation networks need to include all freight movements in their analyses and models to understand the impact of these movements on network performance. The FAF provides information on freight flows between domestic and foreign trading partners, but it reduces metropolitan areas to single nodes on the network and does not include local flows. Moreover, because the FAF is the end product of a data integration process that cannot be fully disclosed because of commercial confidentiality constraints, it is essentially a “black box.” Therefore, regional, state, and local users have no satisfactory means of linking the contents of the box to their own data sets to obtain a complete picture of freight movements. Lack of Documentation Workshop participants, particularly those outside of the federal government, expressed frustration about the lack of official documentation from the Freight Office describing what the FAF comprises, its potential uses, and its limitations. The lack of documentation was also a source of concern to the committee in seeking to inform its assessment of future options for the FAF. Although the draft overview of the FAF provided by the Freight Office (FHWA 2003) was helpful, it does not articulate clearly the FAF’s limitations. For example, the committee only learned about the problems with the FAF forecasts from one of the workshop participants.

13 General Comments The FAF has been variously described by the Freight Office as “a policy and systems analysis tool,” “a methodology to estimate trade flows on the nation’s infrastructure,” and “the most complete database ever assembled of freight transportation flows upon the U.S. transportation infrastructure” (FAF website;8 FHWA 2003). Participants in the workshop provided insights into what they perceive the FAF to be—for example, “a breakthrough product that has enabled people to see the world of freight differently,” “a data model illustrating what we think freight flows would look like if we had better data,” and “a demonstration of the type of product that could be generated with additional and improved input.” Workshop participants agreed that the FAF is a data integration effort rather than a data collection program and is not a replacement for the CFS. Several participants emphasized that the quality of the FAF database depends on the availability and quality of the constituent data. In the case of the STB Rail Waybill Sample, for example, only the public data set can be used in the FAF because of constraints on the release of commercially sensitive data. Thus, the FAF is unable to capture some valuable data on freight movements by rail. Furthermore, improvements to the FAF will largely depend on efforts by organizations other than the Freight Office to collect better data. For example, an improved CFS could greatly benefit the FAF but will require initiatives by BTS and the Census Bureau, who share responsibility for this survey. FAF ACCOMPLISHMENTS Increased Awareness of Freight Issues Finding 1. The FAF has been successful in raising awareness of freight issues, notably among senior policy makers in USDOT, state DOTs, and the U.S. Congress. The FAF is a powerful communication tool that has proved effective in making nonspecialists, particularly high-level policy makers, more aware of major freight issues. The references to freight in both the Senate and House proposals for the reauthorization of TEA-21 signify the increased awareness of freight issues in general among many members of Congress. The maps are an essential part of the FAF communication tool. They have been used to illustrate the importance of different modes of freight transportation at a strategic national level, as well as the connectivity among regions and the associated need to involve many different jurisdictions when freight issues are addressed. Despite some problems with quantitative interpretation of the depicted freight flows, the maps provide an approximate picture of the geographic distribution of freight activity and illustrate the role of freight corridors. By translating economic forecasts for 2010 and 2020 into transportation demand and freight flows, the FAF demonstrates clearly the linkage between trade and transportation. The large anticipated increases in freight movement—a nearly 70 percent increase in domestic volumes and a near doubling of international volumes by 2020—have captured the attention of senior policy makers concerned about the associated demands on the nation’s transportation infrastructure and congestion problems. The FAF has also illustrated how geographic detail and commodity flow forecasts can bolster anecdotal arguments. In this regard it has been useful for FHWA staff in examining national 8 www.ops.fhwa.dot.gov/freight/adfrmwrk/.

14 policy scenarios within the context of the reauthorization of TEA-21—the original goal of the FAF project. Nonetheless, the committee has concerns about the ability of the FAF to provide robust numbers in support of decisions relating to reauthorization. A Valuable Demonstration Project Finding 2. The FAF has successfully demonstrated the potential value of combining data from different sources, both public and private, to create a coherent national picture of multimodal freight flows. The FAF has value over and above its constituent data sets because it provides a coherent national picture across modes, albeit a picture founded in large part on 1993 data. A major effort is required to combine data from different sources, as was done in the FAF project, and not all analysts and planners have the time or knowledge needed for such an undertaking. Thus, the FAF is perceived by some users as a convenient analysis tool, regardless of its limitations. The lack of publicly available truck data was highlighted by many workshop participants as a serious impediment to analysis and planning efforts. Motor carrier flows constitute one of the largest and fastest-growing segments of transportation demand, and good, reliable data on them are needed urgently. The FAF project attempted to address the long-standing lack of a comprehensive truck flow database by supplementing available truck data with synthesized data derived from models. Such a data integration effort is not a substitute for the collection of more and better data, but it does highlight the potential benefits of filling the “truck data gap.” The FAF has also helped demonstrate potential new applications of freight data and new opportunities for analysis, particularly outside the traditional transportation field. For example, the USDA investigation of food/ingredient supply chains made use of the FAF database of freight flows. ISSUES AND OPTIONS FOR THE FUTURE Increased Demand for Freight Data Finding 3. As a result of the greater awareness of freight issues, stimulated in part by the FAF, analysts and planners at federal, state, and local levels are increasingly encountering a need for better freight data to inform decisions about transportation projects. The workshop discussions, particularly those with state representatives, demonstrated that there is a need for a “subnational” analysis tool similar to the FAF. Several participants observed that high-level policy decisions are implemented through actions taken at state and local levels. Decisions about what actions to take require reliable data, often at a detailed level of geography, to evaluate different options and determine how best to allocate limited resources to yield the greatest benefits. Although regions, states, and metropolitan areas in general need to supplement the FAF with additional data at a finer level of geographic detail, not all have similar requirements. For example, port states may require data on truck volumes by corridor and on origins and destinations, whereas inland states may be primarily interested in through volumes. In addition, different states use different data sources that they may want to link to national data. Some

15 purchase data from commercial sources, others conduct their own surveys, and yet others obtain data from administrative records on specialized movements, such as coal hauling. Federal analysts and planners have different geographic data requirements from their regional, state, and local counterparts, but in other respects their data needs are similar. They require consistent forecasts of future freight transportation demand and reliable, transparent data to inform decision making. For many freight studies, consultants play an important role in providing analytical services, particularly to states and MPOs. For example, consultants attending the workshop estimated that at least half the states have made use of the FAF and noted that much of this use has been indirect through services provided by consultants. Thus, in addition to analysts and planners in federal, state, and local governments, there are potentially many private-sector FAF users. The committee observed the widespread “hunger” for data to be a powerful force that, in the absence of a better and more convenient database, is starting to result in use of the FAF for applications far beyond its original intent and, more importantly, inconsistent with its inherent limitations. Limitations of the FAF Finding 4. Despite its successes in illuminating major freight issues and stimulating new opportunities for analysis, the FAF is an inappropriate tool for project-specific applications at state and local levels. The FAF was originally intended to provide a broad-brush picture of freight issues for national policy analysis in support of the reauthorization of TEA-21; it was not intended to fulfill project- specific analysis and planning requirements for data and forecasts. Nonetheless, evidence from the committee’s workshop indicates that some analysts believe the FAF can be used to derive estimates of freight traffic in multistate corridors or of through traffic and control totals for state- level studies. The FAF has fundamental data limitations that render any such subnational freight flow estimates highly suspect. In general, these limitations raise questions about the suitability of the FAF for any applications requiring data of higher quality or at a finer level of geographic detail than the FAF was originally designed to provide. The committee is particularly concerned that ill-informed policy and investment decisions could result because users and potential users are not alerted to the deficiencies in the FAF forecasts and data and to the associated limitations of the FAF as an analytical tool. The lack of FAF user groups and training further increases the risk of misuse. Although combining public and private data to obtain a more comprehensive picture of freight movements certainly has value, the inability to distinguish real from synthesized data, the total lack of information on data reliability, and the unknown assumptions in the economic forecasts render the FAF an opaque analysis tool. The lack of transparency has become an issue because analysts outside of FHWA are applying the FAF in ways that go beyond its intended use. The FAF project as originally conceived had no requirement for public release of proprietary data and forecasts. The FAF has other deficiencies that limit its application. The inability to integrate data from other sources has already been discussed, as has the need to update the economic forecasts. Many of the underlying data used in the FAF are also out of date. In particular, the FAF relies heavily on the 1993 CFS, although 1997 CFS data are available and 2002 CFS data are expected in 2004. The following are further deficiencies:

16 • Lack of a mode choice analysis capability. The FAF uses separate truck and rail freight origin–destination data, so any mode split results apparent in the FAF output are actually modal database comparisons. This feature severely limits the usefulness of the FAF for analysts seeking to investigate truck-to-rail diversion. Such diversion is an important issue in current freight policy, as illustrated by comments from workshop participants and examples such as the expansion of capacity on the I-81 corridor in Virginia (TRB 2003b). • Inadequate validation. Validation of FAF outputs appears to have been limited to general or anecdotal approaches. No quantitative statement about the reliability of the FAF estimates is available, which suggests that the FAF freight flows were not systematically calibrated or validated against independent data sources, such as national trade statistics or state-level motor carrier data gathered for regulatory purposes. Without some assurance that the FAF base year data satisfactorily replicate freight flows at strategic points across the nation, the validity of the FAF as a basis for investment and policy decisions is questionable. • Limited capture of freight data sources. As already noted, data from the STB Rail Waybill Sample are only partially captured in the FAF because of commercial confidentiality constraints. In addition, some useful sources of freight data, such as truck survey data from states and MPOs, were not incorporated into the FAF. The committee shares the concern of many users about the lack of documentation explaining how the FAF was generated and clearly identifying its limitations. The committee is particularly concerned that the FAF may be perceived as an official data and modeling tool carrying the imprimatur of the federal government and that the FAF database may be assumed to be analogous to transparent and well-documented data sources such as the CFS. A Foundation for Further Development? Finding 5. Despite its value as a demonstration project, the FAF has fundamental limitations that render it unsuitable as a basis for a future national freight data program. Some within the Freight Office have suggested that, ultimately, the FAF “could become the basis of a broader, integrated data and modeling program within USDOT, allowing both national and state level policy analysis, planning analysis, and simulation and modeling tools to examine traffic operations” (FHWA 2003). The committee recognizes the pressing need of the community of freight analysts and planners for such a tool. It also recognizes that, for users who understand its origins and limitations, the FAF has been useful in scoping problems and supporting “educated guesses,” particularly when it is used in conjunction with other data sources. Nonetheless, the FAF is not a substitute for primary data sources, such as the CFS and the STB Waybill Sample. Although users seeking to combine freight flow data with economic forecasts may be tempted to use the FAF as their primary data source, the convenience of the FAF has been achieved by sacrificing transparency and, in some cases, data quality. Because the FAF is a data integration effort rather than a data collection initiative, it is not clear to the committee that any future version of the FAF could transcend some of the fundamental limitations of the underlying data sets. For example, the CFS, a key data source used in constructing the FAF, is itself of limited use for a number of applications because of gaps in shipment and industry coverage, insufficient geographic and commodity detail at state and local levels, and the inability to capture rapid changes in economic cycles (see, for example, TRB 2003a; TRB 2003b). The FAF attempts to remedy some of these deficiencies by incorporating

17 data from other sources, even though the inclusion of proprietary data compromises the transparency of the product. In the committee’s judgment, the FAF project has not demonstrated satisfactory models for public–private data partnerships—in large part because the original project had no requirement to develop such models. Although the FAF has demonstrated some of the potential advantages of combining public and private data sets, the fundamental tension between transparency needs and disclosure constraints remains a source of difficulty. Users are frustrated by the lack of transparency in the data and forecasts, and the private-sector FAF partners have legitimate concerns about protecting their business interests. The committee notes that providing some degree of transparency need not necessarily compromise the confidentiality of data providers. For example, information about methods used to develop the FAF data sets could be released without disclosing the confidential data elements. Another possibility, suggested at the workshop, is to flag data items of unknown or questionable reliability, thereby alerting users that additional data purchases or modeling could be useful. Such an approach is attractive in principle, although appropriate statistical methods for assessing the reliability of FAF freight flows may not be available. The committee also identified the lack of a data integration capability as a fundamental design flaw in the FAF. The decision to reduce metropolitan areas to single points in the FAF network obviated the need to include local flows in the model. However, this sensible design decision in the context of the FAF’s intended use as a national analysis tool becomes an important limitation in attempting to evaluate state and local freight flow patterns. In general, planners and analysts require precise and distinct layers of data at local, state, regional, national, and international levels, depending on the subjects under study. The FAF does not provide resolution at subnational levels, leaving the user to speculate about the extent to which state and local freight flows have been incorporated and how they have been calibrated. Furthermore, because the FAF is a black box, the user has no satisfactory means of integrating the FAF with more specialized state and local data sets that provide greater geographic detail. While the committee acknowledges the FAF’s value as a demonstration project conducted under significant time and resource constraints, it concludes that the FAF’s limitations cannot be overcome retrospectively as part of an upgrading and revision process. Initiatives targeting the needs of freight transportation analysts and planners will not ultimately be successful unless they address fundamental issues, such as the following: • Filling data gaps, • Improving data quality, and • Evaluating approaches to combining public and private data sets that respect commercial confidentiality requirements while also providing the transparency needed for informed decision making. The committee’s recommendations address these issues, as well as short-term initiatives to improve some aspects of the FAF.

18 RECOMMENDATIONS Recommendation 1. The Freight Office should limit further work on the FAF to short-term efforts that: • Elevate awareness of freight issues in the context of the reauthorization of TEA-21, and • Clarify and document the FAF to assist analysts and planners in understanding its scope and limitations. In examining various uses—and misuses—of the FAF, the committee observed there to be a hierarchy of data quality needs. Some major freight issues can be communicated effectively to nonspecialists even if some of the underlying data are missing, as illustrated by the FAF maps. Project-specific planning, in turn, requires better-quality data to ensure that investment decisions are not based on unreliable information. Finally, regulatory analyses—such as efforts to estimate mobile source emissions for assessing compliance with clean air statutes—require an even higher level of data quality because of the potential for litigation. The committee recommends strongly against using the FAF for the more demanding requirements, namely, project-specific planning and regulatory analyses, because of concerns about the quality of the underlying data, particularly at subnational levels. However, it recognizes that continued use of the FAF for communication and education purposes may be of value for the purposes of reauthorization. In this context, the Freight Office should consider whether modest updates of some components of the FAF could remedy some of the FAF’s more serious deficiencies in highlighting broad freight issues and could be performed within the time frame necessary to inform reauthorization. The committee is particularly concerned that the outdated and misleading economic forecasts do not form a robust foundation for decision making. While the committee would like to see the FAF project superseded in the near future by a national freight data program, it acknowledges that the FAF will continue to be available to analysts and planners. Therefore, the committee urges the Freight Office to provide a fact sheet for users describing the main features and limitations of the FAF. Some clarification of the term “framework” could be helpful because the FAF data, models, and forecasts do not constitute a tool that is broadly applicable to a variety of freight transportation analyses. In addition, given the widespread use of the FAF maps for communication purposes and the need for caution in their interpretation, the fact sheet should explain how the maps were generated and provide guidance on their uses and limitations, together with illustrative examples. The Freight Office should also make the FAF more transparent to users through the release of detailed information about data sources and methods. Recommendation 2. The Freight Office, in cooperation with other groups within FHWA, BTS, and other USDOT modal administrations, should develop an action plan with immediate, midrange, and long-term activities to transition from the FAF to a broader national freight data program. Experience with the FAF has raised many of the same issues addressed in a recent report describing the need for a national freight data framework (TRB 2003b). The committee urges the Freight Office and its partners to consider the recommendations of this report in developing a plan for a national freight data program.

19 Developing and implementing a national freight data program will likely require a sustained effort over a period of 7 to 10 years. In the interim, users need data to support decision making. Therefore, the committee suggests that the Freight Office and its partners develop a series of illustrative case studies, with emphasis on state and local needs, to provide interim guidance. The case studies would identify useful sources of freight data and the strengths and weaknesses of these data, including data gaps. Guidance on methodologies for combining data from different sources would also be provided and would draw extensively on experience gained in the FAF project. The committee urges the Freight Office and its partners to undertake research on issues pertinent to the development of a national freight data program. The research should focus on challenges highlighted in the FAF project. For example, data collection and processing methods that facilitate combination of data from different sources should be investigated, with emphasis on the integration of national data with more specialized state and local data sets. Pilot truck surveys should be undertaken, and methods for combining truck count and freight flow data by matching point and flow estimates could be valuable in generating much-needed high-quality truck data. Various models for public–private data partnerships should be investigated with a view to maximizing data transparency while respecting the private sector’s need to protect data sources. In this context, the role of private-sector organizations in providing data and analytical services should also be recognized. The committee also suggests that the Freight Office and its partners investigate methods for database validation and for modeling to generate simulated freight flows when data are not available. Opportunities to link transportation data to broader economic measures, such as national accounts, may also merit investigation as part of an effort to make the proposed national freight data program useful for a wide range of applications. CLOSING REMARKS The committee appreciates this opportunity to review and comment on the FAF and hopes that its recommendations will assist the Freight Office in building on the experience of this valuable demonstration project by working with its USDOT partners and others to develop a national freight data program. Sincerely, Arnim H. Meyburg Chair Committee on the Future of the Federal Highway Administration’s Freight Analysis Framework Enclosure A: Committee membership Enclosure B: Workshop presenters and panelists

20 References FHWA. 2003. Overview of the Freight Analysis Framework (unpublished draft). Office of Freight Management and Operations, U.S. Department of Transportation. National Chamber Foundation. 2003. Trade and Transportation: A Study of North American Port and Intermodal Systems. U.S. Chamber of Commerce, Washington, D.C. TRB. 2003a. Letter report on the Commodity Flow Survey. Committee to Review the Bureau of Transportation Statistics’ Survey Programs. gulliver.trb.org/publications/reports/bts_cfs.pdf. TRB. 2003b. Special Report 276: A Concept for a National Freight Data Program. National Research Council, Washington, D.C.