Guide to Effective Methods for Setting Transportation Performance Targets (2022) / Chapter Skim
Currently Skimming:
Pages 78-101
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 78... ...
78 Measures The third FHWA performance measure rule established six measures for assessing the performance of the NHS, freight movement on the Interstate system, and CMAQ. This section of the guide gives an overview of the travel time reliability and freight reliability measures.
|
From page 79... ...
Target-Setting Methods for Reliability 79 become available in the future, this challenge should begin to diminish. For now, it limits states' ability to conduct robust analyses.
|
From page 80... ...
80 Guide to Effective Methods for Setting Transportation Performance Targets Method Strengths Limitations Other Considerations Building off the baseline with assumptions Maintaining the baseline level as the target or making an adjustment based on judgement Simple, easy to communicate, and often brings in stakeholders No rigorous analytical methods are used for the adjustments Method for agencies with limited data; agency will need to decide which exogenous factors are relevant Time-series trend analysis Forecast based simply on historical performance trend Simple while still being data driven No insights into causes of outcomes; misses incorporation of new factors that may influence targets in the future May result in a worsening target, which can pose communication challenges; may be useful for agencies with limited data on exogenous factors Trend plus other factors Expands upon trend analysis to account for other factors that may shift future performance Data driven; allows for consideration of additional factors There may still be no rigorous methods for the adjustments -- sometimes adjustments may not be data driven May result in a worsening target; agency will need to decide which exogenous factors are relevant Performance risk analysis Uses monthly performance data to calculate a standard deviation and then uses the deviation to assess confidence level to set target Data driven; allows for deeper scrutiny of the observed variation in the past performance; helps to make an informed decision on the possible future range for the target Data may be limited for robust analysis; no insight into causes of outcomes, unless paired with other method; misses incorporation of new factors that may influence targets in the future Using target ranges often seems to lean toward selecting conservative targets for which there is a high likelihood of meeting the target Segment risk analysis Focuses on segmentlevel data to assess segments that are at risk of shifting across the threshold of a reliable segment Introduces secondary analysis onto the reliability calculation; more customized approach Requires additional, somewhat complex analysis of individual segments May result in a worsening target Multivariable statistical model Regression analysis or tool developed to account for various factors to predict performance; typically applied at the segment level Fuller understanding of causes of outcomes, fully data-driven, and may support linking the targetsetting process with decision-making by informing what factors can be influenced Complex, requiring analytical and data skills; harder to communicate the method and nuance to stakeholders; may result in a worsening target A sophisticated model will require significant data gathering and indepth knowledge of application of statistical models Table 11. Target-setting methods for travel time reliability and freight reliability.
|
From page 81... ...
81 What It Is Building off the baseline with assumptions refers to a pivot off the baseline value with some assumptions, either to maintain the baseline level or to adjust on the basis of a consideration of factors that might affect future performance. This is a more qualitative approach to target setting, often selected in recognition of the limited data available for freight and reliability measures.
|
From page 82... ...
82 Guide to Effective Methods for Setting Transportation Performance Targets Advantages This method is simple, easy to communicate, and often encourages coordination within agencies. Limitations No rigorous analytical methods are used for the adjustments when setting the targets.
|
From page 83... ...
83 What It Is Time series trend analysis is a simple method that can be used to forecast performance measures for travel time reliability and freight reliability. State DOTs can choose to use a linear forecasting function or one of the other functions available in their analysis software (e.g., exponential function)
|
From page 84... ...
84 Guide to Effective Methods for Setting Transportation Performance Targets a linear function or one of the other functions (e.g., exponential, logarithmic) available in the analysis software to generate the trend line.
|
From page 85... ...
Target-Setting Methods for Reliability 85 pre-COVID regime. The Oklahoma DOT indicated in the pilot test that the agency planned to exclude Interstate reliability data from March 2020 to February 2021 when it developed targets by using linear forecasting of observed monthly Interstate reliability values.
|
From page 86... ...
86 What It Is The trend plus other factors method expands on a trend analysis by including additional data, factors, or other inputs that are expected to influence performance beyond trend line performance levels. This can include relevant external factors such as economic indicators, travel projections, and population growth as well as factors such as anticipated construction impacts or network improvements that are closely tied to travel time reliability.
|
From page 87... ...
Target-Setting Methods for Reliability 87 a linear function or one of the other functions (e.g., exponential, logarithmic) available in the analysis software.
|
From page 88... ...
88 Guide to Effective Methods for Setting Transportation Performance Targets on analyzing isolated unreliable hot spots and conducting customized analyses for different regions to account for differing factors and variables unique to specific regions. This allowed for improved, tailored consideration of unique geographic features of different regions.
|
From page 89... ...
Target-Setting Methods for Reliability 89 Figure 20. Interstate reliability mapped against employment estimates.
|
From page 90... ...
90 What It Is Performance risk analysis is a data-driven approach that uses statistical analysis to explore the variation in performance levels to account for risk in setting a target. Under this approach, data on monthly performance are used to calculate a standard deviation, and then the deviation is used to assess a confidence level for likely future performance levels to account for risk.
|
From page 91... ...
Target-Setting Methods for Reliability 91 Step 4: Identify Target Level Use the statistical software to identify the target level according to the confidence level desired. Advantages This method is data driven.
|
From page 92... ...
Source: Iowa DOT (2020b)
|
From page 93... ...
Target-Setting Methods for Reliability 93 Key 2017 2018 2019 2020 2021 Year Upper Outlier Lower Outlier Median Mean 3rd Quartile 1st Quartile Upper Whisker Lower Whisker Whiskers extend to the minimum and maximum data points within 1.5 times the range from the 1st Quartile to the 3rd Quartile, from the bottom and the top of the box, respectively. Figure 22.
|
From page 94... ...
94 What It Is A segment risk analysis describes an approach to travel time reliability and freight reliability measures that relies directly on an analysis of individual segments to identify those that are likely to shift from reliable to unreliable or vice versa. For the travel time reliability measures, FHWA defined the threshold for a reliable segment as an LOTTR of less than 1.50 for all four time periods analyzed, where LOTTR is calculated as the ratio of the 80th percentile travel time to the 50th percentile (normal)
|
From page 95... ...
Target-Setting Methods for Reliability 95 Step 1: Obtain Segment-Level Performance Measures Calculate or obtain the performance metrics (LOTTR or TTTR) for individual segments with the RITIS NPMRDS analytics tool (e.g., for TMCs)
|
From page 96... ...
96 Guide to Effective Methods for Setting Transportation Performance Targets • Barely Bad (1.5–1.6)
|
From page 97... ...
Figure 23. Minnesota DOT's application of target-setting segment risk analysis method.
|
From page 98... ...
98 What It Is A multivariable model is a formal effort to quantitatively forecast performance by using methods more sophisticated than trend analysis based on historical performance; it typically integrates various factors into a forecast. Whereas multivariable models for safety measures are often developed at an aggregate level to calculate total fatalities or serious injuries, multivariable models in the context of reliability and freight measures are often developed to support the calculation of metrics that are applied at the segment level and then used to calculate the systemwide measure.
|
From page 99... ...
Target-Setting Methods for Reliability 99 models will be tweaked with new findings or data, these factors can be removed if there is no statistically significant relationship or high correlation with other factors. Step 2: Collect Data Collecting data can be the most time-consuming part of developing a new model, especially one that is very detailed and that requires disaggregated data.
|
From page 100... ...
100 Guide to Effective Methods for Setting Transportation Performance Targets to associate segment-level LOTTR/TTTR with volume, capacity, and roadway attributes. Next, it created a forecast of future volume on the basis of current growth rates and updated the levels of future capacity for each segment on the basis of planned projects.
|
From page 101... ...
Target-Setting Methods for Reliability 101 considered negligible. The 2020 error rate was higher and can perhaps be attributed to unusual reliability because of the impact of the pandemic on travel.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.