Transferability of Activity-Based Model Parameters (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

12 As described in Chapter 1, assessment of the effectuality of transferring the Sacramento activity-based model specification to the Jacksonville and Tampa regions followed two primary avenues of inquiry. The first was a strict test of the transfer- ability of estimated parameters, with the overall objective of finding out whether the behavioral sensitivities that drive model specification in one region (Sacramento) are also evident when one tries to estimate parameters for the same specifica- tion in other regions (Jacksonville and Tampa Bay) and where the differences lie. The second test was designed to represent what an agency would actually do if it was to transfer a model. This involved starting with the Sacramento specification, cali- brating regional models for Jacksonville and Tampa Bay, deter- mining whether target values could be met for each region with a reasonable amount of effort, and identifying where the calibrated parameter revealed regional differences. Estimation of Model Components to Test Transferability The consultant team tested reestimation of the models, keeping the specification of the Sacramento models but reestimating all of the parameters using the 2009 NHTS data for the Jacksonville and Tampa regions and comparing the estimated coefficients to the Sacramento-based coefficients to look for significant differences in the estimates. This is a somewhat stricter test of transferability because it tests the transferability of every coefficient in a model, rather than the predictive accuracy of the model as a whole. As it is a stricter test of transferability, it relies on larger survey sample sizes than are needed simply for calibration of the marginal distributions. In that regard, the rather small sample sizes available from the NHTS data for the Florida regions made the estimation-based transferability tests inconclusive, as will be discussed in the next section. The DaySim activity-based model software is designed to be used in both model estimation and model application, with the same code used to specify the models in each case. This has several advantages: • It avoids coding errors when estimated models are prepared for application. • It is easy to go back and change the models slightly by adding or deleting new variables and reestimating. • It is efficient to estimate previously specified models on new survey data when such data become available. For this project, the code for the existing DaySim models for the Sacramento region activity-based model system (SACSIM) was used to estimate models with the same specification on new survey data for the Jacksonville and Tampa Bay regions of Florida. The survey data for both regions are from the 2008–2009 National Household Travel Survey (FHWA 2009). Transferability Tests for Tampa and Jacksonville There are more data for the Florida regions than for most regions of the United States, because the State of Florida paid for an add-on sample as part of the NHTS. The transferred Sacramento model (called SACSIM) was developed from a 2000 survey of 3,942 households. The 2008–2009 NHTS Florida Add-On Survey captured responses from 1,335 households in the Jacksonville region and 2,517 households in the Tampa Bay region; however, it did not include separate records of household members younger than 5 years of age, and some adults did not provide complete diaries. Moreover, 28% of the household diary days in the Jacksonville and Tampa Bay samples were weekends, which could not be used for estimating models of daily patterns, tours, or trips due to the weekday focus of the model specification. Only the auto ownership model could be estimated using the full sample. For each model component in DaySim, the approach was to estimate separate models on the data from the three regions C h a p T E R 3 Findings and Applications

13 (Sacramento, Tampa, Jacksonville) and then compare the estimated coefficients along with the standard errors of those estimates to identify cases in which the estimated coefficients were significantly different across regions. There were six model components that were not included in the estimation process. Three of these were not included because the NHTS data do not have data to estimate three models that are in the SACSIM system. Those are (1) usual school location for all students, (2) transit pass ownership for all adults, and (3) availability of free parking at work for all workers. Three additional models were not included in this particular analysis because they proved too difficult to estimate on the new data, and the consultant team had previous results for those models from a similar FHWA project on model transferability (discussed later in this chapter). Those models are (1) usual work location for all workers; (2) work-tour destination choice, conditional on usual work location; and (3) day-pattern choice. Seventeen component models were included in this analysis. Detailed estimation results and comparisons for those models are provided in Appendix A for three models: • A model estimated on the 2000 Sacramento regional house- hold travel survey data (the basis for the revised SACSIM activity-based model implementation); • A model estimated on the Tampa region 2009 NHTS (weekday) survey data; and • A model estimated on the Jacksonville region 2009 NHTS (weekday) survey data. Each table in Appendix A represents a separate model and includes six sets of values. The first three sections are the esti- mated coefficients and t-statistics for the original Sacramento model and the new Tampa and Jacksonville models, respectively. The next three column groups show differences between parameters, expressed as pairwise t-tests for each pair of models: • Jacksonville versus Tampa; • Tampa versus Sacramento; and • Jacksonville versus Sacramento. The results indicate whether a difference between an esti- mated parameter for each region is statistically significant, either positive (green shades) or negative (red shades). For each pair, there are two sets of t-statistics, one using one city’s standard errors as a base and the other using the other city’s standard errors as a base. For most parameters the test out- comes are the same irrespective of which set of standard errors is used. In the discussion that follows, summaries of sample sizes, numbers of statistically significant parameters, common statistically significant parameters, and significant differences in estimated parameter values for all 17 model components and each region are shown in Tables 3.1–3.5. Sample Size In Appendix A, the relevant sample size (number of obser- vations) is given at the top of each table for each model. As already mentioned, sample sizes were reduced due to incom- plete diaries and unusable weekend observations. A summary of the sample sizes used to estimate various model components appears in Table 3.1. For example, for the work-tour-mode choice model (Appendix A, Table A.10), there are 3,313 relevant work tours in the Sacramento data, compared with just 844 in the Tampa data and 506 in the Jacksonville data. Thus, even the combined sample size for the Florida regions is less than half as large as the Sacramento sample size. The small sample sizes are problematic for this study for a number of reasons, including those already mentioned in the previous paragraphs. Another drawback of limited sample sizes is that there are sometimes not enough cases in which particular choice alter- natives in the models are chosen to be able to estimate utility coefficients for those alternatives. The coefficients that were constrained at a particular value rather than estimated were a result of too few observations to estimate the particular variable from the data. For example, in the mode choice models, there were very few observations in the Florida NHTS where “bike” or “transit” were chosen; so many of those parameters could not be estimated for the Florida regions. It is not possible to say anything about model transferability in cases where parameters cannot be estimated. In such a case, the only options are to (a) transfer parameters from an existing model, such as the Sacramento models, or (b) collect additional travel survey data, preferably oversampling in specific geographic areas to obtain more transit, bike, and walk trips data. Statistical Significance of Estimated Parameters The effects of sample size on the ability to estimate statisti- cally significant parameters are reflected in the results shown in Table 3.2. The 17 models that were estimated as part of this study ranged in complexity from a model with just 14 param- eters to models with more than 100. For this study a statistical significance level of 0.05 is used, implying a threshold of t-statistic ≥ 1.96 and corresponding to a 95% confidence interval, a fairly conservative standard. As can be seen in the table, in the original Sacramento specification more than one-third of the model parameters were not statistically sig- nificant at the 0.05 level. Of these nonsignificant parameters, a majority of parameters were estimated but did not achieve the 0.05 level, while many others were constrained to specific

14 Table 3.1. Number of Observations Used to Estimate Tampa and Jacksonville DaySim Models Tables in Appendix A Name of Choice Model Component Number of Observations (% of sample size of corresponding Sacramento model) Tampa Jacksonville A.1–A.2 Household auto ownership 2,517 (63.9%) 1,335 (33.9%) A.3–A.5 Person exact number of tours 3,820 (38.1%) 2,195 (21.9%) A.6–A.8 Nonmandatory-tour destination 2,912 (46.8%) 1,601 (25.7%) A.9 Work-based sub-tour generation 513 (21.5%) 331 (13.9%) A.10 Work-tour mode 844 (25.5%) 506 (15.3%) A.11 School-tour mode 211 (13.5%) 166 (10.6%) A.12 Escort-tour mode 353 (24.7%) 229 (16%) A.13 Other home-based-tour mode 2,569 (57.1%) 1,373 (30.5%) A.14 Work-based sub-tour mode 107 (18%) 81 (13.6%) A.15–A.17 Work-tour arrival and departure times 846 (25.4%) 511 (15.4%) A.18–A.19 School-tour arrival and departure times 155 (10.9%) 113 (7.9%) A.20–A.22 Other home-based-tour arrival and departure times 3,006 (50.6%) 1,609 (27.1%) A.23–A.24 Work-based sub-tour arrival and departure times 106 (18.1%) 83 (14.1%) A.25–A.27 Intermediate-stop generation and purpose 7,625 (33.3%) 4,370 (19.1%) A.28–A.30 Intermediate-stop destination 1,487 (18.5%) 1,062 (13.2%) A.31–A.32 Trip-level mode 9,876 (33.3%) 5,741 (19.4%) A.33–A.34 Intermediate-stop departure time 3,894 (43.4%) 2,009 (22.4%) values to maintain theoretical relationships. For example, some choice alternatives were made effectively unavailable by setting alternative-specific constants to -10. In total, 64% of the parameters specified in the 17 Sacra- mento models were statistically significant at the 0.05 level or better. In comparison, 40% of the Tampa and 36% of the Jacksonville models’ estimated parameters were significant. The models that had the largest numbers of statistically signifi- cant parameters included the nonmandatory-tour destination choice, intermediate-stop generation and purpose choice, and intermediate-stop destination choice models. Proportionally, there were more significant parameters estimated for the trip- level mode choice and intermediate-stop departure time choice models. Each of these models has more than 1,000 observations for all three model regions. Interestingly, these models are all tour- or trip-level decision contexts that do not focus on work or school. In contrast, the models at the top of Table 3.2 are long-term or day-level choice models. Despite having large numbers of observations and total parameters, the “person exact number of tours model” and the “household auto ownership model” have relatively fewer significant parameter estimates. The work-, school-, and escort-tour mode choice models have both fewer observations and proportionally fewer significant estimated parameters. Among these models, there are many fewer signifi- cant parameters in the Tampa and Jacksonville models com- pared with the Sacramento model. An additional observation is that, with the exception of the work-based sub-tour arrival and departure time model, which is an anomaly due to too few observations, the time-of-day choice models seem to have relatively more significant parameter estimates. Tour- and trip-level choice models are observed more times per person than household-level and day-pattern decisions. In addition, work- and school-related choices are typically observed just once per day. Fewer observations lead to fewer significant parameter estimates. This is only part of the story, however. As shown in Table 3.1, the “person exact number of tours model” and the “household auto ownership model” have more than 1,000 observations in each model region. These models rely primarily on specifications in which household and person attributes explain variations in choice behavior and are further removed from the direct effect of transportation

15 system level-of-service variables, despite including a com- posite accessibility variable. As shown in Table 3.2, the “person exact number of tours model” in particular has more than 100 parameters, nearly all of which are constants that represent interactions between tour purposes or day-pattern dimen- sions and person types. A relatively small proportion of these parameters are significant, particularly in the Tampa and Jacksonville models. Nonsignificant parameters are retained in these models to represent theoretically desirable segmenta- tion of person and household types and can be affected by which alternative is selected as the base alternative with constant of zero, or by which person type is chosen as the reference per- son type, for example. Since decision-maker attributes such as household and person demographics do not vary across choice alternatives, using them to explain variation requires estimating bias constants that are alternative specific or that represent subsets of alternatives. For alternative-specific param- eters to be statistically significant, there must be a sufficient number of observations in which an attribute varies for the same chosen alternative. For example, if all of the persons observed to make choice “A” came from the same income group “X,” then it would not be possible to estimate a parameter for the effect of income on the propensity to choose “A.” In contrast, the intermediate-stop, non-work/school tour destination, trip-mode, and arrival and departure time choice models are less dependent on constants and make more direct use of travel time and cost variables in their specification. Since travel times and costs vary over alternatives, a generic coefficient (e.g., the marginal utility of travel time) may be estimated and applied to multiple alternatives; that is a more efficient way to represent variation in choices and much easier in terms of obtaining statistically significant coefficient estimates. Comparing the model regions in Table 3.2, all of the Sacra- mento model components have more statistically significant parameters than the corresponding Tampa and Jacksonville model components, which is to be expected because Sacra- mento was the source specification and had nearly as many observations as the other two regions combined. In addition, the Tampa model has more statistically significant estimated parameters than the Jacksonville model for all but a few models, Table 3.2. Statistically Significant Estimated Parameters in Sacramento, Tampa, and Jacksonville DaySim Models Tables in Appendix A Name of Choice Model Component Total Parameters in Model Number of Model Parameters Estimated at 0.05 Significance: t-Statistic >– 1.96 Sacramento Tampa Jacksonville A.1–A.2 Household auto ownership 60 38 29 27 A.3–A.5 Person exact number of tours 101 39 16 11 A.6–A.8 Nonmandatory tour destination 94 79 52 45 A.9 Work-based sub-tour generation 14 9 6 2 A.10 Work-tour mode 36 25 3 1 A.11 School-tour mode 41 23 6 9 A.12 Escort-tour mode 15 8 7 8 A.13 Other home-based-tour mode 44 28 23 20 A.14 Work-based sub-tour mode 16 11 3 5 A.15–A.17 Work-tour arrival and departure times 71 48 39 29 A.18–A.19 School-tour arrival and departure times 59 38 15 14 A.20–A.22 Other home-based-tour arrival and departure times 92 57 45 37 A.23–A.24 Work-based sub-tour arrival and departure timesa 47 0 5 7 A.25–A.27 Intermediate-stop generation and purpose 106 88 62 61 A.28–A.30 Intermediate-stop destination 92 60 21 32 A.31–A.32 Trip-level mode 62 48 34 29 A.33–A.34 Intermediate-stop departure time 49 39 30 27 Sum (%) of all 17 models 999 638 (64%) 396 (40%) 364 (36%) a The Sacramento version of the work-based sub-tour arrival and departure time choice model has only constrained parameters because work-based sub-tours are observed for only a limited portion of the day, the majority being around lunch times.

16 which was also expected due to its larger sample sizes. The one noteworthy exception is the intermediate-stop destination choice model for which Jacksonville has 10 more significant parameters than the corresponding Tampa model, an observation that is explored in more detail in the next section. Commonality of Statistically Significant Parameters Several interesting questions arise. How many statistically significant parameters do these regions have in common? And which models have the most significant parameters in common? The answers to these questions may be found in Table 3.3. Aggregating over all 17 model components in Table 3.3, the regional combination of Jacksonville-Tampa has 27% agreement in terms of statistically significant esti- mated parameters, the lowest percentage of the three pos- sible combinations. The Tampa-Sacramento combination has the highest percentage of common statistically significant estimated parameters with 33%. The Jacksonville-Sacramento combination has 31% in common. These values are shown in the row third from the bottom of Table 3.3. If the joint occurrence of statistically significant parameters were random, then the probability of any pair of models having a statistically significant parameter in common would be the product of the individual probabilities. For example, from Table 3.2 it may be seen that the proportion of statisti- cally significant parameters in the Jacksonville and Tampa models were 0.36 and 0.40, respectively. Thus, the joint prob- ability of the Jacksonville and Tampa models having a statisti- cally significant parameter in common under the assumption of independence is 0.36 p 0.40 = 0.14. Joint probability calcu- lations under assumptions of independence are shown on the row second from the bottom of Table 3.3, including the three-way combination for all three models in the right-most column. The last row of Table 3.3 shows the ratio of observed-to- expected ratios (OE ratios) for each pair of regional models Table 3.3. Statistically Significant Estimated Parameters That Sacramento, Tampa, and Jacksonville DaySim Models Have in Common Tables in Appendix A Name of Choice Model Component Number of Estimated Parameters That Were Significant at 0.05 Level Common to Pairs and All Three Models JAX and TPA TPA and SAC JAX and SAC All Three A.1–A.2 Household auto ownership 22 24 21 20 A.3–A.5 Person exact number of tours 9 12 8 7 A.6–A.8 Nonmandatory tour destination 28 45 39 24 A.9 Work-based sub-tour generation 2 6 2 2 A.10 Work-tour mode 1 3 1 1 A.11 School-tour mode 4 4 6 3 A.12 Escort-tour mode 7 5 6 5 A.13 Other home-based-tour mode 17 18 17 15 A.14 Work-based sub-tour mode 3 3 4 3 A.15–A.17 Work-tour arrival and departure times 28 35 29 28 A.18–A.19 School-tour arrival and departure times 6 10 10 3 A.20–A.22 Other home-based-tour arrival and departure times 27 36 29 25 A.23–A.24 Work-based sub-tour arrival and departure times 2 0 0 0 A.25–A.27 Intermediate-stop generation and purpose 49 57 57 48 A.28–A.30 Intermediate-stop destination 14 19 29 13 A.31–A.32 Trip-level mode 25 31 27 24 A.33–A.34 Intermediate-stop departure time 21 26 24 18 P1: Percent of 999 total parameters in all 17 models 27% 33% 31% 24% P2: Joint probability if independent (see Table 3.2) 14% 25% 23% 9% OE ratio: observed-to-expected (P1/P2) 1.84 1.32 1.33 2.59

17 and for all three in combination, where P1 is observed propor- tion and P2 is the expected proportion under independence assumptions. As a rough indicator of correlation between com- mon components, OE ratios indicate that the Jacksonville and Tampa models are more strongly correlated than either the Jacksonville-Sacramento combination or the Tampa- Sacramento combination. For example, the table indicates that the Jacksonville-Tampa pair is 84% more likely to have statistically significant parameters in common than would be expected under the independence assumption, whereas the Jacksonville-Sacramento and Tampa-Sacramento combi- nations are only one-third more likely. While this could say something about the underlying behavior of households and travelers in each region, it does not indicate the direction of correlation, and is more likely measuring the effect of common- alities in the survey instruments and sampling of households. Both Florida regions used the same 2008–2009 NHTS Add-On Survey instrument and sampling methodology, whereas the Sacramento model used a separate 2000 household survey. The OE ratio for the pairs of regional model systems also suggests the degree to which certain parameters that are common to these models are more consistently estimated than others. Here, consistency of estimation refers to the ability to measure the variable of interest consistently and to obtain observations with sufficient variation in values over the observed choices. Considering the joint probability of common significant parameter estimates in all three regional models, the OE ratio of 2.59 indicates a much higher than expected (159%) correspondence between statistically signifi- cant parameters than if the three were independent, despite the fact that two different survey instruments were used. This smaller set of statistically significant parameters that are com- mon to the trio of models may be assessed with even greater confidence. Table 3.3 also shows the number of common, significant parameters for each of the 17 model components. Intuitively, the model components that had the largest numbers of statis- tically significant estimated parameters, as shown in Table 3.2, also tend to have more significant parameters in common with the same model component in other regions, as shown in Table 3.3. The intermediate-stop generation and purpose model, nonmandatory-tour destination, work-tour arrival and departure times, and trip-level mode choice models stand out as having many statistically significant parameters in common. OE ratio analysis was applied to each of the 17 model com- ponents as well, and these results are summarized in Table 3.4. For all 17 model components, the Tampa-Sacramento and Jacksonville-Sacramento pairs have very similar OE ratios, and the Jacksonville-Tampa combination has a higher OE ratio in every case, sometimes much higher. The work-tour- mode choice model stands out for the following reason. Out of 36 modeled parameters, 25 were statistically significant in the Sacramento model, but only three in the Tampa specification and just one in the Jacksonville specification (see Table 3.2). Nevertheless, that one statistically significant parameter was significant in all three regional models, as shown in Table 3.3. As may be seen in Appendix A, Table A.10, this parameter was the “mode nest log-sum,” a coefficient applied to the composite utility of nested mode alternatives. Its significance across all three regions is related to the effect it has of rescaling utilities; thus this one parameter is extremely important in this model because it influences its sensitivity to changes in level of service for every mode alternative. The “person exact number of tours” model also stands out in Table 3.4. Despite a total of 101 parameters, the number of statistically significant estimated parameters in the models for Sacramento, Tampa, and Jacksonville were 39, 16, and 11, respectively, as shown in Table 3.2. There were seven param- eters, however, that were statistically significant in all three regional models (see Table 3.3). As shown in Tables A.3–A.5 of Appendix A, six of these parameters were alternative-specific constants referring to the propensity to make multiple tours of common types: two work tours, three-plus work tours, two school tours, two meal tours, two social/recreational tours, and three-plus social/recreational tours. The other common statistically significant parameter was an interaction effect between two-plus escort tours and being an adult female in a household with school-age children. Thus, it would seem that the propensity to make multiple tours in a single day is significant for each region, and the one demographic variable that consistently shows up as significant is the effect of being female in a household with school-age children on the pro- pensity to make multiple escort tours. To summarize, the OE ratio is a measure of the information value of statistically significant estimated parameters that are common across pairs of models and is analogous to the signal-to-noise ratio. This has implications for determining which model components and parameters to focus on when assessing model transferability. A large OE ratio is evidence that those statistically significant parameters which are com- mon to multiple regions are important predictors of behavior and deserve relatively greater scrutiny when assessing regional transferability. A large OE ratio will tend to occur in choice model components that have proportionally fewer statisti- cally significant estimated parameters. A lower OE ratio simply says that there is not as much information value in any individual parameter, typically because it is one of many com- mon statistically significant parameters that appear in both regional models. There also may be cases of OE ratios in which regional models have many statistically significant param- eters individually, but the pair has proportionally few in com- mon, as is the case with the nonmandatory-tour destination choice model.

18 Statistically Significant Differences Statistically significant differences between statistically signifi- cant estimated parameter values are summarized in Table 3.5 for each of the possible pairs of models. This tally counts only estimated parameters that were statistically significant in both models. Statistically significant differences were based on pairwise t-tests (0.05 level of significance), using the standard errors from the regional model with the fewer observations. For most parameters this was the larger standard error and is therefore a more conservative test of significant differences. Thus, the Jacksonville standard error estimates were used in comparisons with the other two regions, and the Tampa standard error estimates were used in comparisons with Sacramento. Summing across all 17 models, the most striking pattern found in Table 3.5 is a much higher proportion of significant differences between Tampa and Sacramento (42%) than between either Jacksonville-Tampa (29%) or Jacksonville- Sacramento (28%), using the total common significant param- eters in Table 3.3 as the normalizing factor. For purposes of considering transferability from Sacramento to the two Florida regions, these results suggest that the Tampa model may be less similar to Sacramento than Jacksonville. To learn where and why these differences occur, it is neces- sary to study individual model components and parameters. While there are many statistically significant differences, this discussion focuses on the parameters in models with high OE ratios for each model pair, and for the trio, as already discussed and shown in Table 3.4. For example, in the “work tour mode choice model,” discussed for the high OE ratio, there is a statistically significant difference between the Jacksonville and Tampa values for the previously mentioned “mode nest log-sum” parameter. The estimated coefficient value for Sacramento actually lies between the estimated values for the two Florida cities, as may be found in Table A.10 of Appendix A. The lower coefficient value in the Tampa estimation suggests greater levels of substitution between alternatives in the same nest (more unobserved attributes in common), whereas the values in the Sacramento and Jacksonville estimates, which are close to 1.0, imply independence. Nests include Auto [single-occupancy vehicle (SOV), high-occupancy vehicle, Table 3.4. Observed-to-Expected Ratios of Statistically Significant Estimated Parameters That Sacramento, Tampa, and Jacksonville DaySim Models Have in Common Tables in Appendix A Name of Choice Model Component Observed-to-Expected Ratios of Statistically Significant Estimated Parameters Common to Pairs and All Three Models JAX and TPA TPA and SAC JAX and SAC All Three A.1–A.2 Household auto ownership 1.69 1.31 1.23 2.42 A.3–A.5 Person exact number of tours 5.16 1.94 1.88 10.40 A.6–A.8 Nonmandatory tour destination 1.12 1.03 1.03 1.15 A.9 Work-based sub-tour generation 2.33 1.56 1.56 3.63 A.10 Work-tour mode 12.00 1.44 1.44 17.28 A.11 School-tour mode 3.04 1.19 1.19 4.06 A.12 Escort-tour mode 1.88 1.34 1.41 2.51 A.13 Other home-based-tour mode 1.63 1.23 1.34 2.25 A.14 Work-based sub-tour mode 3.20 1.45 1.16 4.65 A.15–A.17 Work-tour arrival and departure times 1.76 1.33 1.48 2.60 A.18–A.19 School-tour arrival and departure times 1.69 1.04 1.11 1.31 A.20–A.22 Other home-based-tour arrival and departure times 1.49 1.29 1.27 2.23 A.23–A.24 Work-based sub-tour arrival and departure times 2.69 0 0 0 A.25–A.27 Intermediate-stop generation and purpose 1.37 1.11 1.13 1.62 A.28–A.30 Intermediate-stop destination 1.92 1.39 1.39 2.73 A.31–A.32 Trip-level mode 1.57 1.18 1.20 1.95 A.33–A.34 Intermediate-stop departure time 1.27 1.09 1.12 1.37 OE ratio for all models (same as Table 3.3) 1.84 1.32 1.33 2.59

19 2 occupants (HOV2), high-occupancy vehicle, 3 or more occupants (HOV3+)], Transit (Walk Access, Drive Access), and Non-motorized (Bike, Walk). When calibrating a mode choice model, the usual process is to adjust alternative-specific con- stants. Other coefficients are typically left alone. Without an estimation exercise such as this, this difference in intra-nest substitution would go unnoticed during calibration; however, it could be detected and adjusted in subsequent sensitivity test- ing if it were found that mode shifts due to changes in level of service were not appropriately sensitive. Also, as already discussed, the “person exact number of tours” model was found to have a significant positive propen- sity toward two work tours in Jacksonville relative to Tampa, and a significant positive propensity toward two and three- plus social/recreational tours for both Florida models relative to Sacramento. (See Tables A.3–A.5 of Appendix A for details.) Both of these differences suggest lifestyle differences that are more leisure oriented in the Florida cities, Tampa more so than Jacksonville. Because these parameters are day-pattern component constants (similar to alternative-specific constants), it is important that they be calibrated in a transferred model specification to adjust for such differences between regions. It is also worth noting that there was no significant difference between regions in the impact of being an adult female in a household with young children and the propensity to make multiple escort tours, and it would have been surprising had there been a difference. In models with lower OE ratios, interpreting significant differences in individual parameters may prove difficult; however, it may still be informative to look for trends among groups of similar variables, such as those affecting a particu- larly choice alternative or demographic group. For example, the auto ownership model does not have a particularly high OE ratio, but it has several variables in which there are signifi- cant differences related to the one-auto alternative and house- holds with just one driver. As shown in Tables A.1–A.2 in Appendix A, there are significant positive differences between the Tampa model and the Sacramento model and between the Table 3.5. Statistically Significant Differences in Estimated Parameters Between Sacramento, Tampa, and Jacksonville DaySim Models Tables in Appendix A Name of Choice Model Component Number of Significant Differences in Estimated Parameter Values Using Pairwise t-Tests at 0.05 Significance Level for Parameters Significant in Both Models (0.05 level) JAX versus TPA TPA versus SAC JAX versus SAC A.1–A.2 Household auto ownership 4 8 5 A.3–A.5 Person exact number of tours 2 2 2 A.6–A.8 Nonmandatory tour destination 11 28 12 A.9 Work-based sub-tour generation 0 1 1 A.10 Work-tour mode 1 0 0 A.11 School-tour mode 4 3 1 A.12 Escort-tour mode 1 4 3 A.13 Other home-based tour mode 3 10 9 A.14 Work-based sub-tour mode 0 1 1 A.15–A.17 Work-tour arrival and departure times 7 4 2 A.18–A.19 School-tour arrival and departure times 2 3 3 A.20–A.22 Other home-based-tour arrival and departure times 4 11 4 A.23–A.24 Work-based sub-tour arrival and departure times 0 0 0 A.25–A.27 Intermediate-stop generation and purpose 4 23 16 A.28–A.30 Intermediate-stop destination 7 13 8 A.31–A.32 Trip-level mode 8 3 4 A.33–A.34 Intermediate-stop departure time 18 25 14 Number (%) of parameter pairs significantly different, given both are significant estimated parameters 36 (29%) 81 (42%) 40 (28%)

20 Tampa model and the Jacksonville model with respect to these parameters. These differences are likely due to the larger proportion of retirees, who are more likely to have a single car and live in a single-driver household. This is further sup- ported by a significant positive difference for the interaction between the retiree-household variable and the one-auto alternative. In other models, there may be no obvious, meaningful pattern to the differences in parameter values. In the “non- mandatory tour destination choice model,” for example, there are 28 parameters that are significant and significantly differ- ent between the Tampa and Sacramento models, as shown in Table 3.5. There are less than half that many significant differences between Jacksonville and Sacramento or between Jacksonville and Tampa. Despite the larger numbers of signifi- cant differences, the types of variables in this model are mostly piecewise–linear distance variables and attraction variables (also called “size” terms) related to various types of employ- ment; and most of these interact with various activity/trip purposes. This model has a very low OE ratio for each of the regional pairs, making it fallacious to read too much into any one parameter. Moreover, the compensating effects of so many distance-terms are difficult to interpret. There is one clear group-parameter trend in this model, however. As shown in Table A.7 in Appendix A, the presence or absence of children in the household affects the “escort” trip purpose and many employment-related size terms in the model. The effect is significantly lower in the Tampa model than in the Sacramento model. The effect is not quite so significant when comparing Tampa and Jacksonville, but the direction of the difference is the same. This lends further credence to the notion that the different sociodemographic makeup of the Tampa region is playing a role. In Tampa, escort tours are more likely to involve providing rides to other adults, whereas in Sacramento and Jacksonville escort trips are more likely to involve chauffeuring children, which may lead to different destination patterns and preferences. That there are so many significant differ- ences in other parameters without a clear group pattern sug- gests that this model should be reestimated using local data, if available, rather than simply transferred and recalibrated to match trip lengths. This is particularly important because the nonmandatory-tour destination choice model will have a large impact on regional origin–destination patterns and is singly constrained, unlike work and school destinations which can be doubly constrained through shadow pricing in DaySim. As shown in Table 3.4, the model that has the highest OE ratio (1.56) for comparisons between either Jacksonville or Tampa and Sacramento is the work-based sub-tour generation model. As shown in Table A.9 of Appendix A, the “no more sub-tours if one or more sub-tours already simulated” variable is significantly more positive for the Florida regions than for Sacramento, meaning that cases of two or more work-based sub-tours reported in the same day are rarer in the Florida surveys. This may be an artifact of the NHTS survey design, as reporting of work-based travel seems lower in general, and is also why the other models of work-based sub-tours have few cases for the Florida regions and thus many nonestimable coefficients and few significant differences. In this case, because there is no reason to think that the propensity of workers to make sub-tours would differ significantly between regions and because underreporting of work-based sub-tours in the NHTS survey is suspected, then the analyst should take care when transferring a model not to put too much faith in the NHTS target values for calibration and remain truer to the original Sacramento specification. The remainder of this section comprises observations on the significant parameter differences in other models listed in Table 3.5. School-tour-mode choice (Appendix A, Table A.11). The only significant differences are between Tampa and Jacksonville, in which the Tampa model shows a much higher propensity toward the school bus mode and shared-ride alternatives (HOV2 and HOV3+ in the model), based on differences in alternative-specific constants. These differences in values may reflect the lower auto ownership levels in the Tampa region, or they may reflect differences in school-bus provision and district boundaries. Escort-tour-mode choice (Appendix A, Table A.12). There are significant negative differences between both Florida models and Sacramento in alternative-specific constants related to HOV2 and HOV3+ and the “logsum from the path type choice model,” which is a composite utility of auto and transit skim values. The effect of being over age 50 on choosing to walk was also significantly lower in the Tampa region compared with both Jacksonville and Sacramento, which again may point to lifestyle differences. Other home-based tour-mode choice (Appendix A, Table A.13). Further supporting the regional lifestyle differences, there are significant negative differences between both Tampa or Jacksonville and Sacramento for alternative-specific constants pertaining to HOV3+ and significant positive differences for the effects of one-car and one-person households on choosing shared rides. There are also significant negative differences in interaction effects between meal tours and shared-ride alter- natives as well as significant positive differences in interaction effects between social/recreational tours and bicycling. The “path type model log-sum” variable is also significantly dif- ferent between the two Florida cities and Sacramento. For Tampa-Sacramento only, there is a significant difference in the effect of zero-car households on the walk-transit alternative. Jacksonville and Tampa differ significantly from each other for the effect of the shared-ride path choice variable, which may be attributed to differences in skims.

21 Work-based sub-tour mode choice (Appendix A, Table A.14). There is only one significant difference between Tampa and Sacramento in a shared-ride-2 alternative-specific constant. Tour-arrival and departure-time-choice models (Appendix A, Tables A.15–A.24). These models include constants referring to specific time intervals of an hour or more and shifts in time between these time intervals. In general, these parameters work as a set and should mimic observed diurnal distributions, which should be similar for all regions for the same tour type. They include tour-purpose or tour-context effects on shifting time or duration as well as person attribute effects on shifting time or duration. To respond to changes in congestion, each model includes an auto-tour path type log-sum parameter, which influences tour duration. For nonmandatory-tour time choices, there are also “time window” variables that effectively constrain choices. As shown in Table 3.4, these models have relatively low OE ratios, making it difficult to find meaningful differ- ences. Where there are significant differences between models for these constants, they are difficult to interpret in isolation because their meaning is in relation to each other. For the work-tour departure and arrival time choice model, there were no significant differences for any of the person attribute effects. For the other home-based-tour arrival and departure- time choice model, there were some significant differences between Tampa and Sacramento for shopping tour duration. Because tour departure and arrival times should be fairly similar within regions, the large number of differences is probably due to insufficient observations for certain tour purposes during certain time intervals in both Tampa and Jacksonville. For this reason, it would be better to simply transfer the Sacramento specification with minimal recalibration. During model valida- tion, these models could be adjusted to respond to regionwide differences in time-period-specific traffic counts. Intermediate-stop generation and purpose choice models (Appendix A, Tables A.25–A.27). As shown in Table 3.5, this is a model with relatively low OE ratios and numerous statis- tically significant differences between Tampa and Sacramento (23) and between Jacksonville and Sacramento (16), but few significant differences between the two Florida regions (4). Again, trends in differences between groups of parameters are the most telling. A closer inspection of the tables in Appendix A reveals a clear pattern of negative differences for adding vari- ous types of personal business, shopping, escort, and meal stops on tours of all purposes; a lower effect of tour duration on stop propensity; and a stronger effect of an intermediate stop occur- ring on the first half of a tour. The most likely explanation for these differences is an underreporting of intermediate stops in the NHTS survey data rather than any true behavioral dif- ference. Even the significantly greater propensity to place inter- mediate stops on the first half of a tour before the primary stop, rather than after the primary stop on the second half of the tour, runs counter to what is typically observed in household travel diaries and probably reflects underreporting of stops on the way home. The implication for model transfer is that calibration to the NHTS target values may result in underrepresentation of intermediate-stop making, which will likely show up during model validation as underprediction of travel volumes relative to counts regionwide, particularly in the post-p.m. peak evening period. Since intermediate stops tend to have shorter average trip lengths than home-based stops for the same purpose, average trip lengths will be biased upward. For these reasons, remaining closer to the original source coefficient values during calibration may be preferred for this model. Intermediate-stop destination choice (Appendix A, Tables A.28– A.30). As already mentioned, this model is interesting because it is the one model in which the Jacksonville and Sacramento models had many more statistically significant parameters in common than Tampa and Sacramento (29 versus 19); yet there were significantly fewer significant differences between the Jacksonville and Sacramento models compared with the Tampa and Sacramento models (8 versus 13). This would seem to be further evidence that the two Florida regions may be less similar in terms of travel behavior and that Jacksonville may actually be more similar to Sacramento. The intermediate-stop destination choice model also has a low-to-medium OE ratio (see Table 3.4), which suggests that it will be difficult to separate signal from noise in interpreting the reasons for these differences. The most important may be the difference on a generalized travel time parameter. There are scalar, squared, and cubed variables used in the specification, and the differences on all three parameters would seem to indicate a nonlinear relationship that is similar in both of the Florida models, but significantly different from the Sacramento model. These differences are partially offset by differences in other parameters, such as hourly parking, making inter- pretation difficult. This might reflect a greater level of effort in coding land-use variables in the Sacramento region, or it could actually reflect more mixed-use land uses and paid parking; the reasons are unclear. In addition, the parameter on the log-size function, a weight placed on all of the size-term variables, is significantly lower in both Florida regions compared with Sacramento; and this parameter would offset travel impedance to a large degree. There are also significant differences in parameters related to shopping and meal destinations, but with offsetting differences in signs that challenge interpretation. In summary, it would seem that, like nonmandatory-tour destination choice, the intermediate-stop destination choice is a model that would be better reestimated on local data rather than transferred and calibrated to match trip lengths, particularly since this model will be singly constrained. When there are many significant differences in parameters that have offsetting effects, it is not clear what those differences portend for forecasting.

22 Trip-level mode choice (Appendix A, Tables A.31–A.32). Another model with a low OE ratio, the trip-level mode choice model does show some clear trends among groups of param- eters, which makes explaining significant differences easier. There are more significant differences between the Tampa and Jacksonville models than between either of the Florida models and Sacramento. In particular, most of the significant differences are related to the HOV3+ alternative. Shared rides involving more than two persons occur much less often in the Tampa data set and somewhat less in Jacksonville, relative to Sacramento. This is consistent with the regional demo- graphic differences in which there are more retirees and fewer households with children in the Tampa region, compared with Jacksonville and Sacramento. In addition, the positive coefficient placed on the composite impedance term called “path type model log-sum” is significantly lower in the two Florida models, compared with Sacramento, meaning that it has less influence on mode choice, perhaps because of the relative strengths of the mode-specific bias constants. Intermediate-stop departure time (Appendix A, Tables A.33– A.34). This model actually models the arrival time of stops on the outbound half of a tour and the departure times for stops on the inbound half of a tour. It is composed of numerous alternative-specific constants representing specific hours of the day as well as duration shift variables that are interacted with either person attributes or tour-purpose attributes. It also includes two composite impedance terms, one for auto path type log-sums and another for transit path type log-sums. It has the second lowest set of OE ratios shown in Table 3.4. It also has the highest number of significantly different parameters between Tampa and Jacksonville (18) and the second highest number of significantly different parameters between Tampa and Sacramento (25), as shown in Table 3.5. There are many parameters that work together and are offsetting, making this a difficult model to interpret in terms of transferability. There is a clear grouping of duration constants that are all negative in the Sacramento model, significantly more negative in the Tampa model; and these same constants are all positively valued and significantly different in the Jacksonville model. Similarly, there is an offsetting group of duration shift constants interacted with tour purposes that are all negative in the Tampa model, all posi- tive in the Jacksonville model, and mixed in the Sacramento model. As with other models, the low signal-to-noise (OE) ratio of this model with so many significant differences makes it difficult to assess transferability other than to say that the differences are not necessarily due to differences in underlying behavior but more likely insufficient observations for certain activity types during certain time periods. Unlike with destina- tion choice, which is more sensitive to regional idiosyncrasies in forecasting trip tables, arrival and departure time patterns should be fairly stable from one region to the next. Similar to the tour-level timing choice models, it should be possible to borrow a model specification from Sacramento, which was estimated from a larger sample size, and obtain more realistic outcomes, even without recalibration. As with the tour-based time-of-day choice models, these models could be adjusted to respond to regionwide differences in time-period-specific traffic counts during model validation. Key Findings from Estimation Tests The relatively low sample sizes for the Tampa and Jacksonville NHTS data made it difficult to make conclusive statements about the transferability of the Sacramento model to Tampa and Jacksonville; however, there was a consistent set of sig- nificant differences noted between the Tampa model and the Sacramento model which suggested that regional lifestyle differences may have a noticeable effect on the transfer and would need to be accounted for. Moreover, the analysis pro- duced several useful observations that could guide future activity-based model system transfers: • Decision contexts that are observed once per household or per person day (or less often) will have fewer observations and thus be more difficult to estimate with statistically significant parameters. • Long-term household decisions, such as auto owner- ship, and daily-pattern choice models will rely heavily on alternative-specific constants or other choice dimension constants (e.g., number of tours by purpose), interacted with various household or person attributes. These models will have proportionally fewer statistically significant param- eters compared with models that make more extensive use of transportation level-of-service and land-use variables, which will vary over alternatives and can be estimated using generic coefficients. Tour- and trip-level destination and mode choice models tend to fall into this latter category. • Comparing the occurrence of statistically significant param- eters in 17 pairs of regional models, the regions studied here tend to have many more statistically significant parameters in common than would be expected if such commonalities were random occurrences. • Relatively high observed-to-expected ratios (OE ratios) of common significant parameters in pairs of models are indicators of the information value provided by specific significant parameters. For purposes of assessing meaningful differences, low OE ratios may indicate excessive noise when doing a paired comparison, although meaningful informa- tion may still be gained by studying groups of parameters with similar attributes that consistently show the same direction in differences. • Comparing OE ratios, the Tampa-Jacksonville pair has a significantly higher proportion of common statistically significant parameters than either Florida region when

23 paired with Sacramento. The most likely reason for this is the common NHTS surveys used in the Tampa and Jacksonville estimations. • Using pairwise t-tests, there were substantially more statisti- cally significant differences between Tampa and Sacramento (42%) than between Tampa and Jacksonville (29%) or between Jacksonville and Sacramento (28%), aggregated over all 17 models, normalized by the total parameters that were statistically significant in both regions. • The nesting log-sum parameter in the work-tour-mode choice model was significantly different among the three regions. This parameter is important because it affects sen- sitivity to changes in level of service for all modes; therefore, in a model transfer it could be adjusted during sensitivity testing. • Looking across the various model components, there are sev- eral statistically significant differences that suggest the influ- ence of regional differences in lifestyle that set Tampa apart from Sacramento and, to a lesser extent, from Jacksonville: 44 Higher propensity in Tampa to make social/recreational tours and a lower propensity for work tours; 44 Stronger influence in Tampa on auto ownership of the single-driver and retiree-household variables, and a higher market share of single-auto households; 44 A significant dampening of the effect of the presence of children in households on escort-tour destination choices; and 44 A lower share of shared rides with more than two persons and stronger influence of zero-car households on mode choice in the Tampa model, compared with Sacramento and Jacksonville. • Destination choice models, which lack alternative-specific constants and are only singly constrained for non-work/ school purposes, will be more difficult to calibrate to match regional origin–destination patterns and therefore should be reestimated if possible. This study found many statistically significant differences between regions for these models. If reestimation is not possible, then these models should receive greater attention during model calibration and sen- sitivity testing. • The NHTS sample provided an insufficient number of observations to represent multidimensional choices with many alternatives, such as with departure- and arrival-time choice models. Since diurnal patterns are relatively stable between regions, it would be better to transfer such a model from a region that has enough observations, such as Sacramento, and perform minimal calibration, possibly to adjust to regionwide traffic demand by time period during model validation. A more scientifically credible scenario would have been to estimate a simpler activity-based or tour-based model using the Sacramento NHTS data and then use that to study behav- ioral transferability. In this way differences between survey methods would not be an issue and the likely resulting speci- fication would reflect similar levels of data availability. FHWA has recently funded an extension of the STEP project, which will provide a better basis for comparison. A summary of find- ings relevant to this project from already completed STEP work may be found in Appendix C of this report. The STEP project extension will study seven regions, including Sacramento, Tampa, and Jacksonville, all using DaySim and NHTS survey data. The project will use more aggregate land-use units, most likely TAZs, and much simpler model specifications that can be estimated using sample sizes comparable to that of Jacksonville. With this more level basis, it should be possible to make more scientifically valid comparisons of activity-travel behavior in different regions. However, estimating a simplified version of the Sacramento model was not part of the scope of the C10A extension which required the study team to deliver production-version models to both Jacksonville and Tampa. To meet the C10A extension deliverable, the study team viewed the transferred version of the Sacramento model as being a more behaviorally realistic and useful starting point than what could have been obtained from the Florida NHTS data, and chose to calibrate it from there. The study team concluded that it would not be beneficial to estimate and apply models based on the pooled NHTS survey data from Jacksonville and Tampa, which had been one of this project’s original objectives. The reasons for this decision are as follows: • As shown in Table 3.1, even the pooled weekday NHTS data from Jacksonville and Tampa were much smaller than the sample size necessary for reliable estimation of models such as those in the DaySim model system. Even the original Sacramento sample of nearly 4,000 households was smaller than ideal, with a sample size of 5,000 or more being more typical for surveys in major regions in the United States. • Compounding the small sample size issue is the fact that the data lack certain types of households and types of choice behavior. There are few, if any, bicycle or transit trips reported in the NHTS data for most tour purposes. There are few households in the data with very low incomes or households who do not own vehicles. There are also very few university students or other young, single households. While these types of survey nonresponse problems may be somewhat typical, especially in regions where actual nonauto trips are scarce, it nevertheless leaves important gaps in the ability to estimate key model parameters for the data. • There are a few additional drawbacks of using the NHTS data, including some missing data items (e.g., usual school location, transit pass ownership), incomplete households, and a scarcity of observed school tours and work-based

24 sub-tours. These types of issues tend to exacerbate the problem of limited sample sizes. • Although there seem to be similarities in the coefficients estimated separately on the Tampa and Jacksonville region data, there are also some large differences; and the limited sample sizes make it very difficult to estimate separate but significant parameters in those cases. The study team’s recommendation for the regional agencies in Tampa and Jacksonville was to use the NHTS data only for calibration of some key constants and parameters for the Sacramento models applied to the Florida regions and to post- pone complete reestimation of the models until new survey data are available. Ideally, that survey will provide sample sizes for weekday travel at least twice as large as the NHTS add-on sample and will use targeted oversampling more aggressively and more successfully to ensure adequate representation of “rare” households and persons and behaviors, such as the following: • Non-car-owning households; • Very-low-income households; • University students and other young, single-person households; • Young children under age 5 who typically require parental supervision, thus creating and constraining activities and travel of adults; • Transit users (including park-and-ride); and • People who commute by bike and by foot. Calibration of Regional Models to Test Transferability This section evaluates the performance of the Tampa and Jacksonville models from a transferability standpoint by com- paring the calibrated coefficients of these two models with the Sacramento model. First, it should be noted that the Sacramento model parameters are those that were estimated from sample data and do not include calibrated coefficient values. Starting with estimated constants from another region, rather than some combination of estimated and calibrated values, avoids the complexity that could be introduced by another region’s calibration process. The other region’s cali- bration process may be subject to multiple benchmark data sources and possibly the kind of ad hoc adjustments that are sometimes done to match traffic count targets. The measure used here for comparison of model coefficients is the Absolute Percentage Logit Difference (APLD), defined as follows: APLD 1Tampa SACOGe= −( )β −β This is an equation in which zero difference in parameters results in a statistic of zero, and larger differences result in larger positive values. APLD is preferred over absolute per- centage arithmetic difference because a positive difference in parameters will have a larger effect on probability calculations than a negative difference due to the exponentiation of utili- ties in the logit models. Using the absolute value also avoids the more complex discussion of sign changes and their net effect. Rather, APLD focuses on “how different” the sets of coefficients may be, making it more of a distance metric (“how far” the calibrated model is from the original model). An overall mean APLD, which is the arithmetic mean of the APLDs across all calibrated variables, was computed for each model. To better understand the calibration effort required to transfer models across regions, one should consider the overall APLD as well as the number of parameters that were calibrated. In the subsequent sections, summaries of such comparisons are made for the Tampa (Table 3.6) and Jacksonville (Table 3.7) models, respectively. Appendix B presents detailed tables com- paring the model calibration results for Tampa and Jacksonville with the Sacramento models for each calibrated model param- eter. In each of these tables, there are columns which indicate whether the calculated APLD was in one of three ranges: • Low: APLD ≤ 20%, • Medium: 20% < APLD ≤ 50%, or • High: 50% < APLD. Tampa Model Calibration The Tampa Bay regional model calibration statistics for each of the DaySim model components are shown in Table 3.6. The first model listed—the usual work location choice model—required a modest level of calibration, as evidenced by the small number of coefficients (5 out of 73) required for calibration and the overall APLD of the calibrated coefficients falling in the medium range (34%). Because the dependent variable in this model is zones, there are no alternative-specific constants, and the calibration parameters represent various impacts on travel distance. (See Appendix B, Table B.1, for details on individual parameters.) In contrast, the auto ownership model required relatively more calibration effort, with nearly half of the estimated parameters used in calibration and an overall APLD of 84%, which is on the high side. As shown in the Appendix B, Table B.2, the coefficients that were calibrated may be grouped into two types of variables: number of drivers per household and household income. The calibrated Tampa model coefficients, for the most part, are noticeably different from the Sacramento model coefficients, reflecting differences in the socioeconomic makeup of the two regions. For example, 58% of households in the Sacramento region have two or more vehicles, and 25%

25 of households fall into the $75,000+ income category. The corresponding statistics for the Tampa region are 48% and 17%, respectively (U.S. Census Bureau 2010). Because auto ownership choice has a large effect on downstream models, the extra effort expended to fit benchmark values may be well justified. For similar reasons, more calibration effort was put into the individual day-pattern model, the work-based sub-tour generation model, and the exact number of person tours model, which together create the daily activity structures and generate daily tours within DaySim. The person-day pattern and exact number of tours models have large numbers of estimated parameters: 350 and 295, respectively. As a conse- quence, they also have large numbers of calibration coefficients: 57 and 47, respectively, most of which are interaction terms between activity pattern dimensions and person and house- hold attributes. As already described earlier in this chapter, it was not possible to estimate the person-day pattern model using the NHTS data. In addition, the person exact number of tours model was estimated but had low statistical signifi- cance for the vast majority of its parameters. Given these outcomes, it should not be surprising that considerably more effort would be needed to align model forecasts with bench- mark values. In the day-pattern choice model, calibrated variables include tour/stop purpose-specific constant, employment and student status, and age. (See Appendix B, Table B.3 and Table B.4.) The APLD between Sacramento and Tampa model coefficients are low to medium for work, school, and escort purposes; for other tour purposes (e.g., personal business, shopping, meal, and social/recreation) these values differ by more than 50%. One of the factors that contributed toward this discrepancy is that variables such as interactions between tour purpose and nonworking adults/retired adults were not included in the Sacramento model specification. Interactions between tour purpose and certain person attributes, namely nonworking adult status for certain day patterns, were not included in the original Sacramento model specification either because they were not statistically significant or because they were over- looked. Regardless of the reason for that omission, when trans- ferring a model to a new region, it is important to consider whether there are any important explanatory variables that may have been excluded from the original specification. In this particular instance, it was found that the retiree market Table 3.6. Tampa Model Calibration Effort Relative to Sacramento Starting Values Tables in Appendix B Name of Choice Model Calibrated/ Total Parameters Calibrated Overall APLD (%) Number of Parameters by Absolute Percentage Logit Difference (APLD) Low (APLD <– 20%) Medium (20% < APLD <– 50%) High (APLD > 50%) B.1 Usual work location 5/73 33.9 1 3 1 B.2 Auto ownership 28/60 83.5 4 4 20 B.3–B.4 Individual person-day pattern 57/350 75.1 15 13 29 B.5 Work-based sub-tour generation 7/15 33.3 3 2 2 B.6–B.7 Exact number of person tours 47/295 50.3 21 4 22 B.8 Intermediate-stop generation 30/106 64.3 8 8 14 B.9 Nonmandatory-tour destination 19/101 515.6 2 5 12 B.10 Tour mode: work 6/36 2.8 6 0 0 B.10 Tour mode: school 6/42 154.8 0 1 5 B.10 Tour mode: escort 3/15 86.8 0 0 3 B.10 Tour mode: hb-other 5/45 77.8 0 1 4 B.10 Tour mode: wb-sub-tour 5/17 57.7 0 3 2 B.11 Tour time of day: work 10/74 38.6 4 1 5 B.11 Tour time of day: school 10/68 92.4 4 1 5 B.12 Tour time of day: hb-other 10/95 10.7 8 2 0 B.12 Tour time of day: wb-sub-tour 10/50 57.1 4 0 6 B.13 Trip time of day 10/55 5.1 9 1 0 Note: hb = home-based; wb = work-based.

26 segment, which is more important in the Tampa region, was not well-represented in the original specification; therefore, parameters were added and calibrated to provide this addi- tional variation across household and person types. Table B.5 in Appendix B presents calibration results for the work-based sub-tour generation model. Also in Appendix B, Table B.6 and Table B.7 show similar results for the exact num- ber of tours by purpose. A comparison between Sacramento model coefficients and calibrated Tampa model coefficients indicates that work-based sub-tour models for these two regions are reasonably similar (i.e., the overall mean APLD is 33.3%); however, this is not the case for all tour generation models. For example, the Sacramento and Tampa model coefficients cor- responding to variables that include interaction with work, shopping, and meal tour purposes seem to be more similar than other coefficients in the exact number of person tours model, perhaps because these purposes are more commonly observed in both surveys. In the intermediate-stop generation model, the interaction terms for work, school, personal business, and escort purposes were more similar in both regions. As shown in Table 3.6, non-work destination, tour-mode choice, and time-of-day choice models require calibrating a relatively smaller set of variables. In particular, tour-mode choice and time-of-day choice models require only alternative- specific constants to be adjusted. Most of the models had low to medium APLDs, with the exception of the school and escort purposes, which had high APLDs. This is likely due to an underrepresentation of university students and children in the NHTS data that was used to calibrate the Tampa model. When calibrating the school-tour generation models for university and K–12 students, tour generation rates were fac- tored up by 2.16 and 1.15, respectively, over the original NHTS data to compensate for this underrepresentation; however, the lack of observations on the actual school-tour destinations (trip lengths) and mode shares is likely to be less accurate for these tour types. Finally, Table 3.6 shows results for just one trip model (time-of-day choice), which required calibration. Within the trip time-of-day choice model, only a small number of coeffi- cients required some minor adjustment (overall mean APLD is 5.1%). In the Tampa model, the trip-mode choice model required no real adjustment, as mode share targets were met solely through calibration of the upper-level tour models. This relatively low level of calibration at the trip level results from the combined effect of two factors: (1) the models were calibrated one at a time in a top-down manner. That is, the upper-level models—such as usual work location and auto ownership, day-pattern, and tour models—were calibrated before the lower-level trip models; and (2) in the DaySim activity-based model, the upper-level models are connected to the lower-level, trip-based models through log-sum variables, which represent the composite utility of lower-level choices. The order in which the models are calibrated matters. When calibrating hierarchical activity-based model systems such as this, the usual practice is to begin at the top of the hierarchy and to calibrate downward. This means calibrating long-term choice models, such as usual work location and auto owner- ship, first, followed by day-pattern models, tour-level models, and finally trip-level models. The upper-level models condition the lower-level model choices. Because of this conditioning effect, more calibration effort spent on the upper-level models indirectly calibrates the lower-level models to some extent. The most obvious example is that calibration of tour-mode shares does a good portion of the work needed to calibrate trip- mode shares. Some of the regional differences are due to these upper-level models. For example, as already mentioned, the addition of day-pattern bias constants for retirees in the Tampa model resulted in different preferences for tour purposes being expressed. In turn, that conditioned the lower-level destina- tion, time-of-day, and mode choices, thus less calibration was needed at these lower levels. Jacksonville Model Calibration The model calibration efforts for Jacksonville are summarized in Table 3.7. When compared with the Sacramento model coefficients, similar patterns emerge. Considering that the Sacramento coefficients were estimated, not calibrated, it is not clear to what extent the differences reflect socioeconomic characteristics as opposed to differences in the survey instru- ments, which were the same for the two Florida models. The usual work location and auto ownership model calibra- tion efforts for Jacksonville were very similar to Tampa. The overall greater difference in APLD (182%) for Jacksonville simply indicates that the calibration coefficients were adjusted further away from the Sacramento starting points than Tampa (84%), but it does not indicate in which direction; indeed, a mix of negative and positive adjustments were made across 28 parameters. The individual person-day pattern model for Jacksonville, presented in Appendix B, Table B.16 and Table B.17, indicates that the differences between the Sacramento and Jacksonville model coefficients are relatively small for variables correspond- ing to work, school, and personal business tour purposes, and relatively large for variables corresponding to shopping and meal tour purposes. A closer inspection of the calibrated results shows that person-level socioeconomic attributes such as student status, employment status, and age are the key con- tributing factors to these differences. Fewer parameters were adjusted in the Jacksonville model, compared with Tampa (48 versus 57), but they were adjusted by a greater distance (84% versus 75% APLD). Similarly, the exact number of per- son tour model for Jacksonville used many fewer calibration parameters than the Tampa model (8 versus 47), but these

27 eight parameters were on average adjusted by a greater distance (325% versus 50% APLD). The intermediate-stop generation model also had fewer parameters (8 versus 30) but with nearly identical APLD (64% versus 63%). The differences between the Tampa and Jacksonville calibra- tion efforts, in part, reflect differences in how two different analysts approached the calibration exercise: one choosing to adjust more parameters (Tampa) and the other choosing to adjust fewer but by larger amounts (Jacksonville). Although different, both approaches matched desired target levels to a similar level of precision. While some of these differences may be attributed to differences in analysts’ judgment, it should be recognized that the NHTS sample data for Tampa was con- siderably larger than that of Jacksonville, which allowed more confident identification of and calibration to subsegments of the population. As may be seen in Table 3.7, overall, calibration of the models pertaining to tour destination, mode, and time of day required more calibration parameters in the Jacksonville model than in the Tampa model (Table 3.6). With the exception of the nonmandatory-tour destination choice model, these models also diverge more from the Sacramento coefficients (larger cali- brated APLD). One reason for this divergence could be that several interaction terms (e.g., interaction between tour pur- pose and person-level information or land-use characteristics) were needed to calibrate the Jacksonville tour-level models that were not found to be statistically significant in the Sacramento model. These terms were left out of the Sacramento model but introduced in the Jacksonville model. This was also observed in the case of the Tampa model for certain demographic seg- ments, highlighting the fact that while there are some com- mon factors that drive the demand for travel in Sacramento, Tampa, and Jacksonville, there are also a number of region- specific factors. Another consideration is that because the upper-level day- pattern models (individual person-day pattern, exact number of person tours, and intermediate-stop generation models) were calibrated using fewer support parameters in Jacksonville than in Tampa, this created more work for the analyst to do at the tour level to compensate. It is not clear, however, whether the smaller sample size of the Jacksonville sample made this inevitable. Table 3.7. Jacksonville Model Calibration Effort Relative to Sacramento Starting Values Tables in Appendix B Name of Choice Model Calibrated/ Total Parameters Calibrated Overall APLD (%) Number of Parameters by Absolute Percentage Logit Difference (APLD) Low (APLD <– 20%) Medium (20% < APLD <– 50%) High (APLD > 50%) B.14 Usual work location 9/73 36.7 4 2 2 B.15 Auto ownership 28/60 182.3 5 8 15 B.16–B.17 Individual person-day pattern 48/350 85.4 14 13 21 B.18 Work-based sub-tour generation 8/15 71.7 1 5 2 B.19 Exact number of person tours 8/295 324.5 0 1 7 B.20 Intermediate-stop generation 8/106 62.8 0 0 8 B.21 Nonmandatory-tour destination 24/101 110.7 1 5 18 B.22 Tour mode: work 8/36 43.4 3 1 4 B.22 Tour mode: school 7/42 75 2 1 4 B.22 Tour mode: escort 3/15 106.3 0 0 3 B.23 Tour mode: hb-other 18/45 653.0 2 5 11 B.23 Tour mode: wb-sub-tour 6/17 27.8 4 1 1 B.24 Tour time of day: work 16/74 42.1 2 10 4 B.25 Tour time of day: school 11/68 70.3 2 3 6 B.26 Tour time of day: hb-other 5/95 15.3 3 2 0 B.26 Tour time of day: wb-sub-tour 10/50 95.5 0 1 9 B.27 Trip mode 1/65 633.7 0 0 1 B.28 Trip time of day 17/55 77.0 4 5 8 Note: hb = home-based; wb = work-based.

28 Finally, Table 3.7 provides calibration results for the Jacksonville trip models. The calibration efforts were confined primarily to alternative-specific constants included in the trip time-of-day model, with only a single parameter calibrated in the trip-mode choice model (transit-walk access). As discussed previously, the DaySim model structure and the top-down calibration approach adopted here are the main reasons trip models required relatively low calibration efforts. Key Findings from Calibration Tests The calibration tests provide a means to consider which aspects of the Tampa and Jacksonville models needed to be adjusted to better match the NHTS observed data for each region. The tests determined • Work location models are similar for both Tampa and Jacksonville. • Auto ownership models for Tampa and Jacksonville are less similar to the Sacramento model due to regional differences in socioeconomic characteristics. For example, there are more households in Sacramento with higher vehicle owner- ship and higher income than in Tampa, leading to differ- ences in the auto ownership model coefficients for these two study areas. • Work and school-tour pattern models are similar to Sacramento models in both Tampa and Jacksonville, but shopping and meal tour patterns are quite different from Sacramento in both regions. A closer inspection of the calibrated results shows that person-level socioeconomic attributes such as student status, employment status, and age are the key contributing factors for these differences. One significant difference in Tampa is the effect of retired persons; since this effect was not included in the original Sacramento models, it is not adequately represented in the Tampa model. • Work-based sub-tour models for Tampa were reasonably similar to Sacramento, but were not similar for Jacksonville. Differences may be due to smaller sample sizes in Jackson- ville rather than differences in attributes or behavior. • The calibrated exact number of person-tour models, intermediate-stop generation models, and non-work-tour destination models appear to diverge more than the tour- mode and time-of-day choice models from their corre- sponding Sacramento models. While there are common factors that drive the demand for travel in the Sacramento, Tampa, and Jacksonville regions, there are also a number of region-specific factors. • The trip models required minimal calibration. The DaySim model structure and the top-down calibration approach adopted here are the main reasons trip models required relatively low calibration effort. • While the average distances between model parameters in the Sacramento model compared with the Tampa and Jacksonville models might suggest that the Tampa model is more similar to the original Sacramento source, and thus a better transfer, this may be a specious conclusion. The comparatively fewer household observations and reduced variation in the Jacksonville household survey sample undoubtedly played a role in making that model more difficult to calibrate. If models that started from a simpler base specification were used or if a larger household sample were available for Jacksonville, different conclusions could likely be drawn. As a side note, when calibrating models using only the NHTS, CTPP, and ACS-derived target values, the modeling team avoided alternative-specific constants with respect to specific geographies. When the projects reached the validation stage, however, it was necessary to account for bridge-crossing bias constants in destination choice models to correct for traffic- flow differentials crossing the St. John’s River in the Jacksonville region and Tampa Bay in the Tampa region. These types of bias constants are commonly included in both trip-based and tour-based models and to represent psychological barriers related to traveling beyond a certain physical threshold such as a large body of water. Network-based generalized costs simply do not capture these biases. These calibration tests were able to identify changes in individual coefficient values and alternative-specific constants that explain “how different” from the original Sacramento model specification were the calibrated Tampa and Jacksonville models. The calibration of the individual models met expec- tations for planning purposes. Interestingly, in estimation it is the trip- and tour-level models of destination and mode choice that are expected to be the most disparate between regions because of differences in transportation networks and urban spatial structure. Due to the strategy of calibrating models from the top down in the activity-based modeling hierarchy, however, the amount of calibration that had to be performed on the lowest-level models (trip and tour-mode choices) was significantly less than the amount of calibration effort per- formed on the upper-level long-term choice models. This is advantageous because the long-term choice models for work location choice and auto-ownership choice were calibrated more reliably to multiple data sources such as ACS and CTPP.