Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
2-1 CHAPTER 2. BUILD REQUIREMENTS In this section, requirements of the six Builds are discussed in detail. These requirements are based on the principles discussed in Section 1. 2.1. Project Objectives and Motivation The primary objective for each successive Build is to advance the ability to conduct multimodal environmental analyses, as compared with previous Builds. Builds 1 and 2 will concentrate on combining outputs and preparation of screening models to help in advanced transportation planning. Subsequent Builds will focus on harmonizing inputs and computational algorithms. The last two Builds will first focus on implementing needed changes into the AEDT structure and then transitioning the common databases and algorithms to a simulation-based architecture. The motivation for this work includes the need for increased flexibility of environmental analysis, including the ability to assess interdependencies between both environmental parameters, as well as the various transportation modes. An expected outcome of this effort will be better coordination among federal agencies. Initial market research of stakeholder requirements was conducted at the start of this ACRP effort. It is recognized that the needs of the stakeholder community will change over time and that it is important to reestablish the baseline for each phase of the Build development. The market research effort may vary by Build but should be part of the formal task of establishing requirements for each Build. Critical to the establishing the Build requirements are the need to include federal requirements and concerns and reassessing technological advances. This is a complicated topic since differing metrics, modeling approaches and assumptions must be considered. Continued coordination with the stakeholder agencies is essential. Chapter 5 discusses further stakeholder involvement. 2.2. Assumptions and Approximations As discussed in Appendix C, gaps in the current algorithmic knowledge base have been identified. As stated previously, the scope of the MDP does not include considerations for advancing the environmental sciences. Consequently, each Build will likely require a number of assumptions and approximations, depending upon the degree to which the science has been advanced at the time the Build is initiated. During planning of each Build, these gaps must be reassessed and an in-depth discussion of assumptions and approximations will be needed. The basis for any assumption must be explained in the Build documentation. New assumptions must be explained in terms of proven theory. Assumptions previously used and carried over from a prior Build must be documented. The rationale for approximations must be explained. It is recognized that some approximations are because of knowledge gaps while others are driven by requirements such as run-time. Each Build will have risks during development and upon implementation. These risks vary for each Build but will become more substantial in later Builds due to the increased complexity. It is important that these risks be minimized by first understanding each and then by planning to minimize each to the degree practical. Because the Builds will be sequential and much work will be spent on each build identifying assumptions, approximations, and gaps in the knowledge comprehensive supporting documentation will be needed, including an annotated bibliography.
2-2 2.3. Details of Each Build As previously stated, each Build will be a usable tool, with an accompanying technical manual and user guide. Administrative requirements will include quarterly reporting, draft final and final reports. Documentation of testing, evaluation, and acceptable performance must also be included with each Build. Environmental impact assessment and cost-benefit economic analysis will be included beginning with Builds 3 and 4. Six Builds are recommended in the MDP, where each Build is a stepping stone for the next. Figure 2-1 illustrates the overall Builds while detail is provided in the following sections on scope, estimated costs and timelines. Figure 2-1. Simple flow chart of Build phases.
2-3 Build 1. Output post processor. The first build will focus on preparing a post processor that is compatible with existing model outputs. Required or preferred models would continue to be used, but this build would allow for integrated analysis of the various toolâs output. The post processor would create an echo file of the output parameters from each model that is included; combine the outputs of each tool into a single set of reports and graphics, thus allowing integrated impact assessment. Build 1 will be the first step in having a greater degree of uniformity in multimodal environmental analysis. Market research, as described in Chapter 5, should be an initial required task. While a beta version of AEDT (a replacement for INM, NIRS and EDMS) is currently available, the first full public release of AEDT is scheduled for 2011 (AEDT2a) with the next release scheduled for 2013 (AEDT2b). In 2013, INM and EDMS will be withdrawn as required regulatory models. Since it is a postprocessor for existing tools, initiation of Build 1 could be coordinated with the release of AEDT2a, to allow compatibility with the AEDT structure. Coordination of the AEDT development and the Build stages is crucial to maximizing the use of resources. Build 3 should not be started until post AEDT2b. However, beginning with Build 1 and effort should be made to begin using the AEDT protocol. The postprocessor for Build 1 should be based on the output content of the current required models. The postprocessor will automate the needed work to combine the output results from the models for all modes. This will provide a uniform combination of data and greatly reduce analysis time. The postprocessor should include as a minimum the output from: AEDT, INM, HNM, AERMOD for air, FTAâs spreadsheet Guidance on Assessing Noise and Vibration Impacts for rail Combination of the outputs will require a rule base development. For example, the differences between policy-driven noise descriptors across the agencies (e.g., FAA with L , TNM, MOVES, and CAL3QHC for motor vehicles; and, input for user specified values for sources that may not have a required or preferred model. Additionally, models in common use by military operations should be addressed, e.g., NOISEMAP. This may also include military non-transportation sources (e.g., BNOISE) if it is determined that a significant effect could change the overall results. Note that some of these legacy models have been included in AEDT such as INM, HNM, and AERMOD. In addition, construction noise models and special source models regularly used for transportation projects should also be included. These would include HICNM for ground and horn noise models for rail. If Build 1 is initiated prior to the release of AEDT Version 2a, INM, EDMS, SAGE and MAGENTA will also need to be included. dn and FHWA with peak- hour Leq Multimodal environmental analysis tools should directly make use of GIS capabilities. GIS is integral to AEDT and TNM Version 3 and should be fully developed in the near future. While GIS architecture will be considered in general, resources are not planned for GIS implementation of the other modes until Build 4. This is because of the resource intensive nature of the work and problems that could occur. For example, one potential challenge in combining the AEDT and TNM structure is that two tools currently use different GIS engines (ESRI for AEDT and Manifold for TNM). There are good technical reasons for this, but further investigation will be needed. ) must be considered and overcome so that the outputs comply with the stated policy of the lead agency. This will require substantial liaison activities with the stakeholder agencies. It is envisioned that this rule base will be expanded in subsequent Builds as well. The rule base must allow processing from all modes of transportation. Combining output into common metrics may require some simplifying assumptions that could lead to uncertainty specification around modeled results. The use of standard combined reporting mechanisms such as extensible markup language (XML) base, using innovative graphics, color, and output tables will be essential to the success of this build. Verification and validation (V&V) are discussed in Section 2.4 (Software Testing), but it should be pointed out that the primary goal of V&V for the first Build would be to ensure descriptors are combined correctly, that the derived combination rule base is performing adequately, and testing against a real world test case with representative data. For Build 1, most of the required testing could be accomplished using a simple spreadsheet template.
2-4 The other vital testing is to demonstrate the capabilities of the tool using actual case studies, which applied existing single mode models. This testing would help address ease of use of tool versus multiple runs of single mode models and the value of assessing the combined effects from all modes. The risks for this Build are considered low since no changes will be made to the underlying models. They will be connected using a post processing capability. Possible pitfalls include misunderstanding of model output and problems establishing the rule base for combination of noise descriptors. This first Build is estimated to take 2 years with funding of $600,000 â $800,000. Build 2. D evelopment o f screening tools. Build 2 will be the development of environmental screening tools to permit advanced planning to occur quickly. Screening tools are very quick, conservative estimates to determine if more detailed modeling is required. It is desirable to minimize the total number of screening tools but it is envisioned that at least four will be needed (air, rail, highway, ports). The advantage of focusing on screening tools in Build 2 is that it will further advance stakeholder understanding of the pros and cons of having a multimodal analysis capability. Important to this overall understanding of multimodal analysis is a reestablishment of the technological advancements of each mode and desires of the stakeholders during development. This Build is important in the total sequence since it will begin the development of mitigation analysis for multimodal projects that will be based on these screening models. While the screening tools will work with the postprocessor developed in Build 1, they are envisioned to be related to a single mode at this time. Note that in Build 3 the screening tools will be combined to permit a single mode to be reviewed in What-if and mitigation analysis. This will allow changes by mode and the change to the total environmental effect. Outputs should include a base case analysis and then allow for various scenarios deviating from the base case. Ease of use is a primary requirement. This Build will also help the various agency stakeholders to begin to plan for possible future policy changes. Existing screening tools, such as the TNM lookup tables, the noise Area Equivalent Method (AEM) for airports, the FTA screening method described in Guidance on Assessing Noise and Vibration Impacts, CO concentration screening tools commonly used for highways, and any screening tools released in the interim time period before Build 2 begins must be evaluated to determine if they may be utilized in this Build. Where screening tools are non-existent or if existing screening tools are not found to supply needed detail, they must be developed in this Build. Continued development of the rule base to combine output from the various in-use models that was began in Build 1 should continue. It is important that this advancement occur in such a way to provide continuity in this Build and subsequent Builds. The screening tools should allow a direct feed into the output processor so that the results of different What-if and mitigation analyses can be easily recognized and quickly determined. Required testing will include verification of the outputs, testing of the rule base, sensitivity analysis for inputs, and demonstration of the capabilities against a real study. Risk for this Build is considered low to moderate due to the conservative nature inherent in screening models. Possible problem areas will be reduced by assuming methods that may over-predict so that any analysis will not under estimate probable impacts. Possible problem areas are federal agency acceptance of the screening model methodology and implementation of screening tools in a single package. For example, if a call statement were to be used for external programs such as the FTA spreadsheet approach, there must be a seamless interface sending needed information to the spreadsheet and bringing output back into the overall process. Build 2 is estimated to take 1 year with a budget of $250,000 - $500,000.
2-5 Build 3. 1 st generation AEDT-based tool. The development of AEDT is currently in the later stages with the first full public release planned for 2011. Interim versions have been developed and achieved limited distribution outside the development team. At the time of this writing, the beta version has been released so the time schedule is expected to be met. Although AEDT is aviation centric, it has been designed to conduct limited multimodal environmental analysis around airports. Consequently, it is an excellent foundation for a more robust multimodal tool. Likewise, TNM Version 3 is slated for release in 2011. It will be the target version in terms of requirements to be integrated within AEDT. At the start of Build 3, it is estimated that AEDT and TNM Version 3 will have been in use for 3+ years and the various pros/cons and areas of improvement will have been identified. Most of the âbugsâ that are inherent to computer programs should have been identified and corrected. In essence, AEDT is expected to be sufficiently mature so as to be the foundation for this Build. Starting with AEDT, Build 3 would include expansions for all modes of transportation, new meteorology input/internal modeling needed for noise analyses, and expansion of the GIS-based GUI to account for the various multimodal requirements. The postprocessor developed in Build 1 was required to utilize AEDT protocol and is expected to be leveraged as part of this Build. Additionally, the screening models of Build 2 would be implemented to allow rapid advanced planning analysis of changes made to each mode. Another round of market research as described in Chapter 5 would be required. Comprehensive inclusion of the non-aviation modes is expected to build upon and leverage the capabilities of existing models such as TNM. All modules that will be developed to interact between existing models must be fully compatible with the AEDT build framework, e.g., Microsoft .NET, etc. In cases where the state-of-the-art predictive capabilities, including the sciences have been advanced, it would also be desirable to include these changes during model implementation. Non-aviation source levels (noise and emissions related to air quality) would be included in a database similar to that of AEDT. As with AEDTâs airports database, it is envisioned that railway and roadway databases will need to be developed. Since these databases would have to be created, each should be compatible with AEDT and would most likely be facilitated using GIS layers. Other possibilities include movements databases, which are available for maritime, and to lesser degree for rail. Development of the GUI should be done in such a way to prevent the model from being unwieldy. Not only must the GUI control input and output, but Build 3 may include limited GIS capabilities to support new module development. The GUI should also allow the user to make selections of the desired functions needed for any particular analysis. The screening tools which were developed in Build 2 would become mitigation tools allowing What-if analysis. This would be accomplished by using the single mode tools as a quick analysis of possible changes adding more flexibility if needed. For example, if rail is considered to be a minor part of the overall air quality concentrations, the screening tool module would permit a range of operations to be quickly analyzed. The output of the rail screening module would then be summed into predicted concentrations, and using the GUI, the user could quickly see the progression of impacts. If the summation shows little change it can be quickly determined that the rail component need not be considered in greater detail. If however, there is a significant change, or concentration levels approach the National Ambient Air Quality Standards, then detailed analysis must be conducted. The screening tools then become an effective mitigation analysis tool for quick planning. While the GUI is anticipated to be the primary input mechanism, other inputs should also be included to assist the user. For example, the model should have the ability to read Extensible Markup Language (XML) files and database files such as the Structured Query Language (SQL). This input should be flexible, related to state-of-the-art at the time of build, and based on available options in the existing agency models. The output will also be streamlined and tailored to the specific analysis to be undertaken by the user. Current thinking is that the GUI would be customizable based on the specific analysis to be undertaken. The user would be queried regarding elements of their specific analysis, and a tailored GUI would help streamline analysis and make the process more user-friendly. Alternatively, all of the
2-6 databases would be open to the user and the more advanced usersâ could write their own tools to write directly to a study specific database and avoid the GUI. In such cases, it is recommended that preapproval of any developed tools by the lead agency during the analysis would be a prerequisite. It should be noted that this follows ongoing development of AEDT and that the primary database will not be changed, only the study specific database. Flexibility will also be built into the model. For example, if an analysis begins and then an additional mode must be considered, the model will have the capability to add this source, such as using a pull down menu, which will change the input GUI and output content. In Build 3, combination of the model results will be both easier and more difficult at the same time, requiring the rule base to be adapted. Since the models will all be contained in a single framework, processing of outputs will be an internal function for the software instead of external manipulations. The framework can be directly used for calculations during runs making the combining of information easier to accomplish and transparent to the user. At the same time, a higher degree of fidelity of the output combination rule base will be required of the combined results than in Build 1 and 2. Advancement of the rule base in Build 3 will need to be based more on the science and less on approximations to reduce the uncertainty of the final output of the combined data. The GIS-based capabilities will be expanded in this Build to allow a limited degree of input and output. It is the foundation for a lot of the capabilities in both AEDT and TNM and should be advanced so that in Build 4 full advantage of this resource may occur. The GUI will essentially serve as a software layer that will invoke the wide range of capabilities of GIS, such as mapping, demographic analysis, standard reporting, 3-D views, etc. in Build 4. In Build 3, the process should begin by allowing some additional modal input into the mapping capabilities and accomplished in such a way to allow other components such as the addition of the demographic analysis to be more easily integrated in Build 4. Depending upon available resources, other parameters could be brought into the model at this time, e.g., vibration due to noise. However, research to establish additional parameters such as vibration criteria is not a part of this process. Only defined parameters would be included. In the case of vibration analysis, the method will come from the FTA Guidance on Assessing Noise and Vibration Impacts. This Build will include a major V&V component. Required V&V will include extensive error testing for GUI input, verification of the outputs, testing of the rule base, sensitivity analysis for inputs, and demonstration of the capabilities against real studies. This will be accomplished by anticipating testing needs as part of regular development. For example, standard testing methods such as unit tests will be included as part of software design and development. The risk for this Build is substantially larger than the first two Builds and thought to be moderate to high. This is because of the increased number and complexity of tasks that need to be completed, the multiple computer languages to be interpreted, and the demands placed on GUI and moderate GIS development. It must also be recognized that all computer algorithms may not be on the same computer, especially in the cases of very large, computational intensive models. In these cases, calls could be made to these computing facilities or servers for the actual processing. This could result in access problems for some users. One possible reduction to this risk would be to divide this Build into different work efforts that could be sequentially completed. However, division of this Build, or any subsequent Build, should be done in such a way that the interim products result in a useable product. Build 3 is a major effort and is estimated to take 3 years to complete. Of note is that consideration could be given to divide Build 3 into two parts if time requirements are thought to be more than 3 years. This would allow the development phases not to exceed 3 years, the time frame for keeping development current and responsive to the stakeholders needs as previously discussed. Resources will also be critical to allow all work to be accomplished on time and will require interagency funding. It is estimated that the budget will be $4,000,000 â $7,000,000.
2-7 Build 4. 2nd The next large change in terms of effort is the expansion of the environmental impact and economic analysis tools in the APMT assumed to be completed by this time, which work in conjunction with AEDT (see Figure 1). If APMT is not available, additional time and resources would be required to establish an economic analysis tool or this task could be scheduled for a later date. These models would need to be expanded to include all modes of transportation. While this is not a trivial effort, it will allow for direct evaluation of cost and benefits as a part of the overall, multimodal environmental planning process. generation AEDT- based tool. Build 4 is expected to have the same look to users as Build 3 but significant major internal architectural and software changes will occur. The largest of these changes will be the use of shared (harmonized) algorithms for the different modal sources. In Build 3, existing models were imported into the overall âumbrellaâ of the program. Beginning in Build 4, these models will be broken down and analyzed. Where possible (e.g., geometric spreading for noise or dispersion modeling for air) the equations will be shared, eliminating the need for multiple models. It is also important to note that specific algorithms may need to be maintained if sharing would in any way reduce the usefulness of the model to a specific federal agency. The sharing of algorithms should not be undertaken if the result is any degradation in accuracy or robustness or if considered to change the needed requirements of a federal agency. Again, market research should be an integral task. Complete documentation of all changes to codes, simplifications, or modifications is critical to acceptance of this approach. Currently APMT uses reduced-order models of air quality and climate (based on more complex simulations) to estimate how changes in aviation emissions inventories may lead to changes in climate and air quality. Changes in physical impacts (e.g. ambient pollution concentrations) are used to estimate changes in health and welfare impacts. Then established methods are used to make economic estimates of these impacts. For noise, contours are overlaid on census data (housing, population, personal income) and estimates are made of the economic impacts. These analyses are conducted within a probabilistic framework as a means of explicitly propagating and communicating the uncertainty in such estimates. With further development, these methods could readily be adapted to be applicable to other transport modes. However, these methods are also the focus of ongoing research (e.g. through the Partnership for AiR Transportation Noise and Emissions Reduction, PARTNER, and the Aviation Climate Change Research Initiative, ACCRI) and it is likely that the methods will be different at the time that Build 3 is complete and consistent multimodal transportation emissions and noise inventories are ready for the assessment of environmental impacts. Therefore, Build 4 will leverage as appropriate the development of APMT and other research to enable environmental cost-benefit analyses of policy and other scenarios for the various transportation modes. Here we have assumed that the industry and consumer cost impacts of different policies and mitigation scenarios would be exogenous inputs and that the multimodal model would produce only economic assessments of the environmental impacts. There would certainly be scope for more fully integrating economic models, but the appropriate models would be different when applied at the national or local level and thus at this time it is felt that it is most appropriate to not integrate these directly into the model. The GUI should also be updated and refined as needed, based on the cumulative experience of users. In this Build, GIS implementation will also be much more prominent. Full use of the changes began in Build 3 will be achieved allowing all modules that interact between existing models and AEDT to be fully compatible with the AEDT build framework, e.g., Microsoft .NET, etc. The emphasis of these updates should be increased computer compatibility of the individual modules, ease of use, flexibility of analysis, and federal agency preference.
2-8 All architecture and software updates should account for the fact that later Builds will be focused on simulation modeling. Any programming decisions that could directly help the transition to simulation modeling should be considered and implemented if resources permit. Extensive testing of this Build will be needed. This should include verification and possible validation as compared to âgold standardâ measurement data for each source, as resources allow. Extensive error testing for GUI input, verification of the outputs, testing of the rule base, and sensitivity analysis for inputs will be needed. Since the basic environmental modeling of the new tool will deviate from established models, compliance with federal requirements such as 40 CFR Part 51, Appendix W (1), and 23 CFR 772 (2) will need to be confirmed. This will need to comply with requirement of all federal guidelines. The level of risk for this Build is moderate to high. Use of shared algorithms, while very important to programming efforts and run time, could result in two major flaws. First is incorrect algorithm implementation. Each of the modes and disciplines has unique functions and nomenclature â extreme care must be taken in harmonizing and rectifying differences in implementations of algorithms. Second is the incorrect inclusion of more obscure physical realities of the various sources, such as emission release temperature, height, directionality, etc., using the shared equations. The economic analysis model expansion could also be a difficult task. The interdependencies of all modes may be difficult to capture for the economic analysis. All modules that will be developed to interact between existing models and AEDT must be fully compatible with the AEDT build framework, e.g., Microsoft .NET, etc., given the varying metrics and analysis techniques/assumptions used across the modes. Additionally, ensuring compliance with federal regulations could be time consuming, contributing significantly to the risk the time, and overall effort required for this Build phase. The expected work effort is estimated to take 2 years, although federal requirements could extend this effort. The possibility of extending the time frame to 3 years and/or dividing into two efforts also could be applied to this Build. The cost is approximated to be $2,000,000 - $2,500,000. Build 5. Hybrid Model. This Build begins the formal transition to simulation modeling. The transition is expected to be complete by Build 6. Changes that can be made without significant redevelopment should occur in this Build to allow simulation modeling to be more easily implemented. These changes include: (1) adapting input data and the GUI to accept information that can be directly used in simulation; (2) the inclusion of emission and reference levels for sources in a form suitable for use in simulation; and, (3) the ability to import existing simulation algorithms. The algorithm import is key to this phase - Build 5 is not intended to develop simulation models, but rather to use what is available from current simulation efforts, including the Department of Defense (DoD) aircraft modeling, the IMAGINE project, as well as models that have been developed and tested for other transportation sources. The modeling for other transportation sources is expected to be better defined by the time this Build occurs and should be a part of the market research as described in Section 5. This is true not only of simulation modeling for environmental analysis but the simulation modeling of operations as well. Where simulation models capabilities do not exist at the time of this Build, the models currently in use will be continued. This is especially important in the case of some transportation facilities such as small airports. The move to a hybrid model will also change the rule base for combining results. Due to the nature of simulation, the output combination rule base will change. For example, simulation modeling would have greater detail on noise levels since values would be available for each step (time or spatial). This would allow multiple noise metrics to be computed since noise levels could be combined using the various time steps. This could eliminate simplifications that may have been required in earlier versions of the rule base. Similar to Build 3, Build 5 will require extensive testing of the integrated versions of the existing models that are used. Testing should include error testing for the GUI input, verification of the outputs, sensitivity analysis for inputs, and demonstration of the capabilities against a real study. Additionally, a
2-9 comparison to âgold standardâ measurement data should be conducted for any source for which the processing is changed in any way. The level of risk for this Build is moderate. Only proven methodologies will be used and the staged fashion of the previous development Builds will allow a similar approach to be utilized. This Build is expected to take 3 years for a cost of $2,000,000 to $4,000,000. The primary driver for the scope is the integration of the new methodologies. Build 6. 1st In Build 6 simulation modeling would be used for all sources. This will require development of simulation modeling techniques for those transportation sources for which they do not already exist. This development would be based on previous work and associated literature, and implemented based on the shared algorithm use protocol. Changes to input parameters would need to occur. This in turn would require adaptations to the GUI. generation simulation model. The end state model is what was determined to be the ultimate goal when resource limitations were not considered (see Section 1.2). When resource limitations (time and cost) were considered, the practicality of the approach came into question. Simulation modeling provides flexibility and options that are not available in steady state modeling but at a higher cost. A staged approach was selected by the research team based on the market evaluation of the stakeholders and the research teamâs expertise. Build 6 is expected to be the culmination of this work. Building upon each phase, multimodal impact modeling for all modes of transportation would be available in a true simulation model. Market research for this Build will be needed to establish that this is indeed the true end state model. While the emphasis would be on software development in Build 6, technical developments and clarifications relating to simulation modeling would also need to occur throughout the Build phase. However, given that this Build would not likely begin until after eleven years from the start of the first Build, it is expected that simulation modeling will see significant advances in this time frame. This could make implementation easier than is currently expected. The extent of testing for this Build will depend on the state of simulation modeling when it is begun. The more development required, the more testing that should occur. At a minimum, testing should include verification and possible validation to measurement data depending upon available resources. This is especially needed for any source for which new simulation algorithms need to be developed. Extensive error testing for GUI input, verification of the outputs, testing of the rule base, and sensitivity analysis for inputs should be conducted. The validation for the database prepared in Build 5 may also need to be repeated. Since the development will represent new models, compliance with federal requirements defined by regulations such as 40 CFR Part 51, Appendix W (1), and 23 CFR 772 (2) may need to be verified. The level of risk for this Build will also depend on the state-of-the-art in simulation modeling at the time of project inception. It is thought to be moderate to high at this time, but could be reduced if significant algorithmic advancements are made in the area of simulation before Build 6 occurs. Similar problems for shared algorithms could exist as those outlined for Build 5, but to a lesser degree. Additionally, confirmation of compliance with federal regulations could again be time consuming, adding delay to the overall Build phase. Build 6 is expected to last 3 years with a cost of $2,000,000 to $4,000,000. Summary of Build strategies. The Build strategy was implemented to allow a desired end state to be reached that was not thought to be practical with resources that would be immediately available for a single Build. The process begins with Build 1 where all modal outputs are combined into a single evaluation package using a controlling GUI. From this first Build, better consistency between individual
2-10 analyses should be realized. A rule base to combine the outputs will be required because of differences in sources, metrics, and modal regulations. Build 2 leads to the development of screening tools for advanced planning. This Build allows the initial development of a methodology to create what-if scenarios and mitigation modeling based on all modes of transportation. It also begins the transition to computer implementation consistent with AEDT. In Build 3, the strong model base made possible by AEDT development is utilized and other modes are included via integration of established models. The concepts of what-if scenario development and mitigation evaluation are brought forward from screening tools from Build 2 and will begin to be implemented as a way to conduct mitigation strategies in advanced planning. Software development must remain consistent with AEDT architecture and associated protocols to be effectively implemented. The rule base for combination of impacts for each source development is also continued. Build 4 continues the AEDT expansion, but begins to use shared algorithms to better harmonize the methodologies from various transportation modes. The APMT is also expanded for all transportation sources in this Build. Build 5 begins the transition into simulation modeling using a hybrid of both simulation and existing non-simulation modeling techniques. The rule base for the combination of outputs would need to be reevaluated and it is envisioned that earlier simplifications could be revisited and/or omitted. Well over eleven years will have passed from the start of Build 1 until the commencement of the Build 6 process of developing a true simulation model for all sources. More than fourteen years from the first Build, the completion of Build 6 would provide true, integrated simulation modeling for all transportation modes. This model would offer flexibility and options not possible without the use of simulation techniques. Programming of chemical reactions in the exhaust plumes, combination of noise levels using multiple descriptors, modeling of any moving transportation source whether at an intersection or moving along a runway would then be possible. The degree of analysis would be far greater for project scale modeling. Times in route for global modeling could be adapted for multiple time related events leading to more accurate environmental estimates. For the schedule outlined in this MDP, the investment cost would range from $800,000 to $1,100,000 per year. Adjusting for any time needed between Builds and for federal compliance, assumed to be 6 additional years, the required investment value is more likely to be an average of $500,000 to $750,000 per year. Each Build is expected to result in a usable product (model or feature). Each Build is expected to be a complete working methodology with technical manual and user guide. Each build may be separated into individual projects or may be combined depending upon the market research. A reevaluation process is expected to occur after each build to determine next step(s) and should include stakeholder input and a review of state-of-the-art practices, especially advances in technology. Documentation of testing, evaluation, and acceptable performance will be included in each Build. Each Build, will permit mitigation options, although this will not be a formal component of development until Build 3. Economic analysis at the local and regional level is expected to be included starting with Build 4. Flexibility must be maintained to accommodate transportation facilities such as smaller airports. 2.4. Other Build Requirements Model evaluation. Model evaluation includes testing for both the software implementation and correct implementation of technical methodologies such as dispersion modeling. During each Build description, general testing requirements were described based on the tasks to be performed. In this section, more description of each type of testing is described. This section is not meant to suggest a specific testing procedure (e.g., spiral testing, âV&Vâ testing, online or offlineâ). These types of tests are described in multiple texts and would best be selected by the project team for each Build. (3) Instead, components of those tests plans are discussed.
2-11 Technical methodology testing. Regardless of the test method chosen it must be verified that methodologies have been implemented correctly and provide accurate results. It is assumed that the methodologies, implemented as computer algorithms, will be based on theory and assumptions and/or simplifications completely justified and documented as discussed in Section 2.2. But a complete testing and quality control plan must be determined with each Build and included in a detailed work plan. In addition to making sure the implementation is correct, sensitivity testing based on range of variable and Monte Carlo testing must occur to establish reasonable input parameters such as the range of input. All final results of each Build should be compared to a real world test case with representative data as well. Software testing. Rigorous assessment, validation, and verification processes are crucial to the success of any software development process. A model similar to that undertaken for the AEDT development process is recommended for this effort. (4) Again, multiple test processes are described in the literature. Whichever is selected, during project work plan definition, it must allow for performance testing, correct implementation of variable and equations, uncertainty of results, and proper operation of control functions such as the GUI. Performance criteria and comparison to known results should be conducted. It is recommended that both individual computational module- and system-level assessments be utilized throughout the development. Assessment should include direct support from domain experts (economic, environmental, and transportation industry) and include the review of any existing bodies of data and results. Formal evaluations such as Global Sensitivity Analyses (GSA) should be undertaken to coordinate and provide input, intermediate, and final data for comparative checks on all releases/analyses versus previous versions. Formal software acceptance test plans and data sets should be developed and maintained. These should include an established battery of tests to be conducted for release of modules and applications and utilize verification against prior analyses and data sets. For example, in the aviation mode, validation against cockpit flight data recorder (CFDR) data, radar and other sensor data (e.g. ETMS, PDARS), external modeling systems, and measured data, should be conducted as available. System database maintenance processes should include cyclic updates to core data, addition of new source data, as required, and a QA process on both intermediate and result data. Comparison to measured data. Quality controlled measurement data is often used as the âgold standardâ. This testing should be required any time an existing model is not used and development has occurred. Any newly developed model component should be compared to this real world test case with representative data. Additionally, federal regulations should be followed to make sure the testing meets requirements of these agencies. Examples of this type of regulations are 40 CFR Part 51, Appendix W (1), and 23 CFR 772 (2). After careful testing, evaluation releases of the model should occur in the typical alpha and beta testing formats. This will help to uncover âbugsâ that may be only pesky to serious. The exact methodology to conduct this testing is not specified here since the quality control and experience by the successful bidder is crucial to the final product development and testing methodologies should be decided by the selected development team working with the stakeholders. However, a good example of a successful process is that currently being used for AEDT. This is described in Chapter 6 of this document. Reformulations/recompile. The model must be refined if test results indicate that the model does not meet preset conditions. This may require only small software âfixesâ to avoid rewriting and recompiling of all computer algorithms. Development Framework and Computer Platform Considerations. At the beginning of each Build, it is possible for the development framework to change. For example, Build 1 and 2 kernel programs are expected to run on a personal computer (PC) with possible calls to a server. By Build 3 this
2-12 could well change so that the PC is only used as a control point to receive input and return output. Reasons for this change would include program security, the large run times that would be expected on PCs, and the large data bases and model base that would result when the models are combined. All computations and all core models could be server based to alleviate these problems. Additionally, it is the general opinion of the research team that future software development for very extensive modeling systems will move to server based systems because of the run times and storage needs. This is necessary because of the extensive computational ability that will be needed in some scenarios, especially for regional, national, and global analysis. This would have to be an open platform, maintained at a single location, so that all users had access. This would require a user fee to maintain the platform but based on the time savings and equipment costs for individual users would still provide an economic benefit to users. Smaller analysis, without the extensive database requirements, could still be possible on stand alone units. GIS and/or Computer Aided Design (CAD) are expected to become more prominent and more desired by modelers and analysts. GIS and/or CAD are included in all Builds. The savings in data input time would be very significant if direct read of various media are possible. Due to the possibility of GIS/CAD functions being resource intensive to develop, a proven methodology should be chosen in Build 1 and unless radical changes are needed, the basic programming should be kept and continually developed with each new Build. To properly implement these strategies, strong graphic support will also be needed as well as export capability into mapping packages. Run times can be reduced in four ways; faster computer speeds, parallel processing, simplification of algorithms and streamlined programming such as the use of shared algorithms leading to fewer call statements. The first two are a function of the platform while the second two are functions of the programming. The program functions have been included in the Build statements. Possible platforms were previously discussed in this section. Testing has been covered in a previous section. Directly related to testing, but a portion of the development framework, is the source database consideration and validation. With Build 3 and subsequent Builds, this must be consistent with AEDT protocol to avoid errors in calls, increased run times, and possible errors due to implementation. External Agency Program Development. During development of the model described in this document, it is possible that changes could occur to the base agency models such as AEDT. Depending upon the degree of change to the agency models, a difference in the computed values could occur when the model is compared to the revised agency models. A time table for Builds has been established in short cycles of about 3 years. During these short cycles, it is expected that the number of changes (newer versions) of the individual agency models will be small. Agencies will be aware of this development effort for a full multimodal model and hopefully support it rather than sticking to the stovepipe culture of having individual models. By Build 3, agency models should be incorporated into the single multimodal model which should prevent development of the individual models. Chapter 5 discusses the importance of stakeholder input to the MDP and Appendix D suggests an interagency forum to oversee the development. Early involvement of the federal stakeholders should minimize any coordination problems between the development of this model and the individual base agency models, but it is an issue that the development team should monitor, especially during Builds 1 and 2. Build 1 uses the outputs of the individual models and the output file structure of the agency models should only change if there is a major modification or replacement of the model. It does not matter if values change since data would be read by the Build module. As such, a problem with the multimodal Build 1 would only be required if a major modification in the output file structure of an individual agency models occurred. In this case there would be two options: 1) redo the processing model (Build1) to account for the changes, or 2) use the user defined input option for the short term. The first option would require additional resources to reprogram Build 1 to read then new output file. This
2-13 would be much less work and effort than originally required for model development. The second option would not require any changes to Build 1 or additional resources. Because of the relatively short time frame, the second option would seem to be the most preferable. The second option would require the user to manually enter the results for the individual model that was changed resulting in additional work during input but no changes to Build 1. Build 2 is the development of screening models. No real changes would be required since it works with Build 1 based on the existing screening models. The screening models are to be based on a review of the science when developed and if changes to agency models are underway or planned; this could be considered during development. Legal i ssues. This document has not recommended changes to Federal or State policy. This however, is a reality that could occur as the project Builds occur. Not only could this add considerable time and resources to estimates but could change the way the work is performed. During the stakeholder input recommended for each Build, these issues should be addressed and the work effort scaled and/or changed as appropriate.