Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
6-1 CHAPTER 6. REPORTING Reporting for all Builds should consist of quarterly reports, a draft final report, and a final report. Quarterly reports are expected to provide a complete administrative review of the project (budget, timelines, etc) as well as all key technical details of development. Draft and final reports should include all technical detail as well the use of appendices or other formats as needed. Key appendices should include user manuals and technical manuals. Depending upon the desires of the funding agency or agencies, the source code may also be required. Validation of the model versus gold standard data, as well as verification versus legacy tools and/or previous Builds, will be important to ensure system robustness and continued stakeholder engagement and confidence. All Builds should report these results and uncertainty evaluations. A crucial part of any software development is the careful documentation of all modules and databases. In addition to the source code, description of the general model architecture should be included. There are multiple formats that this could occur, but a consistent reporting format throughout the Build stages would result in the ultimate usefulness. Since AEDT is projected to become the base upon which to build, following the reporting format established for AEDT would seem to be beneficial. The AEDT developers have adopted a set of three primary documents for doing this: Algorithm Description Document (ADD); Interface Control Document (ICD); and Database Description Document (DDD). All modules and/or applications must have both an ADD and an ICD. The ADD defines all science and computations contained within the module. It does this explicitly and/or by reference, as appropriate. The ICD explicitly defines all input and output requirements for the module. Given the ICD for a module a 3rd party software developer should be able to call a âblack boxâ module, handing it the required input data and receiving back the resultant output data. DDDs describe every data field within a database, including formatting and units, as well as any assumptions associated with the use of the data, and data sources as appropriate. The level of effort required in reporting for each Build will vary. Build 1 programming consists of post processing outputs from existing models and as such is not as substantial of an effort as in later Builds. Additionally, the methodology used in these models will have been previously described in the literature. As such, actual software documentation is expected to be secondary to the description of the rules used to combine the various output. How problems are overcome, both on a technical and regulatory basis in this rule formulation will be the more important and extensive documentation. Build 2, requiring the development of screening models, will also require substantial documentation on the rules and reasoning behind the models, with the model documentation being more substantial than Build 1, but still less than required in the later Builds. The principles used to construct the screening tools could be the major effort during reporting. Builds 3 and 4, based on AEDT, will be able to utilize documentation created for AEDT but will need to be updated for the changes that occur. Changes will include revision and creating of new algorithms, changes to the database, and changes in input/output of the model. Complete documentation of this will be a substantial effort. Consequently, Builds 3 and 4 will require much more documentation of the computer model development than the first two Builds. Implementation and methodology description will also be extensive. Build 5, the beginning of simulation modeling, will require substantial documentation on both the methodology and the programming. If as expected, simulation models are continued to be advanced, then much of the methodology may already be in the literature and could be incorporated by reference which could reduce reporting efforts. Even so, the model implementation documentation is still expected to be substantial. Documentation for Build 5 is expected to require substantially more documentation than previous Builds.
6-2 Build 6, the first generation simulation model, will also require a substantial amount of documentation for both the methodology and programming. It is expected that continued research will have continued on simulation modeling allowing references to be heavily used. But documentation on the implementation of all sources is still expected to be a very substantial effort. In addition to the technical model reporting all Builds will require regulatory discussions to various levels. Since these projects will be multi-modal, differences in regulatory requirement from various agencies is inevitable. This will require very careful documentation of the stakeholder process in the reports and could also require statements of policy with the approval of the various Federal agencies.
