Assessing the Reliability of Complex Models Mathematical and Statistical Foundations of Verification, Validation, and Uncertainty Quantification (2012) / Chapter Skim
Currently Skimming:
1 Introduction
1 Introduction
Pages 7-18
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 7... ...
Briefly and approximately: verification determines how well the computational model solves the math-model equations, validation determines how well the model represents the true physical system, and uncertainty quantification (UQ) plays important roles in validation and prediction.
|
From page 8... ...
correctly solves the equations of the mathematical model. This includes code verification (determining whether the code correctly implements the intended algorithms)
|
From page 9... ...
The modeling framework assumed throughout most of this report is common in science and engineering: a complex physical process or structure is modeled using applied mathematics, typically with a mathematical model consisting of partial differential equations and/or integral equations and with a computational model that solves a numerical approximation of the mathematical model. Referring to the issues listed above, this report considers scenarios in which: • Models are strongly governed by physical constraints and rules, • The availability of relevant physical observations is limited, • Predictions may be required in untested and/or unobserved physical conditions, • The physical system being modeled may be quite complex, and • The computational demands of the model may limit the number of simulations that can be carried out.
|
From page 10... ...
1.4.1 Verification Code verification -- determining whether the code correctly executes the intended algorithms -- presupposes a computer code that has been developed with software-quality engineering practices and results that are appropriate for the intended use. This report assumes that such practices are in play but does not discuss them.
|
From page 11... ...
Measurement and inference errors contami nate the QOIs determined from physical observations or experiments. The difference between the computational model and the mathematical model -- partly caused by numerical approximations and imperfect iterative convergence -- can make it difficult to infer anything about the mathematical model and difficult to determine whether a computational model is getting the "right answer for the right reason." Validation must also take into account the effects of uncertainties in model-input parameters on computed QOIs; thus, the validation process involves UQ processes.
|
From page 12... ...
Even if a new problem can be considered interior to the set of available physical observations, the estimated prediction uncertainty may be unreliable unless the QOI is a smooth function over this domain space. These and related issues are discussed in Chapter 5, "Model Validation and Prediction." 1.4.4 Uncertainty Quantification The definition adopted in this report for uncertainty quantification describes the overall task of assessing uncertainty in a computational estimate of a physical QOI.
|
From page 13... ...
The most common method of dealing with uncertainty in VVUQ is through standard probability theory. In this approach unknowns are represented by probability distributions, and rules of probability are used to combine the probability distributions in order to assess the uncertainty in the predictions derived from computer models.
|
From page 14... ...
Although open to other possibilities, the committee holds the view that currently the use of standard prob ability modeling is often a reasonable mechanism for producing an overall assessment of accuracy in prediction in VVUQ, and it provides a consistent framework in which this report can illustrate its points. Interval descriptions, such as 0.09 < x < 0.11, are treated in this report by conversion to standard probability distributions: for example, by treating p as being uniformly distributed over the interval (0.09, 0.11)
|
From page 15... ...
compared to an analytical solution for a verification assessment of 3 bowling ball drop times between 10 m and 100 m. The physical constant g is assumed to be unknown, but be 2 tween 8 and 12 m/s2 (light lines)
|
From page 16... ...
Here the difference between measured and actual drop times should be within 0.2 seconds 95 percent of the time. These deviations, sometimes described as measurement "errors," may be due to the timing process or to variations in the initial position and velocity of the bowling ball as it is released for the drops.
|
From page 17... ...
Such propagation analyses, which can be carried out in principle using a Monte Carlo simulation, can be very time-consuming when the model is computationally demanding. Dealing with such computational constraints when exploring how the model outputs are affected by input variations is considered in more detail in Chapter 4, "Emulation, Reduced-Order Modeling, and Forward Propagation." 1.6.6 Validation and Prediction At this point, the experimental observations need to be combined with the computational model in order to obtain more reliable uncertainties regarding the simulation-based prediction for the QOI -- the drop time for the bowling ball at 100 m.
|
From page 18... ...
Chapter 6 addresses the use of com putational models and VVUQ to inform important decisions. Chapter 7 discusses today's best practices in VVUQ and identifies research that would improve mathematical foundations of VVUQ.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.