National Academies Press: OpenBook

Staffing Standards for Aviation Safety Inspectors (2007)

Chapter: Appendix A Modeling Methods

« Previous: References and Bibliography
Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×

Appendix A
Modeling Methods

In Chapter 2, we define models and describe the distinguishing features that could be considered for staffing models for the Federal Aviation Administration’s (FAA) aviation safety inspectors (ASIs). We note that predictive, stochastic models, designed as decision support tools, that assist with both allocation and sufficiency decisions, are most appropriate for the FAA’s staffing models. We briefly note the distinction between process models and statistical models and conclude that a process-modeling approach may be superior in the FAA’s context because it is more easily adapted during environmental changes of the type that characterize today’s aviation landscape. The purpose of this appendix is to more precisely describe those two approaches, to emphasize the crucial role that parameter estimation plays in either modeling approach, and to note that statistical procedures are often used to estimate the parameters in process models.

Process Modeling

Process models take many forms, from simple flowcharting to more complex discrete-event simulation. In this discussion, we focus on discrete-event simulation as a more robust form of process modeling that permits prediction of complex system behavior as a result of system design. In this case, the system to which we refer is the FAA regulatory and oversight system that employs ASIs to ensure aviation safety. System design includes all of the factors that drive staffing demand as well as the supply and work capacity of ASI inspectors. In this discussion, we as-

Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×

sume that one wants to determine through the use of process models how the two are interrelated, so that reasonable inferences can be drawn regarding the relationship between staffing system design and staffing system performance.

The specific approach we use to illustrate process modeling is known as task network modeling. In a task network model, system functions (e.g., performing all the required inspections in a region) are decomposed into a series of subfunctions, which are then decomposed into tasks. This is, in engineering terms, a task analysis. The sequence of tasks is defined by constructing a task network .

This concept is illustrated in Figure A-1, which shows a sample task network for a simple procedure—responding to a warning indication. The appropriate level of system decomposition and the portion of the system that is simulated depend on the particular problem. Staffing models have been developed that examine human behavior at the molecular level (e.g., detailed individual user interaction with the human-computer interface) and at a much more aggregated level (e.g., at which the task-level behaviors take hours or days).

In the ASI staffing context, the model might be at a gross level of granularity, since it need only represent inspectors or groups of inspectors as “busy” or “available.” The details of what they are doing are important only to the extent that they relate to factors that drive work demand and capacity.

The task network must also represent in some way the dynamics and dependencies of each task. Such factors include time to perform each task (possibly means and standard deviations), conditions that must be met for the task to start (e.g., available inspector resources, completion of a prerequisite task), and how performance of a given task interacts with other parts of the system.

Every time more than one path out of a task is defined in the network, a decision must be made by a human or other system element on what potential course of action should be followed, and the decision rules must be included in the model. The branching probabilities or decision logic can be represented by numbers, equations, algorithms, and logic of any complexity. In an ASI staffing model, the decision rules would reflect such factors as ASI task prioritization (e.g., required, planned, or demand work).

The paragraphs above describe the essential information that must be defined to adequately represent the relationships underlying a complex process such as the ASI staffing system. However, before a model can be useful in making predictions, it must be capable of accepting and processing changes in its inputs. In particular, the model must be able to accept and process a changing set of demands on the system (e.g., changing the

Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×

FIGURE A- 1 Sample task network for a procedure.

SOURCE: Laughery (2005). Reprinted with permission.

Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×

number, location, and mix of carriers, aircraft, and other elements that generate demand for ASI services) as well as a changing supply of ASIs (e.g., changing the number, type, and location of ASI inspectors in the part of the ASI system being modeled). When the staffing model is capable of accepting and processing changes to inputs, decision makers can use the model to estimate resulting changes in outputs. Those outputs can be as varied as the systems they represent. Typical outputs for staffing process models fall into two general categories: measures of personnel utilization (e.g., how busy will each type of ASI be at each location?) and estimates of delays in work completion associated with the unavailability of staff to perform the work. Such outputs allow users to estimate the effectiveness with which ASI staffing resources are used and their ability to meet work demand. There are many ways to express and quantify such information, and a model can be designed to provide the most appropriate and usable output to serve the needs of its users.

Statistical Modeling

A statistical model is based on observed empirical relationships—in this instance, the relationship between workload and the staffing necessary to accomplish the workload. The relationships are specified initially in mathematical form; the parameters or coefficients of the model are then estimated statistically from the data. Repeated observations that pair the workload accomplished with the staff hours expended to accomplish the workload are the minimum data necessary. Other variables should be added if they affect the relationship between workload and staffing; these might include location-specific factors. For example, some locations may require far more staff hours to accomplish a set of inspections when those inspections require extensive travel time by the ASI; in this case, a location factor that accounted for the relatively lower productivity of ASIs in certain locations would need to be included in the model.

An advantage of this statistical (or empirical) approach is that it does not require a detailed understanding of the process by which the work is accomplished. It does require, however, an understanding of the factors that may differentially affect productivity across work centers, so that the effects of these factors can be captured in the relationship. The process model approach breaks down the work tasks to the fundamental elements and models those elements. The statistical approach, in general, models the aggregate relationship. Under the statistical approach, one does not require an understanding of the detailed steps to estimate the aggregate relationship and apply those estimates in a staffing model.1

1

However, the approach can be used to model detailed steps, much like the process model.

Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×

Methods used in estimating relationships in the model include ordinary least squares regression, nonlinear least squares regression, and maximum likelihood methods. The relationship in its most general form is given by:

where y is staffing, measured in hours, days, or full-time equivalents (FTE), and the Xs are workload and factors affecting the staffing-workload relationship. A multivariate model, in which all important observable factors affecting productivity are captured, will provide the best estimates of prediction.

If the functional form or mathematical relationship specified were linear, then the model would be of the form:

where the parameters (α, β) are estimated from the data using, for example, ordinary least squares regression.

Because the statistical model uses the empirically observed relationship between staffing and workload, actual productivity is embedded in the model. The estimated relationship is usually based on the average observed productivity. It is also possible to estimate a “frontier” production function that provides the maximum (rather than the average) workload that can be accomplished (see, for example, Greene, 2002).

A major limitation of the statistical approach is that when processes or productivity changes, the previously observed statistical relationship is no longer valid. New data, generated under the new conditions, must be used to re-estimate the relationship. Thus, if it is known that the existing relationships between staffing and workload are not the relationships that will govern future staffing, then a process model approach would be more appropriate. With a process model approach, when a process changes, the model must also be modified to reflect the change. The difference is that, for a process model, the new process may be simulated, perhaps based on analysis and expert judgment, without waiting for new data to be generated.

Finally, it is important to recognize that the dichotomy drawn here between a process simulation model and a statistical model may be overstated. A process model may use statistical methods to estimate some or all of its parameters, while a statistical model may include relationships between workload or intermediate output at fine levels of detail, similar to the tasks and steps of the process model.

Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×

Parameter Estimation

Regardless of the type of model—process simulation, statistical model, or another type of model—values must be obtained for the key parameters of the model. These parameters provide the relationship between input variables or factors and intermediate or final output.

In the statistical model, by its nature, the key parameters are estimated statistically using actual data from observations or records of the key relationships. Hence, the parameter estimates are objective and empirically based. Moreover, the standard error of the parameter estimates can be calculated, using standard statistical methods. This provides an estimate of the accuracy of the parameter estimates.2

In the process model, there will also be parameters that must be estimated. In general, because a process model attempts to capture all or most of the key steps or processes in the staffing-workload relationship, there are likely to be more parameters to be estimated than in the statistical model.

Methods of estimating the parameters may include the same statistical methods as the statistical models themselves. That is, if the process model requires an estimate of the time required to make a certain type of inspection, one way to estimate this is through statistical estimation using data on actual inspection times, perhaps including other factors affecting the inspection time.

A second way to estimate parameters is through calibration. If one has data on the outcomes and basic inputs, one can calibrate the model by running it with a set of parameters specified using judgment. The model’s outcomes are compared with actual outcomes, and the parameters adjusted in directions anticipated to improve the fit or agreement between the simulated outcome and the actual outcome. The process is repeated until a satisfactory set of parameters—one that yields a satisfactory fit—is obtained. The calibration process is similar to statistical estimation, but less formal. Because of its informal nature, the properties of the parameter estimates are not formally established. However, it does have the advantage of being empirically based.

Finally, expert judgment can serve as the basis for determining key

2

In addition, estimation using certain statistical methods, such as ordinary least squares or maximum likelihood methods, often permit a claim that the estimate will have certain desirable properties. It may be the optimal or best estimate of the parameter under a given set of conditions regarding the distribution of the process. For example, ordinary least squares regression provides the minimum variance unbiased linear estimate of a parameter. Other methods of estimating parameters typically cannot claim such properties.

Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×

relationships between staffing and workload or intermediate steps in the process. While expert judgment can be useful, it must ultimately be validated by the accuracy of the predictions that are made with the model using the judgment-based parameters. Because expert judgment is subjective, it risks rapid loss of credibility without such validation.

Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×
Page 107
Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×
Page 108
Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×
Page 109
Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×
Page 110
Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×
Page 111
Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×
Page 112
Suggested Citation:"Appendix A Modeling Methods." National Research Council. 2007. Staffing Standards for Aviation Safety Inspectors. Washington, DC: The National Academies Press. doi: 10.17226/11742.
×
Page 113
Next: Appendix B Committee Data-Gathering Activities »
Staffing Standards for Aviation Safety Inspectors Get This Book
×
Buy Paperback | $46.00 Buy Ebook | $36.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

A primary mission of the Federal Aviation Administration (FAA) is the assurance of safety in civil aviation, both private and commercial. To accomplish this mission, the FAA has promulgated a large number of regulations and has established a major division, the Office of Aviation Safety, to enforce and maintain the regulations and effectively promote safety in aviation. Within the office there are several subordinate organizations.

Staffing Standards for Aviation Safety Inspectors is concerned with two of them: the Flight Standards Service (called AFS), charged with overseeing aviation operations and maintenance, as well as other programs, and the Aircraft Certification Service (AIR), charged with ensuring the safety of aircraft through regulation and oversight of their design and manufacture. The objective of the study is to determine the strengths and weaknesses of the methods and models that the FAA now uses in developing staffing standards and projections of staffing needs for ASIs and to advise the FAA on potential improvements.

Staffing Standards for Aviation Safety Inspectors is organized in an Executive Summary and five chapters. This first chapter provides the background of the study and explains the committee's approach to its task. Chapter 2 discusses modeling and its applicability to the development of staffing standards for such organizations as the Flight Standards Service and the Aircraft Certification Service. Chapter 3 traces the recent history of staffing standards in these organizations and considers manpower and staffing models and methods used by other organizations. Chapter 4 examines factors to be considered in the development of ASI staffing standards and the challenges faced by any methodology applied to this task. Chapter 5 presents the committee's findings and recommendations, including a discussion of issues and constraints that must be considered in weighing the implementation of alternative approaches.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!