Why: To date, the majority of obesity prevention evaluation efforts have focused on methods that do not optimally recognize the interactions and relationships among the many factors that comprise a complex health issue such as obesity. As a result, important learnings may be missed. A systems approach to obesity prevention evaluation efforts can build on current evaluation methods and improve our understanding on the relationships and their impact among the Accelerating Progress in Obesity Prevention (IOM, 2012a) report strategies, outcomes, and stakeholders across a variety of settings.
What: Complex systems are a configuration of interacting, interdependent parts, connected through a web of relationships, that form a whole greater than the sum of its parts (Holland, 2000). The explicit recognition by evaluators that obesity is complex will help to facilitate the increased use of systems approaches in evaluation efforts for obesity prevention.
How: From a systems perspective, evaluating progress of obesity prevention efforts should include support for better understanding of the interactions and relationships among individuals, groups, communities, stakeholders, and national efforts across a variety of settings and throughout the evaluation process framework (see Chapter 3).
Obesity is complex. Attempts to treat or prevent it with individual behavior-change strategies (as a simple problem) have generally not been effective. Consideration of a complex systems approach to identify the best choices for obesity prevention is a worthwhile, arguably essential, endeavor. The importance of considering a systems approach to obesity is explicitly identified in the Institute of Medicine (IOM) report Accelerating Progress in Obesity Prevention (APOP) (IOM 2012a). The report notes that “a systems perspective helps to reveal, and create, the potential for combined impacts (or synergies) that can further accelerate progress in preventing obesity” (IOM, 2012a, p. 7). From an evaluation perspec-
tive, systems science provides quantitative methodologies that enable evaluators to consider the dynamic relationships of factors at multiple levels of analysis, but it also includes qualitative approaches to actively engage members of the community in a participatory process. Evaluation deals with challenges of the real world—dealing with reality as it unfolds demands a complex adaptive systems perspective.
The complexity of obesity introduces notable challenges for evaluators. Simple or uncomplicated systems are likely to be homogenous, linear, static, and independent, with little feedback or connection to other levels or subsystems. On the other hand, complex systems are a configuration of interacting, interdependent parts, connected through a web of relationships, that form a whole greater than the sum of its parts (Holland, 2000). Complex systems are mostly nonlinear, dynamic, interdependent, generate feedback, and have several connected levels. In a complex system, a given set of circumstances will not always yield the same result (Holmes et al., 2012).
A systems approach to obesity may help to make sense of the complexity (discussions related to systems and complexity tend to use a unique lexicon, and many of the terms used in this discussion are defined in Appendix B). By identifying leverage points and simple rules among a network of interconnecting nodes, evaluators can advance their understanding of how different components work together under various circumstances to influence movement in weight status. This enables evaluators to better identify where to look, what to measure, when to measure, how to recommend action, and why the proposed approach is important for decision makers to consider. A systems approach elevates the importance of understanding contextual factors and relationships to describe what is happening, why it is happening, and what may be the best action (how) to achieve progress in the intended direction.
To provide context in which a systems approach to evaluation may occur, the Committee considered some situations in which a systems approach could be used to take action. It should be kept in mind that systems bring together a set of elements into a meaningful set of relationships—connected and interdependent—that act together as a whole. In effect, systems work by uniting various elements in a meaningful relationship that acts as a whole. To evaluate these complex systems is to evaluate patterns of relationships, how they are sustained, how they self-organize, and how they emerge. Systems, therefore, can only be understood as an integrated whole; the result of both the sum and the relationships among its elements.
In the context of taking action to evaluate obesity prevention, action-oriented questions to ask include what happened, what will happen, and what is the best choice? All of these questions may apply to dealing with obesity as a system. However, as discussed below in more detail, the first two questions may be addressed by treating obesity as a predictable issue that may be solved through reductionist thinking and, as such, represent simple or at most complicated systems. On the other hand, the third question addresses obesity as a complex system that is not so predictable and, as such, cannot be understood through reductionist approaches (Finegood, 2011).
EVALUATING THE “WHAT HAPPENED?” QUESTION
To describe what happened, a basic reporting approach may be taken. This generally takes the form of surveillance and other monitoring techniques that describe the current state, which is attributed to activities conducted in the past. The “what happened” question context is representative of a static, independent, linear situation that potentially describes some combination of what has happened, when it
might have happened, where it may have happened, and how much, how high, what level, or what score may have been attained.
EVALUATING THE “WHAT WILL HAPPEN?” QUESTION
To describe what will happen as a result of actions taken, evaluation approaches will generally use techniques that predict or estimate the likelihood of certain desired outcome(s) based on the observed association with another set of factors or the measured changes in proximal intervention targets that allow for the prediction of desired ultimate outcomes. To address this question, evaluators use several approaches, including randomized controlled trials and variations on experimental designs that attempt to control for many confounders (Mercer et al., 2007). These evaluation techniques also tend to address the problem in a reductionist manner, attempt to identify cause and effect, and often fall short in solving the complexity of the interactions among the many factors associated with obesity. On the other hand, this type of approach may be very useful for addressing, to some level of confidence, the potential impact of actions proposed or taken, how to prevent adverse outcomes, or how to provide an estimate of anticipated value of interventions.
EVALUATING THE “WHAT IS THE BEST CHOICE?” QUESTION
To address the question what is the best choice of action, considering all available options, complexity science is needed. Complexity science is an attempt to understand how things influence each other within the context of the whole. The Committee considers obesity to be a complex system insofar as it possesses characteristics consistent with complexity that include being heterogeneous, nonlinear, stochastic, interdependent, generating feedback, self-organizing, and involving emergence (Finegood, 2011). A systems approach to taking action demands respecting the characteristics of systems noted above and considering leverage points that may influence change within a complex system—attempting to identify the “best choice” for action. The remainder of this chapter will focus on addressing the third question: “What is the best choice?”
HOW WILL EVALUATION EFFORTS BENEFIT FROM A SYSTEMS APPROACH?
Although relatively new, systems theory is already contributing to evaluations of obesity interventions (C3 Collaborating for Health, 2011). Evalations that do not consider systems dynamics or conditions will likely miss aspects of the intervention and its environment that influence the intervention’s operation and success (Hargreaves, 2010). To optimize the development of practice-based evidence from such studies, researchers and practitioners will need to develop evaluation systems that enable continuous learning and the pooling of findings and best practices (Holmes et al., 2012). To move toward systems-informed evaluation, researchers will need to reconsider the types of evidence needed from a systems point of view. For example, questions around effective leadership to facilitate systems change and leveraging of networks to advance health improvements become increasingly important with increased complexity (Holmes et al., 2012). Systems science (as it applies to community-based prevention) can address how causal structures change over time, including the effect of changes in the type or number of interventions implemented, or how changes in social norms, community practices, and leadership may affect outcomes. Examining these
causal structures can increase understanding about both intended and unintended consequences of interventions (IOM, 2012b; Meadows, 1999; Sterman, 2000; Ulrich, 2000). As one example of how a systems approach may be used to support evaluation, consider multiple uses of mapping the relationships among the many variables and factors of a complex system, including identification of leverage points, anticipation of unintended consequences and unexpected results, and comprehensive assessment of stakeholder views.
Mapping and the Use of Mapping
Systems maps (e.g., causal loop diagrams) help to visualize relationships. The process of developing a causal loop diagram is well suited for group work and facilitating communication among diverse stakeholders (Homer et al., 2006; Milstein and Homer, 2009). A systems map represents a dynamic hypothesis, an evidence-based or evidence-informed depiction of relationships, but the map itself cannot be tested (Homer et al., 2006; Milstein and Homer, 2009). Systems causal maps may be created with specific application to obesity and can be the result of a quantitative or qualitative approach to complex systems mapping. Excellent examples of complex systems maps for obesity are presented in Tackling Obesities: Future Choices—Building the Obesity Systems Map, a report from the United Kingdom Government’s Foresight Program (Vandenbroeck et al., 2007),1 the APOP report (IOM, 2012a), or An Integrated Framework for Assessing the Value of Community-Based Prevention report (IOM, 2012b). An example of the latter is provided in Figure 9-1.
Identification of Leverage Points
A map depicting the underlying theories of change and causal structures of a comprehensive obesity prevention policy system can increase the understanding of the multiple pathways of impact of obesity prevention policies and actions. Mapping these relationships helps to identify key leverage points in the system that may influence multiple other variables in the system. In turn these diagrams can help evaluators to identify and prioritize key variables and associated measures to be assessed. It guides evaluators on “where to look” for short-, intermediate-, and long-term effects of program and policy options, which may enhance the use and collection of relevant data (Diez Roux, 2011).
Anticipation and Identification of Unintended Consequences and Unexpected Results
The ideal, of course, would be to make only the best choices, but complex systems have emergent properties that make them unpredictable. Therefore, it is worthwhile to consider how to use systems approaches to identify and anticipate consequences of policies and actions taken that are not optimal or ideal but might be ameliorated. The use of obesity system causal maps can identify pathways that highlight actions and responses within the system that are more likely than others. For example, a map of the obesity prevention policy system can more explicitly elucidate the pathways from obesity prevention policies and programs to health outcomes as well as the underlying structures that reinforce or hinder change processes (IOM, 2012b, p. 146). This type of mapping can support the entire proposed framework (see Figure 3-1) for evaluating progress in obesity prevention. The feedback identified through mapping, positive and negative, reinforcing and weakening, anticipated and unexpected, provides opportunities for evaluators to explore the mechanisms that fuel or retard relationships and feedback in the system. This
1 See http://www.bis.gov.uk/assets/foresight/docs/obesity/12.pdf (accessed November 12, 2013).
FIGURE 9-1 Example causal loop diagram for value of community-based prevention policies.
SOURCE: IOM, 2012b.
means paying attention to the relationships among actors in the system, among program components, and among levels of the system. One example of unintended consequences in efforts to prevent obesity and increase fruit and vegetable intake among school children is illustrated in Box 9-1.
Comprehensive View of Elements and Stakeholders Involved or Affected by Change Efforts
To generate a useful systems model or map, multiple stakeholders and perspectives invested in the situation need to be engaged in the process (IOM, 2012a; Williams and Imam, 2006). Also known as participatory evaluation, systems-informed evaluators have a responsibility to “increase the voices of those in the margin” (Williams and Imam, 2006, p. 9) in all phases of development and evaluation of the intervention and establish common goals reflecting the various perspectives of the stakeholders. Their participation in building systems maps tends to unearth causal loops that experts and others from outside the community would not anticipate.
yumPower School Challenge
In 2011, a not-for-profit health plan in Minnesota announced a multi-year community collaboration called “yumPower” (www.yumpower.com) to increase “better-for-you” food consumption across a variety of community settings, including elementary schools. During the 2012 school year, 32 elementary schools (about 15,000 students) participated in the yumPower School Challenge pilot program designed specifically to increase intake of fruits and vegetables among elementary school students. The program kicked off with a 45-minute, high-energy, interactive school assembly led by Radio Disney at each school site. Students were given fruit and veggie trackers to record their consumption each day for 4 weeks. Schools received tracking supplies and pencils and notepads to provide students with reminders and incentives for tracking. Some key findings: 16 extra tons of fruits and vegetables were consumed by the students over this 4-week time frame, students increased their fruits and vegetables consumption by 11 percent overall and by 22 percent during weekend days, participation rates were as high as 76 percent, and finally 93 percent of teachers and school staff said they would recommend the program to other schools. An unintended consequence occurred as well: food service staff faced challenges dealing with the increased demand for fruits and vegetables. This conundrum prompted one food service staff person to say “be sure to warn food service before you start because we ran out of fruits and vegetables and needed to order more!” Programs designed to increase healthy food consumption for one group must consider all stakeholders affected by the changes that the program may bring about.
SOURCES: HealthPartners, 2012; Isham et al., 2013.
A SYSTEMS PERSPECTIVE ON THE CHALLENGE OF EVALUATING OBESITY PREVENTION
Complex systems are a configuration of interacting, interdependent parts, connected through a web of relationships, that represent a whole greater than the sum of the parts (Holland, 2000). Research supports the obesity-related influences that stem from genetics, neurobiology, physiology, psychology, behavior, family structure and influences, social context and social norms, environment, economics, markets, and public policy. However, the mechanisms of these relationships are not always well researched (Hammond, 2009). Because obesity is more likely to be the result of a multitude of factors interacting with each other over time than any single causal factor (Glass and McAtee, 2006) drives the field to recognize that obesity is complex and that progress in obesity prevention may gain significant benefits when evaluation strategies include a systems perspective.
An earlier IOM report, Bridging the Evidence Gap in Obesity Prevention: A Framework to Inform Decision Making (IOM, 2010), discusses the need to apply a systems perspective to obesity prevention. A systems approach to obesity prevention makes sense because multiple properties specific to obesity need to be considered in combination in order to address obesity prevention effectively (although this list of properties may not be all-inclusive). Properties to consider include the following:
• Obesity is related to social networks.
• Intervention outcomes for obesity prevention are affected by unanticipated, emergent situations and baseline assumptions that may not hold over time.
• Significant inter-individual variability exists in biological, physiological, and emotional responses to interventions.
• A significant knowledge gap exists related to weight accumulation.
• Resource allocation that addresses obesity prevention is related to other areas competing for the same resources.
With all of these properties to consider, using a systems perspective for evaluating obesity prevention efforts is challenging. Decisions around what systems approaches to consider may be informed by an understanding of which simple rules govern the behavior of the system and at what level of the system provides the most opportunity to make a difference. In addition, costs, scalability, and sustainability of interventions are important considerations (Hammond, 2009; Huang et al., 2009; Levy et al., 2011; Mabry et al., 2010; Nader et al., 2012). To discuss this in more detail, the next two sections address the concepts of “simple rules” and “system levels.”
Complex systems, given their dynamic and interacting characteristics, are inherently unpredictable, and interacting with them leads to continually emerging novel behavior (Plsek and Greenhalgh, 2001). Nevertheless, it appears that this emergent property follows an inherent self-organization facilitated through simple, locally applied rules. Examples of simple rules identified to explain complex behavior in, for example, the context of health systems include those associated with large system transformation and those that attempt to explain how health systems create value for their stakeholders. Best et al. (2012) identified five simple rules that were likely to enhance the success of transformation initiatives: “1) blend designated leadership with distributed leadership, 2) establish feedback loops, 3) attend to history, 4) engage physicians, and 5) include patients and families” (p. 421). Kottke et al. (2012) identified five simple rules that generate value for the health system’s stakeholders: “1) the stakeholders agree on a set of mutual, measurable goals for the system; 2) the extent to which the goals are being achieved is reported to the public; 3) resources are available to achieve the goals; 4) stakeholder incentives, imperatives, and sanctions are aligned with the agreed-upon health goals; and 5) leaders of all stakeholders endorse, promote, and honor the agreed-on health system goals.” Identifying the simple rules that govern the behavior of the system in question may point to a limited number of intervention opportunities that are likely to influence the outcomes and overall impact on the goal of obesity prevention.
Simple rules also relate to the concept of self-organization, that is, the capacity of a system to make its own structure more complex. For example, a researcher may consider the organization of a person’s social network. People who have friends who are overweight are more likely to be overweight themselves (Valente et al., 2009). Furthermore, if their social system changes over time, the likelihood they may gain or lose weight is highly correlated with the weight of the friends to whom they are connected (Christakis and Fowler, 2007). As such, the relationships a person has may be considered a simple rule related to body weight, but caution must be exercised in not equating correlation with causation.
In addition to simple rules, system levels also provide a means to make systems approaches more actionable. For example, when obesity interventions are only applied at the level of the individual and do not take into consideration what happens at the level of behavioral settings (e.g., worksites, schools, health care), sectors influencing (e.g., public health, education, business, government), or social norms and values, it is unlikely that the changes induced will be sustainable (IOM, 2012a). Malhi et al. (2009) simplified an original list described by Meadows (2008) into five system levels at which obesity interventions could be applied. These levels include (1) paradigm (a system’s deepest belief), (2) goals (what the system is trying to achieve), (3) structure (enhancing connections across most of the system as a whole), (4) feedback and delays (to effect self-regulation, self-reinforcement, and adaptation of the system), and (5) structural elements (to affect subsystems, actors, and the physical structure of the system). The paradigm level is the most difficult at which to intervene, whereas the structural elements level is the least difficult. However, interventions at the more difficult levels tend to be more effective and impactful. In the context of childhood obesity, Malhi et al. (2009) found that most recommendations for interventions are made at the structural elements level. This is the level that includes a strong evidence base and is associated with evidence of cause and effect. Paradoxically, it is the level least likely to be effective in creating and sustaining change in a complex system. Table 9-1 describes the intervention levels and associates them with objectives for evaluation.
A clear example of interventions implemented at multiple levels of a system is the tobacco experience over the past several decades. The interrelated set of activities that eventually connected several levels of interventions represents an evolution toward a systems approach to address tobacco use in the United States. As outlined in Bridging the Evidence Gap in Obesity Prevention: A Framework to Inform Decision Making (IOM, 2010), the combined and coordinated set of activities implemented at the individual, com-
TABLE 9-1 Places to Intervene in a Complex System and the Associated Evaluation Objectives
|Structural elements||Subsystems, actors, and the physical structure of the system||Assess level of coherence among the components of the system|
|Feedback and delays||Self-regulation, self-reinforcement, and adaptation of the system. Reinforcing loops for driving growth and balancing loops for constraining goals||Assess degree to which the system operates in a continuous manner, and assess the likelihood that the system will or will not grow|
|Structure||Enhancing connections across most of the system as a whole||Assess degree to which interdependent elements in a network work together|
|Goals||What the system is trying to achieve||Evaluate how well the system adapts to its environment|
|Paradigm||A system’s deepest belief||Understand the degree to which the system acts as a learning environment|
SOURCES: Adapted from Finegood, 2011; Malhi et al., 2009; and Meadows, 2008.
munity, population, system, and public education levels all contributed to overall tobacco use reductions in the country. Figure 9-2 presents this set of activities.
Simple rules and system levels can be used to identify opportunities to take action and may be regarded as a fundamentally different way of thinking about how to approach complex problems, such as obesity.
USING A SYSTEMS PERSPECTIVE TO ADDRESS POPULATION HEALTH: A REVIEW OF PRIOR IOM REPORTS
Previous IOM consensus committees have applied systems thinking to identify the types of evidence and research needed to guide decision making around obesity prevention; to identify promising strategies and actions; to anticipate the effects and unanticipated consequences of actions; and to explore different pathways through which interventions can lead to changes—for better or worse—in population behavior and health outcomes. This report builds from this knowledge base by applying a systems approach to evaluation design and implementation. The Committee built on three reports that present applicable discussion of system approaches to obesity prevention; they are briefly presented in Table 9-2 below.
EVALUATION TOOLS AND SYSTEMS SCIENCE
A diverse set of qualitative, quantitative, and mixed-methods evaluation and analytic tools are available to help to evaluate actions to address a complex problem such as obesity. The tools can be used to
FIGURE 9-2 Evolution of tobacco control approaches toward system thinking.
NOTE: ASSIST = American Stop Smoking Intervention Study; COMMIT = Community Intervention Trial for Smoking Cessation; ISIS = Initiative on the Study and Implementation of Systems; NCI = National Cancer Institute.
SOURCE: Adapted from NCI, 2007.
TABLE 9-2 IOM Reports That Used Systems Perspectives
|IOM Report||Description||Emphasis on Systems|
|Bridging the Evidence Gap in Obesity Prevention: A Framework to Inform Decision Making (IOM, 2010)||This report developed the L.E.A.D. (locate, evaluate, assemble, inform decisions) framework to help researchers locate, evaluate, and analyze evidence to inform and advance decision making. This framework can help evaluators to maximize opportunities to generate evidence from policy and practice, and it can help other end users—such as public health practitioners—collect, analyze, and present information.||
• Frame the problem
• Understand potential causes
• Identify critical leverage points of influence
• Take effective action
• Maintain a systems perspective throughout
• Give consideration to external validity of the evidence for applicability in other populations or settings
|Accelerating Progress in Obesity Prevention: Solving the Weight of the Nation (APOP) (IOM, 2012a)||The APOP recommendations addressed five environments—physical activity, food, message, health care and work, school— with the understanding that synergies across actions in these environments would advance greater movement in preventing obesity than action in any one area alone.||To understand the process of system change, the committee proposed monitoring engagement, communication, and leadership among all sectors to increase the development, implementation, and coordination of common messages, processes, and strategies.|
|An Integrated Framework for Assessing the Value of Community-Based Prevention (IOM, 2012b)||This report developed a framework for assessing the value of community-based nonclinical prevention policies and wellness strategies.||This report highlighted the utility of a systems science approach to understanding the collective impact of community-based interventions. It discussed the value of a systems approach to coalition formation, research design, and analytical descriptions and evaluations.|
identify changes in patterns of system behavior over time and to advance an understanding of the factors and conditions underpinning these changes. The tools range from sophisticated computer models, such as systems dynamics modeling (Homer and Hirsch, 2006; Sterman, 2006), dynamic microsimulation modeling (Mitton et al., 2000), and agent-based modeling (Axelrod and Tesfatsion, 2006; Epstein, 2006), as well as qualitative approaches such as focus groups and Concept Mapping (Kane and Trochim, 2007), which require little technological support. Some examples of tools presented in Box 9-2 are discussed in more detail below.
Simulated Virtual Worlds: A Quantitative Model
Simulated virtual worlds, an example of a quantitative method that is based on formal models, are widely used to advance hypotheses about how a system behaves over time. High-quality empirical research, especially well-designed experimental and quasi-experimental studies, is needed to inform systems models (IOM, 2010), which use data-driven assumptions to guide their predictions (Levy et al.,
Examples of Evaluation Tools/Methods for Complex Systems
|Quantitative Methods||Qualitative Methods|
|Agent-based modeling||Case studies, interviews, and focus groups|
|Time-trend analysis||Observation of activities|
|Observational or cross-sectional studies||Document reviews|
|Retrospective analyses||Outcome mapping, concept mapping|
|Adaptive learning measurement systems||Analyses of emergent system-wide patterns, tracking of events, encounters, and policy changes|
|Geographic information system spatial analysis||Use of simple rules and conditions of self-organization|
|Simulated virtual worlds||Soft systems methodology|
|Systems dynamic modeling||Appreciative inquiry, reflective practice|
|Dynamic microsimulation modeling||Group model building|
|Systems mapping/causal loop model building|
SOURCE: Adapted from Hargreaves, 2010.
2011). The ReThink Health dynamics simulation model was designed to help planners to test different quantitatively estimated effects of scenarios by tracking changes in health status, utilization, costs, and equity following different intervention options (ReThink Health, 2012). To varying degrees, the proposed interventions address risk, care, capacity, cost, funding, and/or economic and health care trends (e.g., prevalence of uninsured individuals, local economic climate). These simulated scenarios are also useful for evaluators because they shed light on what types of impacts can be expected under different conditions in the short- and long-term. As a result, they point to where in the system evaluators should monitor and measure change, and at what intervals of time. In essence, they can help to identify the best choice among a variety of options.
Group Model Building: A Qualitative Model
Group model building (GMB), a qualitative method example, presents a participatory process for developing and analyzing a formal model. This iterative approach, conducted over several meetings, fosters collaboration and stakeholder involvement, and it can help to address issues of transparency by articulating underlying assumptions. It may be especially valuable when working with marginalized communities (Hovmand et al., 2012). GMB requires teamwork, distinct roles (e.g., facilitator, recorder), and facilitation to help to advance collaboration and manage conflict. Process maps can help teams to visualize and plan the overall sequence of GMB sessions.
SYSTEMS IN THE CONTEXT OF THE COMMITTEE’S PROPOSED EVALUATION FRAMEWORK
The framework, as presented in Chapter 3, outlines five components, including inputs, activities, outputs, intended outcomes, and impacts (see Figure 3-1). Each of these components can be viewed through a complexity lens. The various evaluation methods and tools discussed above can be applied to each component’s set of activities shown in Figure 9-3 to support the evaluation and measurement effort. Some tools or methods can be applied to multiple sections or the entire framework at once, whereas others may be applied to a single step in the model. For example, a systems map can encompass sources of input, activities in the evaluation efforts, identified outputs and intended outcomes, and anticipated impacts.
Examples of systems approaches to obesity prevention evaluation that may be applied to the framework are provided in Figure 9-3. The examples are presented as part of each step of the model as well as in the context of the entire model. The “Inputs” box dialogue sessions may be one example of collecting insights that follow a systems approach methodology. The “Impacts” box the need to collect School Health Policies and Practices Study and School Nutrition Dietary Assessment Study–like data on child care settings on a national and periodic basis represents another opportunity for cross-agency collaboration. In addition the ability to identify sociocultural and socioeconomic determinants, timing of exposure, and living and working conditions across populations illustrates another important multi-level relationship that follows a systems approach methodology (further described in Chapter 5 and applied in Chapters 6, 7, and 8). Looking at the framework as a whole, and recognizing that complex systems include, for example, characteristics such as nonlinearity, interdependency, feedback loops, and emergence (Finegood, 2011), one realizes that complex social networks serve as an example of a systems approach to evaluation that includes
FIGURE 9-3 Examples of how various systems approaches to evaluation may be applied to obesity prevention efforts in the context of the Committee’s evaluation framework.
many of the systems characteristics noted and applies to the entire framework. Complex social networks, and their inherent feedback loops, have been shown to affect obesity. Using social network analysis, Christakis and Fowler (2007) were able to examine the spread of obesity over time among subjects of the Framingham Heart Study. These researchers showed that obesity was clearly associated with social relationships (e.g., friends, spouses, and siblings) and that the geographic distance among people (an example of physical environment) was less important than social environment. They were also able to show that obesity followed the friendships and concluded that obesity is, in fact, contagious.
Change in body weight is associated with an imbalance between energy content of food stuffs consumed and energy expended. Hence, to consider the potential of any obesity intervention, its impact on both energy intake and expenditure needs to be properly accounted for. Hall et al. (2011) provide an example of how predictions about the potential effect of an intervention may differ between more traditional approaches and systems approaches to obesity prevention. These researchers compared the results of a projected impact of taxing caloric sweetened beverages on obesity prevalence based on the extrapolation of a static weight loss prediction model to the results of a dynamic simulation model. The static model, presented in a report by the U.S. Department of Agriculture (USDA) (Smith et al., 2010), estimated the impact of a 20 percent tax to be an overall decrease in energy intake of about 40 kcal per day and a 1.8 kg body weight loss per year. Extrapolated over 5 years, this would result in almost 10 kg body weight loss. In comparison, the dynamic simulation model estimated that the overall impact of a decrease in energy intake as a result of the tax increase would reduce average body weight by about 1.8 kg over 5 years and about 1 kg of weight loss after 1 year (Hall et al., 2011). This result is in stark contrast to the estimates of the USDA analysis and highlights the importance of inter-individual variability of weight loss (caused by the same intervention), uncertainty of the assumptions made about energy expenditure, and the trajectory of human weight change. This example in the context of obesity prevention recognizes the importance of a careful consideration of the interactive and dynamic relationships among all factors and variables that influence the outcomes of interest over time.
Another case example outlines how a systems approach to obesity prevention evaluation may be used in a larger context to support decision making and dissemination of successful interventions. Figure 9-4 illustrates how a complex obesity systems map has been used to inform action (e.g., planning, decision making, and implementation) around obesity prevention. The case example starts with the explicit recognition that the workplace may be regarded as a complex adaptive system and that addressing obesity, considered complex in its own right, should be considered from a systems perspective. The Foresight obesity system causal map, created by the UK government (Butland et al., 2007), is used as the starting point to identify the best choice(s) for addressing obesity at the workplace. The workplace is mentioned in the context of physical activity (energy expenditure) in two of the seven main thematic clusters—the individual physical activity and physical activity environment clusters. More specifically, these clusters indicate the issues of overall level of occupational activity and dominance of sedentary employment. Hence, two places that can provide leverage to address obesity prevention successfully include the potential to increase person-level activity during the workday and to change the work environment by reducing the sedentary nature of the work tasks in which employees are engaged.
FIGURE 9-4 Case example of an obesity system casual map applied to a workplace setting to identify intervention opportunities Figure 9-4.eps broadside and evaluate energy balance (physical activity) interventions
SOURCE: Vandenbroeck et al., 2007. Foresight: Tackling Obesities: Future Choices—Project Report, 2nd Edition. Final Project Report. The Government Office for Science: London.
As a result of this evaluation of the Foresight obesity maps, program planners decided to invest resources in programmatic solutions. They designed and tested intervention concepts in a pilot project (Pronk et al., 2012). In the context of a multi-component, comprehensive health and well-being program, sit-stand devices were deployed in the work stations of employees with sedentary job types (call-center employees). Employees were engaged in a participatory manner, a supportive environment was established to optimize participation, and minimally invasive measurement methods were deployed to monitor performance. The results of the pilot study proved meaningful from the perspective of both the employees and management, and, as a result, the solution was deployed across all areas of the company that housed sedentary jobs (Koster, 2012).
This chapter presents a systems approach to evaluate obesity prevention efforts in context and considers it from an end user’s perspective by asking the question, “what is the best choice?” when taking
action. It describes what a systems approach entails, what are its key components, how opportunities for action may be identified, where readers may find other discussions of systems applied to obesity, what evaluation tools and methods are available, and what benefits a systems approach may provide to evaluation efforts in this area. Finally, the chapter considers how systems may be applied to the Committee’s proposed evaluation framework.
A systems approach may be useful to describe interrelationships among the many variables and elements that are involved in obesity and prevention efforts. It can help policy makers and implementers to identify the most important leverage points to achieve progress to determine “what is the best choice?” regarding what action to take and how best to invest resources.
Taking a systems approach to evaluating the APOP strategies will enable researchers and decision makers to think about evaluation efforts in new ways—identifying the types of data to collect and the methods for analyzing the data, identifying core indicators and measures that leverage networks and feedback loops among different sectors of society, identifying leaders, and engaging stakeholders in all aspects of evaluation. The result of these activities will improve the capacity and infrastructure of the collective evaluation system for obesity. And because obesity itself is an integral part of the larger systems of public and population health, it may be viewed as a major element in the larger complex system of the health of populations in communities, across states, and across the nation.
Axelrod, R., and L. Tesfatsion. 2006. Appendix A. A guide for newcomers to agent-based modeling in the social sciences. In Handbook of computational economics, vol. 2: Agent-based computational economics, edited by L. Tesfatsion and K. L. Judd. Amsterdam, the Netherlands: North-Holland. Pp. 1647-1659.
Best, A., T. Greenhalgh, S. Lewis, J. E. Saul, S. Carroll, and J. Bitz. 2012. Large-system transformation in health care: A realist review. Milbank Quarterly 90(3):421-456.
Butland, B., S. Jebb, P. Kopelman, P. McPherson, S. Thomas, J. Mardell, and V. Parry. 2007. Foresight: Tackling obesities: Future choices–project report, 2nd edition. London, UK: Government Office for Science.
C3 Collaborating for Health. 2011. Beyond Foresight: Can systems thinking help us address complex problems like obesity and chronic disease? http://www.c3health.org/wp-content/uploads/2011/03/Diane-Finegood-seminar-FINAL-20110322.pdf (accessed January 30, 2013).
Christakis, N. A., and J. H. Fowler. 2007. The spread of obesity in a large social network over 32 years. New England Journal of Medicine 357(4):370-379.
Diez Roux, A. V. 2011. Complex systems thinking and current impasses in health disparities research. American Journal of Public Health 101(9):1627.
Epstein, J. M. 2006. Remarks on the foundations of agent-based generative social science. In Handbook of computational economics, vol. 2: Agent-based computational economics, edited by L. Tesfatsion and K. L. Judd. Amsterdam, the Netherlands: North-Holland. Pp. 1585-1604.
Finegood, D. T. 2011. The complex systems science of obesity. In The Oxford handbook of the social science of obesity, edited by J. Cawley. New York: Oxford University Press. Pp. 208-236.
Glass, T. A., and M. J. McAtee. 2006. Behavioral science at the crossroads in public health: Extending horizons, envisioning the future. Social Science and Medicine 62(7):1650-1671.
Hall, K. D., G. Sacks, D. Chandramohan, C. C. Chow, Y. C. Wang, S. L. Gortmaker, and B. A. Swinburn. 2011. Quantification of the effect of energy imbalance on bodyweight. Lancet 378(9793):826-837.
Hammond, R. A. 2009. Complex systems modeling for obesity research. Preventing Chronic Disease 6(3):A97. http://www.cdc.gov/pcd/issues/2009/jul/09_0017.htm (accessed June 6, 2013).
Hargreaves, M. 2010. Evaluating system change: A planning guide. Princeton, NJ: Mathematica Policy Research, Inc.
HealthPartners. 2012. YumPower school challenge. Summary of key findings. Bloomington, MN: HealthPartners.
Holland, J. H. 2000. Emergence: From chaos to order. Oxford, UK: Oxford University Press.
Holmes, B. J., D. T. Finegood, B. L. Riley, and A. Best. 2012. Systems thinking in dissemination and implementation research. In Dissemination and implementation research in health: Translating science to practice, edited by R. C. Brownson, G. A. Colditz, and E. K. Proctor. New York: Oxford University Press. Pp. 175-191.
Homer, J. B., and G. B. Hirsch. 2006. System dynamics modeling for public health: Background and opportunities. American Journal of Public Health 96(3):452-458.
Homer, J., B. Milstein, W. Dietz, D. Buchner, and E. Majestic. 2006. Obesity population dynamics: Exploring historical growth and plausible futures in the U.S. Paper presented at 24th International System Dynamics Conference, Nijmegen, Netherlands.
Hovmand, P. S., D. F. Andersen, E. Rouwette, G. P. Richardson, K. Rux, and A. Calhoun. 2012. Group model-building “scripts” as a collaborative planning tool. Systems Research and Behavioral Science 29(2):179-193.
Huang, T. T., A. Drewnowski, S. K. Kumanyika, and T. A. Glass. 2009. A systems-oriented multilevel framework for addressing obesity in the 21st century. Preventing Chronic Disease 6(3):A82. http://www.cdc.gov/pcd/issues/2009/jul/09_0013.htm (accessed June 7, 2013) IOM (Institute of Medicine). 2010. Bridging the evidence gap in obesity prevention: A framework to inform decision making. Washington, DC: The National Academies Press.
IOM. 2012a. Accelerating progress in obesity prevention: Solving the weight of the nation. Washington, DC: The National Academies Press.
IOM. 2012b. An integrated framework for assessing the value of community-based prevention. Washington, DC: The National Academies Press.
Isham, G. J., D. J. Zimmerman, D. A. Kindig, and G. W Hornseth. 2013. HealthPartners adopts community business model to deepen focus on nonclinical factors of health outcomes. Health Affairs 32(8):1446-1452.
Kane, M., and W. M. K. Trochim. 2007. Concept mapping for planning and evaluation. Thousand Oaks, CA: Sage Publication.
Koster, K. 2012. Sit-stand workstations incorporate movement into work days of sedentary employees. Employee Benefit News, December 1.
Kottke, T. E., N. P. Pronk, and G. J. Isham. 2012. The simple health system rules that create value. Preventing Chronic Disease 9:110179. http://dx.doi.org/10.5888/pcd9.110179 (accessed June 7, 2013).
Levy, D., P. Mabry, Y. Wang, S. Gortmaker, T. T. K. Huang, T. Marsh, M. Moodie, and B. Swinburn. 2011. Simulation models of obesity: A review of the literature and implications for research and policy. Obesity Reviews 12(5):378-394.
Mabry, P. L., S. E. Marcus, P. I. Clark, S. J. Leischow, and D. Mendez. 2010. Systems science: A revolution in public health policy research. American Journal of Public Health 100(7):1161-1163.
Malhi, L., O. Karanfil, T. Merth, M. Acheson, A. Palmer, and D. T. Finegood. 2009. Places to intervene to make complex food systems more healthy, green, fair, and affordable. Journal of Hunger and Environmental Nutrition 4(3-4):466-476.
Meadows, D. 1999. Leverage points: Places to intervene in a system. Hartland, VT: The Sustainability Institute.
Meadows, D. H. 2008. Thinking in systems: A primer, edited by D. Wright. White River Junction, VT: Chelsea Green Publishing Company.
Mercer, S. L., B. J. DeVinney, L. J. Fine, L. W. Green, and D. Dougherty. 2007. Study designs for effectiveness and translation research: Identifying trade-offs. American Journal of Preventive Medicine 33(2):139-154.
Milstein, B., and J. Homer. 2009. System dynamics simulation in support of obesity prevention decision-making. Paper read at Institute of Medicine Workshop on the Application of Systems Thinking to the Development and Use of Evidence in Obesity Prevention Decision-Making, March 16, 2009, Irvine, CA. http://www.iom.edu/~/media/Files/Activity%20Files/PublicHealth/ObesFramework/IOMIrvine16Mar09v52MilsteinHomer.ashx (accessed September 23, 2013).
Mitton, L., H. Sutherland, and M. Weeks, eds. 2000. Microsimulation modelling for policy analysis: Challenges and innovations. Cambridge, UK: University Press.
Nader, P. R., T. T. Huang, S. Gahagan, S. Kumanyika, R. A. Hammond, and K. K. Christoffel. 2012. Next steps in obesity prevention: Altering early life systems to support healthy parents, infants, and toddlers. Childhood Obesity 8(3):195-204.
NCI (National Cancer Institute). 2007. Greater than the sum: Systems thinking in tobacco control. Tobaco control monograph no. 18. NCI Tobacco Control Monograph Series. NIH Pub. No. 06-6085. Bethesda, MD: Department of Health and Human Services, National Institutes of Health, National Cancer Institute.
Plsek, P. E., and T. Greenhalgh. 2001. Complexity science: The challenge of complexity in health care. British Medical Journal 323(7313):625-628.
Pronk, N. P. 2009. Physical activity promotion in business and industry: Evidence, context, and recommendations for a national plan. Journal of Physical Activity and Health 6(Suppl 2):S220-S235.
Pronk, N. P., A. S. Katz, M. Lowry, and J. R. Payfer. 2012. Reducing occupational sitting time and improving worker health: The Take-a-Stand Project, 2011. Preventing Chronic Disease 9:E154. http://dx.doi.org/10.5888.pcd9.110323 (accessed June 7, 2013).
ReThink Health. 2012. Summary of the ReThink Health dynamics simulation model. http://rippelfoundation.org/docs/RTH-Dynamics-Model-Summary-v6.pdf (accessed November 16, 2012).
Smith, T. A., B.-H. Lin, and J.-Y. Lee. 2010. Taxing caloric sweetened beverages: Potential effects on beverage consumption, calorie intake, and obesity. Economic Research Report No. 100. Washington, DC: Economic Research Service, U.S. Department of Agriculture.
Sterman, J. D. 2000. Business dynamics. Systems thinking and modeling for a complex world. Boston, MA: McGraw-Hill.
Sterman, J. D. 2006. Learning from evidence in a complex world. American Journal of Public Health 96(3):505-514.
Ulrich, W. 2000. Reflective practice in the civil society: The contribution of critically systemic thinking. Reflective Practice 1(2):247-268.
Valente, T. W., K. Fujimoto, C. P. Chou, and D. Spruijt-Metz. 2009. Adolescent affiliations and adiposity: A social network analysis of friendships and obesity. Journal of Adolescent Health 45(2):202-204.
Vandenbroeck, P., J. Goossens, and M. Clemens. 2007. Foresight tackling obesities: Future choices—building the obesity system map. London, UK: Government Office for Science.
Williams, B, and I. Imam, eds. 2006. Systems concepts in evaluation: An expert anthology. Los Angeles: EdgePress, American Evaluation Association.
This page intentionally left blank.