8

Developing and Sustaining a Learning System to Support Military Family Readiness and Well-Being

In this chapter, the committee draws heavily from dissemination and implementation science and a learning system framework.¹ In Chapter 7, we showed why advancing military family well-being within a complex adaptive system, such as the Military Family Readiness System (MFRS), requires a comprehensive approach optimally informed by research and models from several convergent fields. The development of an integrated information infrastructure is needed (see Chapter 7, Figure 7-4) that can support the monitoring and delivery of data-driven programs, services, and resources to promote military family well-being and ultimately mission readiness.

Here we detail specific requirements to build a dynamic, sustainable MFRS that would lead to high quality in programs, services, and resources. Thus, this chapter presents a review of the evidence from research on translating and scaling up evidence-based and evidence-informed programs, services, and resources into larger systems, an adaptive process central to building and sustaining an effective MFRS that can be responsive to emerging and future challenges facing the U.S. Department of Defense (DoD). We also examine opportunities for utilizing advancements in big data analytics and mobile platforms/wearables to enhance the MFRS.

___________________

¹ This chapter draws partially on papers commissioned by the committee (Marmor, 2018; Nahum-Shani and Militello, 2018).

BACKGROUND ON FAMILY-BASED PROMOTION AND PREVENTION INTERVENTIONS

In developing, implementing, evaluating, and improving military family readiness policies, programs, services, and resources to promote well-being and prevent behavioral health problems, many of the challenges faced by the MFRS within DoD are similar to those found in civilian communities. These challenges are amplified by the limitations of existing research on military child and family resilience and well-being, as well as by a complex and dynamic landscape of military contexts, services, and policies. The fields of applied developmental science and prevention science can provide relevant guidance for developing policies, programs, services, resources, and practices that are guided by evidence. We examine the range of available evidence with a focus on building on previous knowledge, including decades of research on prevention science and child development, incorporating available and relevant theoretical models, observational studies, and experimental intervention design consistent with considerations of the best available evidence (as established in Chapter 1) (Deaton and Cartwright, 2018).

For example, family research in civilian populations has consistently demonstrated that couples’ relationship quality, parenting, parent-child relationship quality, and other family processes (e.g., co-parenting, family conflict) influence a range of social, emotional, and behavioral outcomes over life course development (IOM, 2000; NRC and IOM, 2009b; Teubert and Pinquart, 2010). A growing body of research has documented similar influences in military families (see Chapters 5 and 6 for a review). There have been several decades of intervention research demonstrating the effectiveness of family-centered interventions in supporting child, adult, and family well-being in civilian populations across a range of adversities. In this context, family-centered interventions are those that address family members’ well-being and target positive parent-child relationships, parenting practices, and other family processes (NRC and IOM, 2009a, 2009b; Siegenthaler et al., 2012).

Reviews of promotional efforts as well as universal, selective, and indicated preventive interventions show that evidence-based interventions can be effective in preventing and reducing substance use (Blitz et al., 2002; Lochman and van den Steenhoven, 2002; Spoth et al., 2008), violence and antisocial behavior (Wilson et al., 2001, 2003), and mental health problems (Durlak and Wells, 1997; Hoagwood et al., 2007), as well as in promoting positive youth development (Catalano et al, 2002; Eccles and Gootman, 2002). Indeed, findings from a meta-analytic review indicate that the results of these interventions are both statistically and practically significant, representing reductions of one-quarter to one-third in base rates in some cases (Wilson and Lipsey, 2007). There is also a growing body of evidence-based and evidence-informed practices, interventions, and programs (referred to

collectively as EBPs and EIPs as defined in Chapter 1) that demonstrate the positive impact of couples/relational preventive interventions across a range of health and mental health risks (Crepaz et al., 2015; Kardan-Souraki et al., 2016; Martire et al., 2010).

A review of EIPs and EBPs consistently identifies core elements and processes across a range of contexts. For interventions designed to improve social, emotional, and behavioral outcomes in children and families at risk, core elements often include issue-specific education and developmental guidance, individual and family-level skill development (e.g., emotional regulation, problem solving, communication) and positive parenting practices (IOM, 2000; NRC and IOM, 2009a; Spoth et al., 2002). Furthermore, family-centered programs that emphasize collective processes, resilience, and strengths have been found to be more engaging and culturally acceptable than those interventions focused on addressing individual problems in other contexts (Kumpfer et al., 2002; NRC and IOM, 2009a, 2009b).

BARRIERS TO TRANSLATING EVIDENCE INTO PRACTICE

Over the last 20 years, the National Academies of Sciences, Engineering, and Medicine have convened expert committees to review research and make recommendations of prevention interventions for children and families (for summaries, see Eccles and Gootman, 2002; NRC and IOM, 2009a, 2009b). These studies have usually examined interventions designed to address specific problem areas or risk factors (e.g., parental depression) and have consistently made recommendations both for expanded prevention research and for the wider practice of prevention interventions. As noted by Rotheram-Borus and colleagues (2014), these recommendations have led to the development of hundreds of evidence-based practices (EBPs) that have typically been designed to address a specific problem and then tested within a selected population, within a geographic region, and for a specific delivery setting, such as at home, in schools, or in a community setting. Funding agencies, organizations, and researchers have invested decades of research and financial resources into the development of practices, programs, services guidelines, and interventions demonstrated through rigorous research studies to affect individual-and family-level outcomes.

Although the benefits of using EBPs to support positive developmental and well-being outcomes in children and families with a range of risk factors are solidly grounded in empirical studies,² the translation of this evidence

___________________

² Using a standard validation model for establishing evidence for each problem, context, population, and platform, programs are expected to be tested in at least one randomized trial plus an effectiveness trial to be ready for large-scale diffusion—a model that can take almost two decades to come to fruition (Hawkins et al., 1992; Olds et al., 1988). Research trials of such programs may also include testing of adaptations of EBPs using individualized delivery platforms, such as internet-based or mobile-application delivery tools.

into practice has lagged far behind, as it has for other evidence-based interventions (Bumbarger and Perkins, 2008; Glasgow and Chambers, 2012; Kazdin and Blase, 2011; NRC and IOM, 2009b). Despite the availability of hundreds of evidence-based interventions—such as the Clearinghouse for Military Family Readiness’s Continuum of Evidence (Perkins et al., 2015), Blueprints for Violence Prevention,³ and the Promising Practices Network on Children, Families, and Communities⁴—few proposed EBPs are implemented and sustained in everyday community service settings.

Research trials of EBPs also include testing adaptations of them using individualized delivery platforms, such as internet-based or mobile-application delivery tools. Using a standard biomedical validation model for establishing evidence for each problem, context, population, and platform, programs are then expected to be tested in at least one randomized trial plus an effectiveness trial to be ready for large-scale diffusion—a model that has often taken almost two decades to come to fruition (Hawkins et al., 1992; Olds et al., 1988). Despite extensive investment in randomized trials to establish the benefits of these interventions on a range of child, youth, and adult outcomes, the dissemination of existing EBPs remains quite low within most civilian settings. This well-documented “translational gap” from research to practice poses multiple obstacles within civilian settings, and those problems are only amplified within the DoD context, given its highly diverse population (e.g., diverse by service branch, geography, and family constellation) and the highly dynamic context of military service and military family readiness related to wartime service demands, emerging types of warfare, and changes in policies (e.g., Beardslee et al., 2011, 2013; Dworkin et al., 2008), as well as in highly stressful situations (as described in Chapter 6).

A growing body of research has been examining the underlying assumptions that contribute to this translational gap for both clinical and preventive interventions. This research recognizes the challenges of selecting and implementing evidence-based practices that are relevant to the needs of specific populations across different settings, as well as the limitations of an overreliance on randomized controlled trials in establishing the necessary evidence to inform both internal and external validity in real-world settings (Deaton and Cartwright, 2018; Wike et al., 2014). The research consistently identifies a range of barriers to successful implementation (Perkins et al., 2015), including the limitations of existing EBPs for emerging issues, lack of cultural relevance of the EBP for specific populations, and limitations in available resources required for rigorous implementation, including training, monitoring, infrastructure, and technical support (for a review, see Rotheram-Borus et al., 2012).

___________________

³ For more information, see https://www.blueprintsprograms.org.

⁴ For more information, see http://www.promisingpractices.net/programs.asp.

These challenges may emerge early, as soon as an organization or system faces the selection process for determining the best program for a specific context or problem. Criteria within some EBP registries have prioritized randomized controlled efficacy trials, which focus on internal validity, that is, having minimal chance of confounding variables in the study. Such trials do not commonly include evidence about a program’s cultural relevance, adaptability, scalability, or sustainability relevant to external validity, that is, how well the results can be generalized across settings or populations. It is also the case that some registries are designed more to assist practitioners in making informed decisions in selecting a program based on their needs, situational factors, and available resources (Karre et al., 2017). An example of the challenges that face local providers and system leaders when they need to identify an EBP is evident from civilian child intervention research, which found that for a large community clinical sample of children, 86 percent of the children were not included in the 435 randomized clinical trials of EBPs when matched for age, gender, and ethnicity (Chorpita et al., 2011).

Others have noted that research on most identified EBPs largely lacks information about or inclusion of community participation or practice in the development and testing of the interventions (Weisz et al., 2006), resulting in a misalignment between the interventions and the realities of community systems. Such misalignments may be especially likely to emerge when providers’ and families’ voices and experiences, as well as larger system contexts, are not incorporated into the development, measurement, adaptation, and implementation of the EBP and not included as part of the criteria for inclusion into EBP registries (Burkhardt et al., 2015; Means et al., 2015; Santucci et al., 2015; Weisz et al., 2015). As Chambers and Norton (2016) note:

However, these mismatches are often attributed to lack of organizational readiness for disseminating an intervention rather than a potential or actual misalignment between the EBP and the setting (Weisz et al., 2013).

Some researchers have proposed a paradigm shift in how evidence-based interventions are applied, expanded, and disseminated. For example, Chorpita and colleagues (2007) developed and evaluated a so-called “common elements framework” to identify, coordinate, and monitor the delivery of components from an established EBP. This framework focuses on professional training and development and supports a flexible approach to

evidence-informed delivery across different settings and populations. Many researchers have advocated for an emphasis on testing core principles and elements that can be flexibly implemented rather than focusing on developing and testing new individual programs (Mohr et al., 2015), as well as an emphasis on identifying intervention “kernels” as fundamental units that underlie effective interventions (Embry and Biglan, 2008). A common elements framework allows researchers and providers first to apply empirical evidence about treatment efficacy and effectiveness, and then to incorporate local evidence and outcomes regarding individual progress through the delivery process (Becker et al., 2013; Chorpita and Daleiden, 2009; Chorpita et al., 2005; Morgan et al., 2018). This may be even more relevant in military communities, where implementations are required to address rapidly emerging requirements in wartime.

REMEDIATING THE BRIDGE FROM EVIDENCE TO PRACTICE

More than a decade ago, the field of dissemination and implementation science began to focus on understanding and improving the evidence-to-practice gap. This new approach arose primarily from failures in the adoption, implementation, and sustainability of evidence-based practices (Kelly, 2012). Dissemination and implementation science (also referred to as “implementation research”) can be defined as “a multi-disciplinary set of theories, methods and evidence aimed at improving the processes of translation from research evidence to everyday practices across a wide variety of human service and policy contexts” (Kelly, 2013, p. 1). This science is devoted to rigorously studying research-to-practice gaps to identify effective ways to improve the adaptation, adoption, implementation, and sustainment of evidence-informed and evidence-based practices in routine delivery settings. It is also committed to fostering partnerships with practice organizations to accelerate the transition of interventions from research- to practice-focused settings.

A paper commissioned by the committee (Chambers and Norton, 2018, p. 5) has this to say about implementation science:

Dissemination and implementation science includes all the components of this process, including the decision to adopt an intervention within a system, its development and engagement on it with stakeholders, workforce

___________________

⁵ The authors cite work by McKibbon et al. (2010) to support this point.

skills development (i.e., training, coaching/consultation, and workforce well-being), quality monitoring (i.e., measurement selection, data collection, and quality monitoring and reporting), and administrative management. It can inform behavioral health and social service research and service delivery to guide the processes that can bridge the research-to-practice gap and lead to greater integration of EBP and evidence-informed practice (EIP) into routine service settings (Atkins et al., 2016; Durlak, 2013). While the field has advanced in recent years, Chambers and Norton (2016) assert that it has been limited by current models in which

Within this traditional sequence, Chambers and Norton (2016) have identified a number of assumptions that may contribute to challenges in scaling EBPs, as follows. First, they include the assumption that once established, the evidence base for an intervention is stable. In fact, many of the established national registries are well populated by EBPs tested decades ago with relatively small and, in many cases, nonrepresentative convenience samples recruited in community and clinical contexts that have continuously evolved. The assumption that these established EBPs will remain efficacious when implemented at scale with diverse populations and in new contexts is also problematic. The implications of the lag between the research testing cycle and the application are highlighted by trials involving new technology platforms or mobile tools, as the tested delivery platforms may become outdated even within the duration of a single efficacy trial (Kumar et al., 2013).

A second assumption contributing to implementation challenges is that deviation from the established delivery process or manual implementation is considered an erosion of program fidelity inherently leading to reduced impact. This assumption overlooks the potential of “positive drift” that may occur as the intervention is adapted within new settings and populations. Finally, the assumption that dissemination and implementation “come after everything else” may result in a failure to develop interventions that leverage existing resources and local knowledge to improve the relevance and fit of the intervention to the context (Chambers and Norton, 2016).

As mentioned and explicated in this report, the well-documented “translational gap” from research to practice not only poses multiple obstacles within civilian settings; those problems are amplified within DoD, given the highly diverse population it embraces (e.g., service branch, geography, family constellation), the dynamic context of military service and military family readiness related to deployments, and other demands and changes in policies (Beardslee et al., 2011, 2013; Dworkin et al., 2008). The remainder of this chapter documents the committee’s suggested approach to addressing this issue.

ONGOING ADAPTATION FRAMEWORK FOR A COMPLEX MILITARY FAMILY READINESS SYSTEM

Applying both the population-level and ecological models presented earlier in this report to examine military family well-being, the committee extends these to inform the continuum of military family readiness services that would be responsive to the complex and emergent needs of a complex adaptive system (see Figure 7-1 in Chapter 7). Using an ecological model to inform implementation enables providers, installation services, and leaders to comprehensively address the various levels and contexts influencing military families. As discussed in Chapter 7, the continuum of coordinated support within the Military Family Readiness System builds on local capacities, strengths, and resources and incorporates both DoD-level and local knowledge within the selection, adaptation, adoption, and implementation of support services.

As Atkins (2016, p. 215) argues, “This paradigm shift for dissemination and implementation science, away from an overemphasis on promoting program adoption, calls for fitting interventions within settings that matter most to . . . . healthy development, and utilizing and strengthening available community resources.” Developing a comprehensive approach to support implementation requires the MFRS to utilize embedded assessment and monitoring in the implementation of programming and to develop an integrated information infrastructure (refer to Figure 7-4 for detailed components) that supports continuous quality improvement analogous to a learning health system, characterized here as a “learning MFRS.”

Systematic, planned adaptation, often considered necessary to support the effective implementation of an EBP, can occur at multiple phases during the lifecycle of the implementation process. Note that rapid implementation, while sometimes necessary, should be avoided. At the very least, clear systematic review and data is required to assess the implementation and identify areas of needed improvement. Adaptation can be defined as the degree to which an EBP is modified by a user during adoption and implementation to suit the needs of the setting or to improve the fit to local

conditions (Rabin et al., 2008; Rogers, 2010). Indeed, Chambers and colleagues (2013) have proposed that sustainability should be reconceived as the ongoing adaptation of an intervention that is supported by continuous learning and problem solving, with a focus “on fit between interventions and multi-level contexts.”

The Dynamic Sustainability Framework (see Figure 8-1) illustrates how the adaptation of interventions may occur over time. It also conveys the role of continuous monitoring in supporting the integration and sustainability of interventions as they are adapted to the ever-changing context in which they are delivered, including changes occurring in the delivery setting, the target population, the evidence base, the political context, and other key variables that are known to occur over time (Chambers et al., 2013). To this end,

Planned and purposeful intervention adaptation can occur during all phases of the implementation process. Intervention adaptation can be informed by feasibility and acceptability testing, with potential end-users being asked to deliver the intervention or to define the outcomes related to well-being. Both qualitative (e.g., focus groups, interviews) and quantitative (e.g., surveys) data can be used to guide the planned adaptation of interventions, recognizing that too many significant or substantive changes to an intervention may be less desirable—and to some extent could resemble a new intervention that should then undergo its own rigorous testing before being branded as evidence-based. How much adaptation occurs before an existing EBP resembles a different intervention and should undergo separate evaluation is a significant yet unanswered question in the field. Optimally, implementers are given guidance on exercising flexibility in areas of program delivery that are not hypothesized to be directly responsible for program outcomes, while adhering to program components that are core to the EBP’s theory of change. In other words, there can be “adaptation with fidelity” (Domitrovch et al., 2012; Weist and Murray, 2008).

There are many types of adaptations that can (or should) occur to an EBP. Effective interventions proposed for implementation should include parameters for fidelity monitoring that anticipate adaptation, that is, by naming which elements can and should be modified for context and culture and which are core, essential elements that cannot be modified (Bumbarger and Perkins, 2008). Stirman and colleagues (2013) proposed a framework and coding system for modifications and adaptations to EBP based on a systematic review of the literature. Their intervention adaptations were classified into five broad categories and associated subcategories (Stirman et al., 2013, Figure 2, p. 6). Their five main categories reflect five key questions about the adaptation process:

1. By whom are modifications made (e.g., individual, team, researcher)?
2. What is modified (e.g., content, context, training and evaluation)?
3. At what level of delivery (for whom/what) are modifications made (e.g., group level, hospital level, network level)?
4. To what are context modifications made (e.g., to format, to setting, to population)?
5. What is the nature of the content modification (e.g., tailoring, substituting, reordering)?

Building on this taxonomy, Chambers and Norton (2016) expanded the types of intervention adaptations as part of what they call the “Adaptome,” a proposed set of approaches, processes, and infrastructure needed to advance the science of intervention adaptation. Sources of intervention adaptations (and example questions) include service setting (e.g., Who

delivers the intervention? How does the proposed intervention fit with other interventions?); target audience (e.g., literacy, comorbid conditions, age-appropriateness), mode of delivery (e.g., dose of core components, number of sessions), culture (e.g., cultural sensitivity, use of imagery), and core components (e.g., mechanisms of action, core components identified through testing). See Figure 8-2.

The “Adaptome” approach to implementation provides a methodology that can support the integration of evidence—including both traditional standards of evidence and phases of EIP and EBP development and validation—while also addressing local needs. The latter aspect is important because distinctive local needs sometimes lead local providers to design and deliver their own programs ahead of evidence for effectiveness (Hallett et al., 2007). Using the Adaptome approach, and supplementing this with existing literature on the science of intervention adaptation, in the following pages we present several examples of ways in which existing EBPs can be adapted, monitored, and refined over time to meet the needs of military family resilience and well-being in the military health care and community settings in which they could be delivered. This methodology supports the integration of evidence-informed and evidence-based practice with “practice-based evidence” within a Dynamic Sustainability Framework (DSF) (Chambers and Norton, 2016).

Adaptation to Service Settings

A well designed adaptation to a service setting will seek to better align the original evidence-based intervention with the original setting in which it is delivered. This may include changes to who delivers the intervention to military families (including active service members, children, parents, couples, caregivers, and other family members), assessment of the fit between the proposed EIP/EBP and interventions that are already being delivered in the setting that are consistent with the organizational mission but complement other available interventions, and resources and capacity to deliver the intervention within existing systems, such as schools, early childhood programs, and primary care and community centers. Resources and capacity, in turn, include personnel, funding, organizational culture, absorptive capacity, time constraints, and competing demands.

Task-shifting is one approach commonly leveraged in low-resource settings, whereby systems with a dearth of professionally trained providers rely instead on community health workers, lay personnel, peers, or volunteers to deliver prevention programs. In this situation, it may be decided that the alternative implementers of the new prevention program need additional training and ongoing coaching or supervision, but are nonetheless able to receive that while maintaining or even improving patient-level outcomes. Regular program monitoring is required to identify any additional needs of the implementers within the system and how those needs may change over time. Such monitoring may involve tracking customer outcomes, feedback from staff, input from implementers, and practical measures of ongoing intervention adaptation. The results of the monitoring may have implications for the training of new implementers as others may transition to other responsibilities.

Target Audience Adaptations

Target audience adaptations involve adjustments to create a better fit between the intervention and the proposed target population. These adaptations may include changing the format and language used in the intervention (e.g., materials, workbooks, flyers) to better match the literacy levels of the target population. They could include use of instructional examples that are more relevant to the target population, such as having pictures included in intervention materials, or having names and locations of delivery sites, or localizing the available resources. To better achieve such matches, rapid-cycle usability testing⁶ can be done on an

___________________

⁶ A model of using small tests to accelerate improvement. See http://www.ihi.org/about/Pages/innovationscontributions.aspx.

individual basis or in a group setting. Interviews or focus groups could be employed to identify needed adaptations, such as by asking members of the target population to review intervention materials and identify aspects that they feel are relevant to them, aspects that are irrelevant, and aspects that should be modified to fit their needs. This approach could be leveraged over time to make improvements to intervention materials delivered within the same setting, or in subsequent iterations of the intervention as it is delivered to members of the target population in different geographic regions, having different literacy rates, or within different age ranges.

Delivery Adaptations

This type of adaptation focuses on changes that may be needed concerning how the intervention is delivered in terms of number of sessions (e.g., 5 vs. 10 sessions), length of sessions (e.g., 60 minutes vs. 3 hours), frequency of sessions (e.g., 4 weeks vs. 10 weeks), and mode of delivery (e.g., online vs. in-person; individual vs. group-based; clinic vs. telehealth; text messages vs. phone; active vs. passive telephone outreach).

For example, consider an evidence-based intervention that was originally developed in a group-based setting for 2 hours a week for 12 weeks. Although retention rates were high in the original study, this may be due in part to the participants’ having received a generous incentive to complete the intervention. In fact, given the military’s high operational tempo, as discussed in Chapter 2, military service members and their families may be unlikely to have time to attend all sessions along with their other responsibilities, interests, and demands. This is a common barrier in implementation and receptivity among target populations: Interventions are often designed without consideration of their viability outside the context of a highly controlled environment in a research trial, and subsequently they prove to be of no interest to the target population. This is particularly true for interventions that require a lot of time or frequent off-site visits or that rely on the participation of other group members to be effective. As an alternative, one might explore the possibility of delivering part of an intervention through private, group-based portals and reducing the number and frequency of sessions. If child-care duties are barriers to in-person participation, one may consider using barrier reduction components (Morgan et al., 2018), such as delivering the intervention in a school-based or daycare setting, to reduce impediments to participation.

Suggestions for ways to increase participation and interest in an intervention while maintaining sufficient delivery and dosage should be informed by input from the target population (as noted in Chapter 7 under “Community Engagement and Participatory Partnerships”) in advance of intervention adaptations. Again, rapid-cycle evaluations can be used to inform

iterations to the intervention over time as well as across geographic regions or other characteristics that may suggest the need for altered adaptations to the delivery (i.e., in dose, frequency, format, or length) as it is scaled up to other areas.

Cultural Adaptations

Cultural adaptations are essential to consider as an ongoing part of the implementation process. Often, cultural adaptations require important yet relatively subtle changes to the content of an intervention that are critical to its perceived acceptability, relevance, and credibility to the target population. Cultural adaptations go beyond minor changes to the names, locations, and lists of relevant resources and services. They include changes to culture-specific nomenclature used in intervention materials, which may vary by geographic region or by subpopulation, for example by urban versus rural; in the use of “y’all” versus “you all” for Southern versus non-Southern target populations; or by African American young adults versus African American middle-aged adults. They also include changes to culture-specific pictures (including age, gender, race/ethnicity, appearance), examples, or scenarios used in intervention materials or content, and changes that may need to occur such that the adapted intervention is consistent with the general beliefs of the target population (e.g., religiosity, stigma, social and personal attitudes, medical mistrust). Additional guidance for what types of cultural adaptations should be considered during the adaptation process are available in the literature (Bernal and Domenech Rodríguez, 2012; Cabassa and Baumann, 2013).

Core Component Adaptations

Finally, core components (also defined as core elements) are conceptualized as the “active ingredients” of an EBP, without which one would not see the intended impact on changes in behaviors and well-being outcomes among the target population (Chorpita, 2007; Embry and Biglan, 2008). The research designs to identify the exact core components of an intervention (unlike peripheral components that can be significantly changed or deleted without affecting was outcomes) are logistically challenging, costly, and impractical. Thus, alternative modes are recommended for identifying core intervention components that should be neither removed nor significantly altered during the adaptation and implementation process. This may be done, for example, through conversations with the original intervention developers and by relying on those theoretical constructs that have been demonstrated to be required for effective behavior change (i.e., skills-training and education versus education only).

An Integrated Information Infrastructure to Inform Adaptation

As noted in the Dynamic Sustainability Framework (2013) and further articulated by the Adaptome, multiple data sources and types of data are needed to inform and guide intervention adaptations for greater impact at the population level and to achieve higher quality in promotional and preventive practices for the MFRS. Within the military service context, several existing datasets could be leveraged to guide intervention adaptations, and others could be developed to inform the adaptation process consistent with learning from practice-based evidence that has been generated through local delivery in community settings such as schools and primary care. For example, better dissemination of the findings from the Status of Forces Survey of Reserve Component Members and others administered by the Office of People Analytics could be used to systematically identify the needs of different subgroups of reservists and the context in which those needs can be met. Responses could help prioritize what interventions are most needed by different types of personnel in the community (e.g., social network capacity building, alcohol prevention, spousal communication, reintegration) and the preferred delivery format (e.g., individual, group, phone, text, online). This information could be used to help guide the selection of interventions for their target populations, such as existing interventions developed for civilian populations found within the Clearinghouse for Military Family Readiness⁷ (Perkins et al., 2015), an online intervention compendium.⁸

An infrastructure to support quality monitoring and integrated information is required to ensure quality service delivery. It is also required to capture local innovations by identifying emerging practices and adaptations responsive to the voices of military-connected communities and families. Additional items could be added to the Status of Forces Survey of Reserve Component Members to help monitor beliefs, attitudes, health literacy, and other characteristics that can help guide the selection of and inform initial adaptations to evidence-based interventions across various sources of intervention adaptation. This may be an efficient way to help select and initially adapt an intervention. It could be bolstered by select follow-up local surveys or group-based feedback on specific adaptations that may be needed to further enhance the fit between the intervention and the overall setting.

Additional surveys and studies—such as the Millennium Cohort Study, Military Family Life Project, Deployment Life Study, Veterans Metric Initiative, and organizational climate and community assessment surveys—can help track trends over time and indicate what additional adaptations may be needed for certain types of service members and their families. They can highlight what priorities, conditions, and contexts change over time for

___________________

⁷ See https://militaryfamilies.psu.edu.

⁸ See https://militaryfamilies.psu.edu/wp-content/uploads/2018/08/A-Tool-for-Assessing-Fitand-Feasibility-9-9-17.pdf.

different subgroups. Administrative and reporting databases can help inform intervention designers and implementers of the needs of target populations, adoption and use of evidence-informed and evidence-based interventions (e.g., guidelines delivered in clinical care), suggested adaptations, and monitoring of adaptations over time. For example, datasets in the Defense Manpower Data Center could be triangulated to identify target populations in greatest need of additional or more intense mental health treatments, based on prevalence rates of posttraumatic stress disorder, suicidality, or depression. Additional civilian datasets can be augmented with military identifiers to track military child and family needs, align policy, and monitor interventions.⁹

___________________

⁹ For more information, see https://www.militarychild.org/resources/policies-initiatives?topic=36.

Two promotional and prevention models that have used continuous quality data monitoring in civilian communities have demonstrated their success in improving youth development outcomes: the PROSPER (PROmoting School-community-university Partnerships to Enhance Resilience) model (Spoth et al., 2004, 2011) and the Communities That Care system (Hawkins et al., 1992, 2002). Each of these community-level interventions includes a data infrastructure for monitoring and mapping, as well as an infrastructure for support of innovation, analytics, training, service delivery, and technical support (coaching) that promote ongoing learning (Chilenski et al., 2016). These components are consistent with the integrated information infrastructure presented in Chapter 7 (refer to Figure 7-4). Box 8-1 provides an example of how to apply such a system-level approach.

Applying Implementation Science to the MFRS

Implementation involves a deliberate set of change strategies to integrate a program, intervention, or practice across contexts and settings (Damschroder and Hagedorn, 2011; Fixsen et al., 2005). Kelly (2013) has defined discrete implementation strategies as involving a single process or action, such as establishing reminders or educational meetings. By contrast, multifaceted implementation strategies include those that use two or more discrete strategies—such as training and technical assistance, organizational change, and external facilitation—to facilitate the adoption and integration of an evidence-based intervention into routine-care settings (Powell et al., 2012). To date, more than 60 implementation strategies have been identified from literature reviews and expert input. These strategies include planning strategies, educational strategies, financial strategies, restructuring strategies, quality management strategies, and policy context strategies (Powell et al., 2012). Generally, a combination of strategies (rather than a single strategy) is needed to effectively move an evidence-based intervention into routine practice.

To monitor an intervention or policy implementation as part of a complex adaptive system such as MFRS requires that one continually assess the implementation itself. As noted in Chapter 7, the implementation outcomes to be assessed include the program’s acceptability, feasibility, appropriateness, adoption, cost, fidelity, penetration, and sustainability, as described by Proctor and colleagues (2011). Outcomes can be assessed across the phases of implementation; for example, acceptability, feasibility, and appropriateness may be best assessed at the planning phase for implementation within a specific context, whereas fidelity and penetration may be best suited for assessment during the implementation and maintenance phase. Sustainability is often assessed approximately six months to two years after the funding for the initial implementation of an evidence-based intervention has ceased (Scheirer and Dearing, 2011), so that what is assessed is essentially the extent to which the intervention can be integrated into routine delivery settings or institutionalized as standard practice. As noted in Chapter 7 and as outlined in the measurement section, qualitative and quantitative approaches can be used to assess implementation outcomes.

Dissemination and implementation science has supported the delivery of a tiered-population approach to promotion and prevention, consistent with the Spectrum of Coordinated Support presented in Chapter 7 (refer to Figure 7-1). Screening, promotion, and prevention practices can be integrated into community-, school- and family-care settings so that they are customized to suit family needs or to suit the timing, dose, provider, or platform needs. One of the most well developed and researched examples of

this type of population-level approach within civilian settings is the Triple P (Positive Parenting Program), designed as a comprehensive strategy to promote skilled parenting and to prevent parenting problems early, delivering on-demand services that are standardized to include evidence-based components and tiers based on higher levels of need (Turner and Sanders, 2006).

Another example of a population-level approach, described previously in Chapter 7, is the New Orientation to Reduce Threats to Health from Secretive Problems That Affect Readiness (NORTH STAR) Program, designed to prevent substance use problems, family maltreatment, and suicide. Designed to be integrated into an existing delivery system within active-duty Air Force installations, NORTH STAR is an integrated delivery system involving commanders and providers partnered with Air Force community action and information boards at each of the 10 major commands (Heyman et al., 2011). The partners at each command selected the programs that matched their specific risk and protective factor profiles, using a guide on evidence-based programs that called for rating the programs according to evaluation outcomes and targeted risk and protective factors. The guide also includes training, implementation, and survey evaluation protocols. The use of a framework, delivery system, and guide to select prevention programs that fit a particular base’s risk and protective factor profile is based on extensive community-based prevention research strategies that have been evaluated in civilian populations (Heyman and Smith Slep, 2001; Pentz, 2003; Riggs et al., 2009).

Sustainability

Sustaining effective programs and services is one of the critical goals of the MFRS. Sustainability definitions in the literature vary, but most include the continuation of the implementation of effective programs or services with the intent of maintaining positive outcomes in the served communities (Johnson et al., 2004; Scheirer, 2005; Scheirer and Dearing, 2011). The effective functioning of a complex adaptive support system, like MFRS, is designed: (1) to facilitate the high-quality implementation and ongoing management and improvement of effective programs and services; and (2) to provide capacity for the programs and services to overcome potentially disorganizing changes, such as staff turnover or shifts in funding availability (Gruen et al., 2008; Scheirer, 2005).

Multiple factors are linked to increased sustainability; however, for the complex adaptive system infrastructure two factors seem essential: a continuous quality improvement process and ongoing, proactive technical assistance (e.g., implementation coaching) (Bumbarger and Perkins, 2008; Chilenski et al., 2015, 2016; Rhoades et al., 2012; Tibbits et al., 2010). For the complex adaptive system, a continuous quality improvement process

provides actionable data linked to various outcomes, such as implementation, service, and customer/participant outcomes (Procter et al., 2011). These data are employed by the system to ensure that the proactive technical assistance addresses any unwanted “reactionary drift” from protocols within programs and services, compared to planned and tested adaptations, as well as guiding specific adaptations or innovations. Moreover, identifying what programs or services need to be sustained or what components of those programs and services should be maintained (e.g., partial sustainability) demands data garnered from a continuous quality improvement (CQI) process.

To be strategic, effective, and efficient, a complex adaptive support system, like MFRS, demands a systematic formal process for determining (1) how to initiate a new program or service, (2) how to sustain an existing program or service, and (3) how and when to sunset or decommission a program or service. The evidence of effectiveness of programs and services, through rigorous evaluation, to meet real-world needs provides clear guidance as to whether those efforts should be sustained or discontinued. As newly identified family needs emerge, the system is required to engage in a service-design or program-identification process that effectively addresses those needs. Box 8-2 describes an example of how a program was sunsetted.

ADAPTATION AND CONTINUOUS QUALITY MONITORING USING A LEARNING SYSTEM FRAMEWORK

As noted earlier, an integrated approach to implementation and adaptation requires a spectrum of coordinated support and an integrated information infrastructure that supports the mapping of emerging needs as well as continuous quality monitoring (also refer to Figure 7-1 in Chapter 7).

Supporting a Learning Infrastructure

A useful model for achieving greater accountability, agility, and family/client-centered outcomes in the complex adaptive MFRS may be drawn from the “learning health system” framework, defined here as a learning infrastructure. The use of a learning infrastructure within a complex adaptive system helps ensure that implementation strategies are used, enhances interpretability of research findings, and bolsters the use of critical implementation strategies (Ferlie and Dopson, 2006; Pawson et al., 2005; Proctor et al., 2013).

The Institute of Medicine (IOM, now the National Academy of Medicine) defines a learning infrastructure as a structure in which “science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with evidence-informed and/or promising practices seam-

lessly embedded in the delivery process and new knowledge captured as an integral by-product of the delivery experience”¹⁰ (IOM, 2011). The IOM has organized a continuous learning infrastructure into four foundational elements: (1) science and informatics, with real-time access to knowledge and digital capture of the service experience; (2) partnerships between providers, families, and data scientists with engaged and empowered families; (3) incentives that are aligned for value, but with full transparency; and (4) a leadership-instilled culture of continuous learning with supportive system competencies (IOM, 2013).

Big Data

Within a learning infrastructure, big data and predictive analytics have significant potential to promote military family readiness and well-being by

___________________

¹⁰ See https://nam.edu/programs/value-science-driven-health-care/learning-health-system-series.

supporting forward-looking data-driven decisions and policies. The service delivery, provider education and training, military family and community partnerships, civilian sector partnerships, research and development, and performance improvement strategies included in the MFRS lend themselves to the development of predictive analytics. Large amounts of service delivery data and sociodemographic and socioeconomic data can be merged and harmonized in order to systematize practitioner-supported practices and to respond to emerging needs. If combined with the delivery of EIP/EBP, broad military family participation would provide extensive data points from both quantitative and qualitative data sources and would facilitate optimal service delivery, maximizing military family readiness and well-being.

Other systems using a learning infrastructure have already had a successful track record using big data to improve outcomes (Dabek and Caban, 2015; Raghupathi and Raghupathi, 2014). These could serve as models for MFRS. For instance, within the health care field, systems that harvest data across multiple service delivery systems, registries, and payers have allowed providers with real-time tools to improve the quality and value of care, allowing for a renewed focus on preventative practices (Coffron and Opelka, 2015). Private-sector initiatives that trawl through patient data to provide better care for low-income Medicaid beneficiaries (Farr, 2018) account for socioeconomic and sociodemographic differences; accounting for such differences is similarly important in military communities and for military providers, considering the diversity of trainees, active-duty members, and their families. Ultimately, the big data that are generated by military service members, their families, and networks form the backbone of the learning infrastructure within an optimized complex adaptive system for military families.

Continuous Quality Improvement

Continuous quality improvement (CQI) is a necessary component of the learning infrastructure in a complex adaptive system. CQI enables the system to be data driven with an aim of cultivating adaptations and adjustments within services, programs, and resources. Standard CQI protocol involves the systematic and constant collection of data (Langley et al., 2009) whose content should be multilevel, spanning administration, implementation, service, and customers. Being data driven within a complex adaptive system also means that all stakeholders make use of the data in their daily decision making as they implement and adapt policies, services, programs, resources, and practices. The data being collected on outcomes (i.e., implementation, service and customer/participant outcomes) provide regular feedback about adaptations and adjustments in terms of feasibility, outcomes, and impact. Client-level assessment tools can be incorporated

into the routine delivery of programs for military families, with assessment occurring at program entry, program exit, and ongoing as appropriate. CQI can then be embedded in the implementation infrastructure of a complex adaptive system, which uses the continuous data to provide a practice-learning hub. The hub can use active communication feedback loops to advance the innovations among the various stakeholders (e.g., practitioners and policy analysts). To accomplish this, the MFRS needs to build a strong information systems infrastructure that can support the collection, management, storage, and analysis of these data. This could provide important evidence regarding the implementation, service, and participant outcomes.

The goal of CQI is to provide actionable data that enable the complex adaptive system to address various outcomes, such as implementation, service, and customer outcomes, through specific identifiable adaptations or innovations (Procter, 2011). Thus, the CQI process is constantly testing specific identifiable adaptations with an emphasis on substantive change, such that the “art of improvement is combined with the science of improvement” (Langley et al., 2009, p. 6). In short, CQI is embedded within learning infrastructure that involves strategic, action-planning models to develop, manage, improve, and evaluate interventions (i.e., policies, programs, services, resources, and practices) (Davidoff et al., 2008).

The Dynamic Sustainability Framework provides a strong conceptual description for system-level CQI that recognizes and accommodates the constancy of change in the use of interventions over time, the characteristics of service settings, and the broader system contexts (whether military or civilian) that determine how services are delivered and by whom (Chambers et al., 2013). A CQI process sensitive to change is critical in the military context. For example, military members and their families are highly mobile, and every summer large numbers of them receive permanent change of station (PCS) orders, requiring them to move to other jobs and/or installations. Similarly, changes to staff, new leadership, and changing military priorities can all disrupt the system’s efforts. Informed by CQI data to continually improve services, programs, and resources, a complex adaptive system like MFRS acknowledges these constant changes and provides adaptive, dynamic, and fluid strategies to support the MFRS.

Centering the CQI process on the innovation, the context in which the intervention is delivered (e.g., a child development center), and the broader ecological system within which the practice operates (e.g., the Service Branches and DoD) helps ensure that the ultimate benefit of the innovation will be for family well-being and readiness outcomes within a practice setting and context. Characteristics of the setting and context include human and capital resources, organizational culture and climate, power structures, and processes for training and supervision of staff. These setting and contextual characteristics directly influence the ability of an interven-

tion to reach the targeted population. As a result, the system’s CQI process demands on-going measurement of setting and context such that strategic changes to the context and/or adaptions to the intervention can be made to resolve problems of fit. In most cases, the practice setting needs support to build its capacity for innovation and progression.

An Example of a DoD Family Readiness Program

FOCUS (Families Over-Coming Under Stress) is one example of an existing DoD military family readiness program with the potential to apply an adaptive approach to implementation consistent with a dynamic sustainability framework. Designed to strengthen family resilience, the program was adapted from the developers’ evidence-based practices, which had been found through randomized control trials over longitudinal follow-up to improve parenting, family functioning, and youth and parent outcomes.

The common-core intervention elements in FOCUS were defined through expert consensus on shared contributing structures, processes, and other elements (Beardslee et al., 2003, 2007; Layne et al., 2008; Rotheram-Borus et al., 2004; for review see Lester et al., 2016). Four common core elements were defined, namely: (1) evidence-based assessment and real-time personalized guidance; (2) context-specific education, such as trauma-and resilience-informed education, positive parenting, and developmental guidance; (3) individual and family-level skill development (for such skills as emotional regulation, problem solving, communication, goal setting, managing separation/trauma reminders); and (4) the development/sharing of individual and family-level narrative communication timelines. These elements were customized, piloted, and manualized using a community participatory methodology (as reviewed in Chapter 7) with military providers, families, and leaders that informed intervention tailoring and implementation design (Beardslee et al., 2013; Lester et al., 2010; Saltzman et al., 2011).

Delivered within DoD as a suite of services based on EBP core elements, FOCUS services are delivered as a tiered continuum of prevention consistent with a population health model (National Research Council and Institute of Medicine [NRC and IOM], 2009b). These services range from universal to indicated¹¹ prevention services, and they use multiple platforms to support flexible engagement, screening, and intervention delivery, including educational workshops, web-based/mobile tools, skills groups, consultations, and in-person and in-home tele-prevention video-teleconferencing multi-session family interventions (Beardslee et al., 2011, 2013). Between 2008 and 2018, FOCUS services have been implemented for active

___________________

¹¹Indicated care signifies the care designed only for those individuals showing warning signs of a problem.

duty families at 24 installations with consistently high levels of engagement and participation across the continuum of tiered services, as well as high adherence by families within the multisession models.

Follow-up evaluations of the multi-session family intervention and its adaptations have demonstrated significant and sustained individual and family-level outcomes up to six months later. In adults, these evaluations showed reductions in depression, anxiety, and PTSD symptoms; in children, they found decreased internalizing and externalizing symptoms, improved prosocial behaviors, reduced anxiety, and improved coping; and they further found improved family/couple adjustment (Lester et al., 2011, 2016; Saltzman et al., 2016). The CQI process embedded in the implementation has informed multiple adaptations of the model based on data monitoring and community participation, needs, and trends. These adaptations have included specific adaptations of FOCUS for specific family constellations (e.g. FOCUS Couples; FOCUS Early Childhood), context (e.g. FOCUS-Wounded, Ill and Injured) and platform (e.g. TeleFOCUS, FOCUS On the Go!) (Ardslee et al, 2013). As described in Chapter 7, the adaptation for early childhood delivered as an in-home telehealth platform has recently been evaluated through a randomized trial, which found that it demonstrated improvements in reported parenting stress, parent-child relationships, and observed parenting and reduced parental PSTD symptoms compared to a web-based parenting curriculum (Mogil et al., in review). Lessons from the large-scale implementation have been translated to reach military-connected couples and families in a range of settings, including school systems, international military, community mental health, and veteran-serving organizations (Garcia et al., 2015; Ijadi-Maghsoodi et al., 2017; Karnik, 2018; NATO, 2019; Tanielian et al., 2018), providing an example of the relevance of this approach across multiple systems.

How Big Data Can Support an Effective Learning System Framework

Big data, first defined in 2003, refers to the rapidly increasing volume of available data, the velocity at which data are generated, and the ways in which the data are represented (Hashem et al., 2015). Big data provides opportunities to successfully build a culture and infrastructure to support a learning MFRS that aligns with CQI monitoring as described above. To be successfully utilized for military family readiness and well-being, big data must be integrated into four major categories while also following the principles of Plan-Do-Study-Act or PDSA, (Agency for Healthcare Research and Quality, 2008; Deming, 1986) as outlined in Table 8-1.

Several important National Academies reports have outlined the benefits of using big data to improve services and health care for active service members and to provide insight into how future military members will

access services. For example, in a 2014 Institute of Medicine report, the committee recommended that DoD implement comprehensive family- and patient-centered evidence-based prevention programming directed toward psychological health in military families, spouses, partners, and children (IOM, 2014). Such targeted strategies are likely to be most successful when born out of an effective learning system that incorporates the PDSA cycle outlined in Table 8-1.

Applying the Donabedian Framework in the Context of Big Data

The conceptual model shown in Figure 8-3 illustrates how collection, analysis, and dissemination of big data can use the Donabedian framework (as described in Chapter 7) to provide higher-quality services to and improve the well-being of military families. The structural elements include the infrastructure and data components from which the data points are collected, including the provider teams, active service members and their families, and service programs, schools, and community facilities, as well as population-level data for the group as a whole. As part of the process measures, the data would be transferred to a web services platform that allows for data cleaning, standardization, and visualization. Output to

TABLE 8-1 Big Data Utilization Within a Continuous Learning System Using the IOM Learning System Framework Integrated with Plan-Do-Study-Act (PDSA) Principles

SOURCE: Marmor (2018).

the end user is through application programing interfaces. This stack of informatics, which draws on many primary data sources, including service program and provider notes, supplies the analytics needed to give leaders, providers, researchers, families, and systems valuable, real-time information on military family well-being and needs, and provides the integrated information infrastructure to inform CQI.

This integrated information infrastructure requires operational analytics support to oversee compliance with data use agreements and data safety and to prevent misuse of data to ensure that military policy makers and end users do not misinterpret big data results and analyses. The real-time information in the outcomes portion of the model offers provider teams and families with performance measures, provider alerts, notifications on safety, quality, and value, and research data points, all of which can be used for future analyses as part of a larger monitoring effort. Military families need

personalized predictions about well-being and response to interventions, as well as a deeper understanding of the complex factors and their interactions that influence well-being and readiness. Big data initiatives allow for cluster-level queries with multilevel stratifications when evidence does not exist for a decision. For example, analysis of data with learning-enhanced approaches may be used to detect mental health issues, suicidality, and other risks to family well-being, and ultimately lead to more effective methods of comparing screening and prevention options.

Meeting the challenges to developing a learning MFRS will be contingent on having sound, robust data and predictive analytics. Issues involving data inaccuracy, erroneous or ambiguous data points, missing data, and selective measurement must be addressed in designing an infrastructure. Building interoperability into the system’s data infrastructure will require thoughtfulness and foresight. Currently available service record and management systems have heterogeneous architectures not always built for big data analyses and a learning system framework, and they are further limited by ways the data is entered, which may interfere with preparation for predictive analytics.

Foundational work implemented now could establish cost savings in the future as predictive analytic tools mature. This cost savings is most apparent in data safety, given the significant costs associated with the storage and necessary protections of data. Creating a learning health system in which data safety is paramount will avoid a need to spend future resources on preventing data breaches. A ground-up approach in building data systems with safety at the forefront will minimize future costs and maximize the trust and usability of the systems for the military and their families.

Cost savings can also be seen if scalable systems are built to incorporate future data sources. The advanced interconnectivity and data collection of mobile and wearable devices provides an opportunity to scale military family programming beyond traditional delivery platforms. Ensuring that the MFRS is prepared to integrate usable data from wearables would minimize future costs associated with resource expansion once data are more readily available.

Military families need personalized predictions about well-being and response to services treatments. Big data initiatives allow for cluster-level queries with multilevel stratifications when evidence does not exist for a decision. For example, analysis of data with learning-enhanced approaches may be used to detect mental health issues, suicide and family well-being, and safety problems with drugs and devices. Ultimately, this would lead to more effective methods for comparing prevention, diagnostic, and treatment options.

Although the promise of big data is enormous, the benefits gained through a big data learning MFRS include security risks that need to be addressed. Cearley and Burke (2018) note that digital ethics and privacy are critical to any technological decision that an organization may make. Consumers of technology are demanding that their personal information be protected and are concerned about how their personal data are being used. The authors observe that privacy and digital ethics are intertwined and are built on the trust of customers. The Pew Research Center (2014) surveyed more than 2,000 experts on the future of digital privacy and found that privacy and security are foundational components of the digital world. Pew’s report did not find a consensus among these experts, with some of them believing that by 2025 there will not be an accepted privacy infrastructure in place and privacy concerns will worsen as individuals’ wearables and other “things” will spy on them and report on their behaviors.

Others in this expert survey believed that by 2025, consumers will have increased access to tools that will give them more control over their personal information in a tiered fashion so that they can choose who to provide access to their information, such as a health care professional. Some experts predicted that a privacy system or infrastructure will be put in place that will allow individuals the ability to set their own terms and policies about who accesses their data. At any rate, experts note that a challenge will be to put in place a system that specifies standards for data protection.

Any big data learning system must strive to create safeguards that protect classified military information and have the highest levels of protection for military families. Current military regulations may need to be re-examined to maximize the benefit from data collection while simultaneously maintaining operational security and minimizing risk to service members and their families. In the wake of several data scandals involving the use of private data (e.g., Granville, 2018; Yang and Jayakumar, 2014), data use agreements must involve participants in decision-making processes, set clear standards for ethical rigor, and specify sanctions for data misuse and abuse.

The Role of Mobile Technologies and Other New Digital Technologies in Supporting a Complex Adaptive MFRS

In a recent report on the top 10 strategic technology trends for 2019 by Gartner, a technology consulting firm, several trends were identified with direct implication for the delivery of programs to military families (Cearley and Burke, 2018). For instance, one top technology trend they cite is “autonomous things” (p. 6), which can come in the form of robotics, drones, vehicles, appliances, or agents, such as virtual assistants. The MIT Media Lab is experimenting with “social robots,” robots that use artificial

intelligence (AI) systems to interact with people. Social robots are being used as personal health coaches, pet surrogates, companions, and more. When Breazeal and colleagues (2019) compared the use of a digital assistant versus a social robot among adults ages 50 and older, they found that older adults were more socially engaged with the robot compared to the digital assistant, and that the robots served as social catalysts and promoted human-human interaction. Social robots can also be used to foster connections with family members who are remotely located, which may be particularly applicable to spouses, partners, and children of service members.

The MIT Media Lab is also experimenting with “emotionally intelligent” virtual agents that can personalize how they interact with individuals based on their emotional state (Ghandeharioun et al., 2018). Ghandeharioun and colleagues conducted two randomized controlled trials examining different ways in which the Emotion-Aware mHealth Agent, or EMMA, improved individuals’ well-being. EMMA provides wellness suggestions to participants using “micro-interventions” that use positive psychology, cognitive behavioral, meta-cognitive, or somatic psychotherapy strategies. The researchers found that EMMA was likeable but needs to be focused on more specific moods and contexts and to be less predictable in order to improve well-being. In particular, individuals who were classified as extroverts found EMMA to be likeable. In addition, the authors offered several design guidelines for emotionally intelligent virtual agents, such as “do not interrupt a good mood” and offer “short, simple, and effortless activities” (Ghandeharioun et al., 2018, pg. 23).

Integrated Mobile Technology

Another top 10 technology trend is the movement from the use of individual devices and wearables to a multichannel and multimodal experience (Cearley and Burke, 2018). In this multimodal experience, virtual reality and augmented reality will be integrated with mobile technologies and wearables. The increasing utility and acceptability of mobile platforms for the delivery of health and mental health services can be adapted to provide a special opportunity for DoD to strengthen individual and family well-being through screening and program delivery across the spectrum of coordinated support of MFRS, as described in Chapter 7 (refer to Figure 7-1). In health care settings, program components are referred to as “mHealth” when mobile or other wireless devices (e.g. smartphones, tablets, wearables¹²) have been applied to promotion and prevention contexts to support resilience and well-being (Kumar et al., 2013).

___________________

¹² Wearables are technological devices that are worn on the body or incorporated into clothing. These include fitness tracking devices, smart watches, and other devices that use wireless systems.

Mobile and wireless devices allow for more accessible and cost-effective interventions because: (1) their widespread use, acceptability, and convenience can help reduce certain societal and structural barriers (Amico, 2015) and (2) they offer strong capability for scaling across geographic locations (Muessig et al., 2017), including within resource-limited (Haberer et al., 2017), hard-to-reach (McInnes et al., 2014), and deployed settings (Gifford et al., 2014). Mobile devices are capable of giving round-the-clock, real-time reminders and feedback. Smartphones and tablet computers are also able to host applications (apps) with therapeutic content as well as a multitude of capabilities, such as social networking and gaming (Pellowski and Kalichman, 2012). Using these technologies to support well-being through behavioral health in a family’s natural environment can mitigate the logistical burdens (e.g. scheduling conflicts, childcare, travel) associated with traditional in-person offerings of programs and services. The option to receive care outside of standard settings (e.g., in the privacy of one’s own home) may be particularly appealing to service members and their families who live off-installation or who are concerned about perceived stigma associated with seeking assistance (Luxton et al., 2016).

Mobile in Stepped-care Delivery Strategies

Because mobile interventions can be disseminated conveniently and have the potential to promote behavioral change at low cost, they may have particular utility for stepped-care policies and adaptive interventions. Consistent with a spectrum of coordinated support as described in Chapter 7, stepped-care practices are evidence-based, staged systems comprising a hierarchy of interventions, from the least to the most costly/intensive, matched to the individual’s needs.¹³ As compared to a fixed, one-size-fits-all program, a stepped-service approach initially requires minimal support, since it starts with the least expensive and/or burdensome intervention component, and then applies more costly or more burdensome components only to those who need them the most, such as those showing early signs of nonresponse. Even less costly or less burdensome components can be offered to individuals who show adequate response to minimal support. The goal is to step up and down the intensity or cost of prevention based on early signs of progress in order to achieve a more cost-effective outcome.

Stepped-service delivery strategies are a form of adaptive intervention, an intervention design in which information about the individual’s prog-

___________________

¹³ See Australia Department of Health, PHN Primary Mental Health Care Flexible Funding Pool Implementation Guidance, Stepped Care, at http://www.health.gov.au/internet/main/publishing.nsf/content/2126B045A8DA90FDCA257F6500018260/$File/1PHN%20Guidance%20-%20Stepped%20Care.PDF.

ress in the course of the intervention, such as early signs of nonresponse or poor adherence, is used to modify aspects of the type, dosage, intensity, or delivery modality of an intervention. Adaptive interventions aim to address not only the unique needs but also the changing needs of individuals or families over time, so they might be uniquely suited for understanding and addressing the diversity of military family needs (Nahum-Shani and Militello, 2018). By providing appropriate interventions only to those who need them, when they need them (Carels et al., 2005, 2007), adaptive interventions can improve long-term outcomes for greater numbers of individuals, increasing the reach and impact of treatments.

Mobile in Just-in-time Delivery Strategies

Mobile tools also offer novel opportunities for delivering just-in-time adaptive interventions (JITAIs). A JITAI is a form of adaptive intervention that aims to address in real time the rapidly changing needs of individuals or families (Nahum-Shani, Hekler, and Spruijt-Metz, 2015; Spruijt-Metz and Nilsen, 2014). Consistent with the notion of personalized medicine, JITAIs put into practice the personalized real-time selection and delivery of intervention strategies based on real-time data (Spruijt-Metz and Nilsen, 2014). JITAIs have been developed and evaluated for a wide range of behavioral health issues, including physical activity (King et al., 2013; Thomas and Bond, 2015), alcohol use (Gustafson et al., 2014; Witkiewitz et al., 2014), mental illness (Ben-Zeev et al., 2014), and smoking cessation (Free et al., 2011; Riley et al., 2008). Most recently, they have been applied to support the well-being veterans with PTSD and their caregivers, using mobile applications, with promising initial outcomes for improving intervention engagement (Barish et al., 2014). See Box 8-3 for an example of a JITAI.

Adaptive interventions contain four key elements: (1) decision points, that is, points during an intervention when a decision is made about whether and what type of intervention to provide; (2) tailoring variables, that is, the information provided to decide whether and how to modify the intervention; (3) intervention options, that is, the different types of treatment, tactics, intensities, dosages, or modalities used to deliver the treatment; and (4) decision rules, which link information about the individual (i.e., the tailoring variable) to intervention options. The decision-making rules specify for each decision point what intervention option should be offered under various conditions.

Specific Advantages of Adaptive Interventions for Military Families

Adaptive interventions hold great potential for advancing the well-being of service members and their families in the ways they could support a complex adaptive MFRS. First, as discussed earlier, they could help

address the varying needs of military families over time, concerning both within-person variation and within-family heterogeneity. Moreover, the demands of military life, such as frequent moves, can impact continuity of care and treatment response for service members and their families (Gleason and Beck, 2017; Marshall et al., 2011). During times of transition, when traditional forms of treatment such as in-person clinical visits are not feasible, mobile monitoring and interventions can be utilized at any time and anywhere, unleashing the potential to facilitate continued access to some form of treatment.

Second, there is great fluctuation in the responses to interventions aiming to improve the health and well-being of service members and their families. As an example, consider therapies for posttraumatic stress disorder (PTSD). Although treatments such as prolonged exposure and cognitive processing therapy were found effective in reducing military-related PTSD symptoms, across a range of studies at least half of veterans still meet diagnostic criteria for PTSD following treatment (Steenkamp and Litz, 2014). This led Steenkamp and Litz (2014) to conclude that “overall,

dissemination models must move beyond simple one-size-fits-all conceptualizations of treatment if they are to adequately reflect the evidence base and the complexity of PTSD in veteran populations” (p. 706). Adaptive interventions can be used to address such between-person heterogeneity by identifying individuals who show early signs of nonresponse and modifying their treatment (e.g., by providing additional support) in order to ensure that they ultimately respond.

Third, barriers to promotional and prevention services for military families include limited availability (e.g., shortage of qualified providers, long wait times), accessibility (e.g., absence of reliable transport to off-base services, limited availability of childcare), and acceptability (i.e., stigma and negative attitudes toward support services) (American Public Health Association, 2014; Verdeli et al., 2011). Using a stepped-care approach that capitalizes on mobile tools as minimal support has the potential to address these barriers. Mobile interventions may attract and retain those service members and their families who are unlikely to seek out traditional therapeutic interventions (e.g., due to burden or stigma), especially when low-intensity tools are integrated into a system of services (Monk et al., 2017).

Given the widespread use, convenience, and acceptability of mobile tools, utilizing them to deliver initial minimal support can further increase access to care. This is particularly critical in the context of military families, given that more than 70 percent of active-duty military families live off-installation (Boberiene et al., 2014; National Military Family Association, 2011; Whitestone and Thompson, 2016). Living off-installation decreases access to care due to the lack of services for military families far from military installations, and it also hinders integration into a military support network. Low-cost, low-burden, accessible mobile tools can be used to deliver to military service members and their families universal screenings and interventions and screening, that is, those designed to reach and target an entire population. Individuals or families identified as needing more than minimal support can then be linked to more costly or more intense indicated care to address their specific needs. Moreover, once the desired outcome is obtained, it is not always clinically appropriate to stop treatment completely or resume an intense treatment schedule (Borsari et al., 2011), and mobile tools can be used to “step-down” treatment instead, that is, to deliver boosters and maintain gains, such as through extended monitoring (McKay et al., 2010).

Developing Adaptive Interventions for Military Families

Although multiple evidence-based adaptive interventions exist, limited attention has been given to the systematic development and implementation of adaptive interventions for military families. First, stepped care is loosely

implemented in the treatment of service members and their families, and the decision rules regarding when to step treatment up or down are not well defined. Second, current stepped-care approaches for military service members do not explicitly specify how, when, and for whom mobile tools should be used in service delivery. Finally, while mobile technology tools are natural candidates to be integrated into a stepped-care approach for military service members, these tools have not yet demonstrated evidence of effectiveness in this population (Jai et al., 2016; Miller et al., 2016; Shore et al., 2014). More evidence is needed to evaluate and optimize existing stepped-care models in military settings.

As an example of a current adaptive-intervention research effort underway for active duty families, the After Deployment, Adaptive Parenting Tools (ADAPT) Program is an intervention recently adapted to include mobile platforms to support delivery of prevention services with military families. Designed to be delivered following a parent’s military deployment, ADAPT targets key parenting practices in order to strengthen children’s resilience (Gewirtz et al., 2014); randomized controlled trial results have demonstrated that the program improves parenting, strengthens children’s social, emotional, and behavioral functioning, and reduces parental distress (DeGarmo and Gewirtz, 2018; Gewirtz et al., 2016, 2018b; Piehler et al., 2018). The program is currently being evaluated as an adaptive intervention aimed at offering multiple formats, dosages, and sequences to deliver precision (i.e. personalized) programs for families (DeGarmo and Gewirtz, 2019). This evaluation research will inform a population-level implementation of ADAPT, which is currently planned for a large military installation and includes a universal self-directed web-based program. Following this first universal dose, parents complete an online assessment of parenting efficacy. The results of the evaluation (concerning family needs and consumer/parent preferences) determine whether and what services are subsequently offered and may include group-based and workshop programs, individual family telehealth, and face-to-face interventions.

JITAIs supported by mobile technology, including wearables, have particular applicability for military family interventions. As mentioned earlier, JITAIs are motivated by the need to address conditions that change rapidly, unexpectedly, and in the person’s natural environment (Nahum-Shani et al., 2015, 2017). These conditions can represent vulnerability (high risk) or opportunity for positive changes. Use of JITAIs can also be motivated to capitalize on states of opportunity for positive changes. For example, a JITAI for promoting physical activity can use information about the person’s location to identify when she or he is close to a park or a recreational facility to trigger a recommendation for the person to engage in physical activity. Here, rather than focusing on states of vulnerability to adverse

outcomes the JITAI targets proximity to opportunities for engaging the individual in positive activity (refer to Box 8-3 for a detailed example.)

Stress is another example of a health risk that new advances in mobile and wireless devices can help address. Stress episodes often occur rapidly and unexpectedly. Research shows that every minute a person can transition from experiencing no-stress to experiencing stress, and it is not possible to predict exactly when a person will experience stress during the day. Stress episodes also occur in a person’s natural environment, that is, while at work, due to job demands, or at home as a result of family-related demands. Identifying stress when it occurs, as soon as it occurs, and responding quickly would require the capability to continuously monitor the person’s state and context as well as to deliver interventions “in the wild,” that is, outside of a standard medical treatment setting. JITAIs enabled by wearable or other mobile devices may make this possible. The conditions JITAIs attempt to address are expected to emerge in the person’s natural environment, where multiple demands compete for the person’s time, effort and attention, and these interventions are also designed to explicitly minimize disruptions to the daily lives and routines of individuals. At the same time, this can be done only by providing an intervention only when the person is receptive, namely able and willing to capitalize on a given intervention.

Research on the behavioral health and adjustment implications of military deployments for families highlights the dynamic relationship between these military life experiences and military connected families as a source of both vulnerability and opportunity for personal growth. This dynamism has implications for the timing of preventive interventions (Najera et al., 2017). For example, empirical evidence concerning “dyadic coping”—the interplay between the stress of one partner and coping reactions of the other (MacDermid Wadsworth and Riggs, 2010)—indicates that in instances of chronic illness, such as cancer, positive or common dyadic coping (when both partners work symmetrically) is associated with improved relationship quality, self-care, and psychological functioning in both the individual coping with illness and their partners (Denham, 2002; Jones and Fiese, 2014; Monasta et al., 2010; Najera et al., 2017; Spagnola and Fiese, 2007). JITAIs have the potential to promote positive dyadic coping by providing real-time feedback and suggestions to address conditions specific to each person in the relationship, such as stress experienced by the months of service due to family separation or parenting stress experienced by the non-deployed spouse (Lara-Cinisomo et al., 2012). They may also promote positive coping by addressing dyadic conditions that arise between the service member and his/her partner, such as asymmetry in relational maintenance associated with communication restrictions (Merolla, 2010; Rea et al., 2015).

Military service can also impact child well-being and family functioning (Cronin et al., 2015; Gewirtz et al., 2018a; Hardy, Power, and Jaedicke, 1993; Saltzman et al., 2011). Potential sources of stress for military family children include frequent moves, often every 2 or 3 years, parental deployment(s), and/or parental injury (Chandra et al., 2010; Collins, 2015; Sogomonyan and Cooper, 2010). According to Rosenblum and Muzik (2014), although these large-scale disruptions pose challenges for parent-child relationships, meeting children’s needs and addressing smaller-scale disruptions during everyday experiences are important processes by which relationships can be restored and strengthened. Additionally, resilience is the dynamic process of positive adaptation within the context of significant adversity (Collins, 2015; Cronin et al., 2015).

Critical to resilience is self-regulation (Mestre et al., 2017; Saltzman et al., 2011), namely the flexible modulation of cognition, behavior, and emotion (Nielsen et al., 2018). Empirical evidence suggests that engaging in evidence-based self-regulatory activities, such as behavioral substitution, mindful attention, relaxation techniques, and searching for strengths within common everyday experiences, can promote self-regulatory skills (Bratt et al., 2017; Elwafi et al., 2013; Goldberg et al., 2014; Padesky and Mooney, 2012). JITAIs can be used to address, in real time, parents’ and children’s need for hands-on strategies for managing everyday small-scale difficulties and building self-regulatory skills. Current family-based models encourage the training of parents and children to identify personal- and family-level triggers and the development of collaborative strategies for modulating their impact (Saltzman et al., 2011). Such models can be used to guide the development of JITAIs that address family needs more holistically, by detecting personal- and family-level triggers in real time and delivering commendations to engage in collaborative self-regulatory activities and supportive familial transactions.

SUMMARY

The opportunity to learn from the implementation of evidence-based interventions for military family well-being extends well beyond the specific population and service systems where military families receive services in DoD-supported systems. It includes civilian settings that are also critical to the well-being and readiness of military families, such as civilian schools and primary care settings. Learning from the implementation of evidence-based interventions in the contexts that serve military families enables the ongoing study of evidence-informed and evidence-based practice implementation, the monitoring of the impact of implementation on families, providers, and service systems, and the development of methods and measures that can serve to build tools for the field of implementation science.

There are several contributions that a functional implementation laboratory for military families’ well-being can make for the field at large. First, the military context provides a rich, multilevel, multisectoral, and multiorganizational environment that is a microcosm of the ecology within which implementation science findings are applied. The constituent families must be reached across substantial geography and with a recognition of diversities in culture, socioeconomic status, access to resources, and family needs. Unlike many other settings, however, treating the MFRS as an “implementation laboratory” is an approach whose strengths could be replicated in other systems. DoD has the benefits of an integrated care system, with multiple data sources that together inform service needs, services received, and outcomes. Military families, given their diversity, have needs that can be generalized to the larger population and, thus, insights on effective implementation with military families could be generalized as well.

Second, some of the same interventions are delivered within and outside of the military context, and as such the implementation strategies to bring evidence-based interventions to military families can be simultaneously tested in external settings as well. The cross-context comparisons can help to isolate what characteristics of health care and community settings improve the adoption, implementation, and sustainability of interventions. They can also add to the field’s understanding of how and why implementation strategies improve the successful integration of interventions into practice.

Third, data resources within the military context are more advanced than elsewhere, and the availability, volume, and scope of data can be leveraged to identify areas where interventions are needed for families, inform how a package of evidence-based interventions can be assembled to meet those needs, guide strategies selected to implement the set of interventions, monitor implementation progress over time, and adjust what is implemented, how it is implemented, and where it is implemented over time. This ability to study and improve understanding of intervention adaptation, intervention sustainment, and (where needed) decommissioning is generally lacking in the field. Such an effort could form the basis of a learning implementation system (IOM, 2013; Stein et al., 2016) whose benefits to our knowledge base could be immense.

Fourth, the durable, strong support of the country for its service members and their families offers an opportunity for sustained study of implementation over a longer time horizon than is typically possible in a research study. In addition, an ongoing study of implementation, adaptation, sustainment, and decommissioning that does justice to the inherent dynamism in real-world settings will require a commitment to apply the lessons learned over time, toward ongoing support-system improvement. The military context may provide one of the few examples where this

commitment will persist in the coming years. Therefore, this is a natural setting in which to test innovative strategies to more effectively support implementation, adaptation, improvement, and (as appropriate) discontinuation of a variety of evidence-based practices. A significant investment in applied research that identifies core mechanisms affecting implementation processes will produce value, most directly for military families and more indirectly for the larger implementation science community.

Finally, the implementation laboratory within the collective MFRS, designed to provide support to service members and their families, allows for the investigation of a range of different research questions, a small sample of which are these:

1. What strategies and approaches to adaptation work, and how do they work?
2. How should systems optimally select and scale a combination of complementary evidence-informed and evidence-based programs?
3. Can we identify the rate at which different evidence-informed and evidence-based programs and implementation strategies can scale up across systems and communities?
4. What are the comparative effectiveness and cost-effectiveness of variable strategies to adopt, implement, adapt, sustain, or decommission interventions to promote resilience, readiness, and well-being?

Answering these and other questions can directly benefit military family well-being and ultimately contribute to the mission-readiness of the force. In addition, learning from this endeavor would greatly contribute to the larger knowledge base of implementation science. In concert with other research activities, the field can substantially improve the integration of research and practice and improve population-level well-being. The military infrastructure is uniquely positioned to both learn from and contribute to the science of intervention adaptation specifically and the field of implementation science more broadly.

CONCLUSIONS

CONCLUSION 8-1: Implementation research and models can help facilitate the integration of evidence-based and evidence-informed practices into care delivery by identifying likely barriers toward implementation and providing guidance for how best to overcome such barriers. The insights generated by the field of dissemination and implementation science can serve as a mechanism for improving the Military Family Readiness System.

CONCLUSION 8-2: A sustainable Military Family Readiness System (MFRS) requires ongoing adaptation of programs, services, resources, and practices that is supported by a continuous learning system. An effective MFRS includes, whenever possible, evidence-based and evidence-informed practices, processes, programs, and policies and uses a community participatory approach to adapt and implement within service settings and geographical contexts.

CONCLUSION 8-3: Effective implementation of military family readiness services requires an integrated information infrastructure to support the measurement, analytics, and organizational leadership infrastructure necessary for continuous quality improvement processes to inform adaptations, accountability, workforce training, and sustainability.

CONCLUSION 8-4: An effective Military Family Readiness System requires a learning system framework that is data driven and culturally responsive to family and community diversity as well as to the complex and emergent challenges of military service.

CONCLUSION 8-5: Military families and the Military Family Readiness System will benefit from the utilization of big data and predictive analytics to monitor and tailor interventions that influence well-being at the level of the individual and the family.

REFERENCES

Beardslee, W. R., Gladstone, T. R. G., Wright, E. J., and Cooper, A. B. (2003). A family-based approach to the prevention of depressive symptoms in children at risk: Evidence of parental and child change. Pediatrics, 112, e119-e131.

Beardslee, W. R., Klosinski, L. E., Saltzman, W., Mogil, C., Pangelinan, S., McKnight, C. P., and Lester, P. (2013). Dissemination of family-centered prevention for military and veteran families: Adaptations and adoption within community and military systems of care. Clinical Child and Family Psychology Review, 16(4), 394–409.

Beardslee, W., Lester, P., Klosinski, L., and Leskin, G. (2011). Family-centered preventive intervention for military families: Implications for implementation science. Prevention Science, 12(4), 339–348.

Beardslee, W. R., Wright, E. J., Gladstone, T. R. G., and Forbes, P. (2007). Long-term effects from a randomized trial of two public health preventive interventions for parental depression. Journal of Family Psychology, 21(4),703–713.

Becker, K. D., Lee, B. R., Daleiden, E. L., Lindsey, M., Brandt, N. E. and Chorpita, B. F. (2013). The common elements of engagement in children’s mental health services: Which elements for which outcomes? Journal of Clinical Child & Adolescent Psychology, 44(1), 30–43.

Benbenishty, R. (2014). Building Capacity in Military Schools: Final Technical Evaluation Report. Los Angeles: University of Southern California School of Social Work.

Ben-Zeev, D., Brenner, C. J., Begale, M., Duffecy, J., Mohr, D. C. and Mueser, K. T. (2014). Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophrenia Bulletin, 40(6), 1244–1253.

Bernal, G. E., and Domenech Rodríguez, M. M. (2012). Cultural Adaptations: Tools For Evidence-Based Practice With Diverse Populations. Washington, DC: American Psychological Association.

Blitz, C. C., Arthur, M. W. and Hawkins, J. D. (2002). Preventing alcohol, tobacco, and other substance abuse. In L. A. Jason and D. S. Glenwick, (Eds.), Innovative Strategies for Promoting Health and Mental Health Across the Life Span (pp. 176–201). New York, NY: Springer.

Boberiene, L. V., and Hornback, B. J. (2014). How can policy strengthen community support for children in military families? American Journal of Orthopsychiatry, 84(5), 439.

Borsari, B., Capone, C., Mastroleo, N. R. and Monti, P. M. (2011). Clinical considerations in the treatment of substance use disorders with veterans. Journal of Contemporary Psychotherapy, 41(4), 247–253.

Bratt, S., Semaan, B., Britton, L., Dosono, B., and Zeno, F. (2017). Translation in personal crises: Opportunities for wearables design. In Proceedings of the 14th International Conference on Information Systems for Crisis Response and Management– Albi, France, May 2017.

Breazeal, C. L., Ostrowski, A. K., Singh, N., and Park, H. W. (2019). Designing social robots for older adults. The Bridge (National Academy of Engineering) 49(1), 22–31.

Bumbarger, B., and Perkins, D. (2008). After randomised trials: Issues related to dissemination of evidence-based interventions. Journal of Children’s Services, 3(2), 55–64.

Burkhardt, J. T., Schröter, D. C., Magura, S., Means, S. N. and Coryn, C. L. (2015). An overview of evidence-based program registers (EBPRs) for behavioral health. Evaluation and Program Planning, 48, 92–99.

Cabassa, L. J., and Baumann, A. A. (2013). A two-way street: Bridging implementation science and cultural adaptations of mental health treatments. Implementation Science, 8(1).

Carels, R. A., Darby, L., Cacciapaglia, H. M., Douglass, O. M., Harper, J., Kaplar, M. E., Konrad, K., Rydin, S. and Tonkin, K. (2005). Applying a stepped-care approach to the treatment of obesity. Journal of Psychosomatic Research, 59(6), 375–383.

Carels, R. A., Darby, L., Cacciapaglia, H. M., Konrad, K., Coit, C., Harper, J., Kaplar, M. E., Young, K., Baylen, C. A., and Versland, A. (2007). Using motivational interviewing as a supplement to obesity treatment: A stepped-care approach. Health Psychology, 26(3).

Catalano, R. F., Berglund, M. L., Ryan, J. A. M., Lonczak, H. S., and Hawkins, J. D. (2002). Positive youth development in the United States: Research findings on evaluations of positive youth development programs. Prevention & Treatment, 5(1).

Cearley, D., and Burke, B. (2018). Top 10 Strategic Technology Trends for 2019. Stamford, CT: Gartner, Inc.

Chambers, D. A., Glasgow, R. E., and Stange, K. C. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science, 8(1), 1–11.

Chambers, D. A., and Norton, W. E. (2016). The adaptome: Advancing the science of intervention adaptation. American Journal of Preventive Medicine, 51(4), 124–131.

________. (2018). Application of Implementation Science to Population-based, Behavioral Health Prevention Continuum of Care for Military Families. Paper commissioned for the National Academies Committee on the Well-being of Military Families.

Chandra, A., Lara-Cinisomo, S., Jaycox, L. H., Tanielian, T., Burns, R. M., Ruder, T., and Han, B. (2010). Children on the homefront: The experience of children from military families. Pediatrics, 125(1), 16–25.

Chilenski, S. M., Olson, J. R., Schulte, J. A., Perkins, D. F., and Spoth, R. (2015). A multi-level examination of how the organizational context relates to readiness to implement prevention and evidence-based programming in community settings. Evaluation and Program Planning, 48, 63–74.

Chilenski, S. M., Perkins, D. F., Olson, J. R. Hoffman, L., Feinberg, M. E., Greenberg, M. T., Welsh, J., Crowley, D. M., and Spoth, R. L. (2016). The power of a collaborative relationship between technical assistance providers and community prevention teams: A correlational and longitudinal study. Evaluation and Program Planning, 54, 19–29.

Chorpita, B. F., Becker, K. D., Daleiden, E. L., and Hamilton, J. D. (2007). Understanding the common elements of evidence-based practice: Misconceptions and clinical examples. Journal of the American Academy of Child & Adolescent Psychiatry, 46(5), 647–652.

Chorpita, B. F., Bernstein, A., and Daleiden, E. L. (2011). Empirically guided coordination of multiple evidence-based treatments: An illustration of relevance mapping in children’s mental health services. Journal of Consulting and Clinical Psychology, 79(4), 470.

Chorpita, B. F., and Daleiden, E. L. (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77(3), 566.

Chorpita, B. F., Daleiden, E. L., and Weisz, J. R. (2005). Identifying and selecting the common elements of evidence based interventions: A distillation and matching model. Mental Health Services Research, 7(1), 5–20.

Coffron, M., and Opelka, F. (2015). Big promise and big challenges for big heath care data. Bulletin of the American College of Surgeons, 100(4), 10–16.

Collins, E. (2015). Experts explain mental state of military children. Soldiers Magazine, May 1. Retrieved from https://www.army.mil/article/147786/experts_explain_mental_state_of_military_children.

Crepaz, N., Tungol-Ashmon, M. V., Vosburgh, H. W., Baack, B. N. and Mullins, M. M. (2015). Are couple-based interventions more effective than interventions delivered to individuals in promoting HIV protective behaviors? A meta-analysis. AIDS Care, 27(11), 1361–1366.

Cronin, S., Becher, E., Christians, K. S. and Debb, S. (2015). Parents and Stress: Understanding Experiences, Context, and Responses. St. Paul: University of Minnesota.

Dabek, F., and Caban, J. J. (2015). Leveraging big data to model the likelihood of developing psychological conditions after a concussion. Procedia Computer Science, 53, 265–273.

Damschroder, L. J., and Hagedorn, H. J. (2011). A guiding framework and approach for implementation research in substance use disorders treatment. Psychology of Addictive Behaviors, 25(2), 194.

Davidoff, F., Batalden, P., Stevens, D., Ogrinc, G., and Mooney, S. (2008). Publication guidelines for quality improvement in health care: Evolution of the SQUIRE project. BMJ Quality & Safety, 17(1), i3-i9.

Deaton, A., and Cartwright, N. (2018). Understanding and misunderstanding randomized controlled allocation and dynamic adaptive intervention designs for family psychology trials. Social Science & Medicine, 210, 2–21.

DeGarmo, D. S., and Gewirtz, A. H. (2018). A recovery capital and stress-buffering model for post-deployed military parents. Frontiers in Psychology, 9, 1832.

________. (2019). Fixed. In B. Fiese, M. Celano, K. Deater-Deckard, E. N. Jouriles, and M. A. Whisman (Eds.), APA Handbook of Contemporary Family Psychology: Foundations, Methods, and Contemporary Issues Across the Lifespan. Washington, DC: American Psychological Association.

Deming, W. E. (1986). Out of the Crisis. Cambridge, MA: Massachusetts Institute of Technology, Center for Advanced Engineering Study.

Denham, S. A. (2002). Family routines: A structural perspective for viewing family health. Advances in Nursing Science, 24(4), 60–74.

De Pedro, K. T., Astor, R. A., Gilreath, T. D., Benbenishty, R., and Berkowitz, R. (2018). School climate, deployment, and mental health among students in military-connected schools. Youth & Society, 50(1), 93–115.

Domitrovich, C. E., Moore, J. E., Thompson, R. A., and CASEL Preschool to Elementary School Social and Emotional Learning Assessment Workgroup. (2012). Interventions that promote social-emotional learning in young children. In Robert C. Pianta (Ed.), Handbook of Early Childhood Education (pp. 393-415). New York: The Guilford Press.

Durlak J. (2013). The Importance of Quality Implementation for Research, Practice, and Policy. ASPE Research Brief. Washington, DC: U.S. Department of Health and Human Services.

Durlak, J. A., and Wells, A. M. (1997). Primary prevention mental health programs for children and adolescents: A meta-analytic review. American Journal of Community Psychology, 25(2), 115–152.

Dworkin, S. L., Pinto, R. M., Hunter, J., Rapkin, B. and Remien, R. H. (2008). Keeping the spirit of community partnerships alive in the scale-up of HIV/AIDS prevention: Critical reflections on the roll-out of DEBI (Diffusion of Effective Behavioral Interventions). American Journal of Community Psychology, 42(1–2), 51–59.

Eccles, J. S., and Gootman, J. A. (2002). Features of positive developmental settings. Community Programs to Promote Youth Development, 86–118.

Elwafi, H. M., Witkiewitz, K., Mallik, S., Thornhill IV, T. A. and Brewer, J. A. (2013). Mindfulness training for smoking cessation: Moderation of the relationship between craving and cigarette use. Drug and Alcohol Dependence, 130(1–3), 222–229.

Embry, D. D., and Biglan, A. (2008). Evidence-based kernels: Fundamental units of behavioral influence. Clinical Child and Family Psychology Review, 11(3), 75–113.

Ertin, E., Stohs, N., Kumar, S., Raij, A., al’Absi, M., and Shah, S. (2011, November). AutoSense: unobtrusively wearable sensor suite for inferring the onset, causality, and consequences of stress in the field. In Proceedings of the 9th ACM Conference on Embedded Networked Sensor Systems (pp. 274-287). New York: Association for Computing Machinery.

Farr, C. (2018). Alphabet spinoff Cityblock raises $20 million to help low-income Americans get health care. CNBC, January 4. Retrieved from https://www.cnbc.com/2018/01/04/alphabet-spin-off-cityblock-raises-20m-for-low-income-health-care.html.

Ferlie, E., and Dopson S.(2006). Studying complex organizations in health care. In S. Dopson and L. A. Fitzgerald (Eds.), Knowledge to Action? Evidence-Based Health Care in Context (pp. 8-25). Oxford, UK: Oxford University Press.

Fixsen, D. L., Naoom, S. F., Blase, K. A., and Friedman, R. M. (2005). Implementation Research: A Synthesis of the Literature. FHMI Publication no. 231. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network.

Free, C., Knight, R., Robertson, S., Whittaker, R., Edwards, P., Zhou, W., Rodgers, A., Cairns, J., Kenward, M. G., and Roberts, I. (2011). Smoking cessation support delivered via mobile phone text messaging (txt2stop): A single-blind, randomised trial. The Lancet, 378(9785), 49–55.

Garcia, E., De Pedro, K. T., Astor, R. A., Lester, P., and Benbenishty, R. (2015). FOCUS school-based skill-building groups: Training and implementation. Journal of Social Work Education, 51(sup1), 102–116.

Gewirtz, A. H., DeGarmo, D. S., and Zamir, O. (2016). Effects of a military parenting program on parental distress and suicidal ideation: After deployment adaptive parenting tools. Suicide and Life Threatening Behavior, 46(1), S23–S31.

________. (2018a). Testing a military family stress model. Family Process, 57(2), 415–431.

________. (2018b). After deployment, adaptive parenting tools: One-year outcomes of a parenting program for military families. Prevention Science, 19, 589k–599k.

Gewirtz, A. H., Pinna, K. L., Hanson, S. K., and Brockberg, D. (2014). Promoting parenting to support reintegrating military families: After deployment, adaptive parenting tools. Psychological Services, 11(1), 31–40.

Ghandeharioun, A., McDuff, D., Czerwinski, M., and Rowan, K. (2018). EMMA: An Emotionally Intelligent Personal Assistant for Improving Wellbeing. Cambridge, MA: MIT Media Lab.

Gifford, T., Taylor, E., Morgan, J., Nessen, S., Freedman, B., Boedeker, D., and Boedeker, B. (2014). Establishing a low-cost telecommunications method to provide tele ENT consultations from a military medical center to deployed locations. Journal of the International Society for Telemedicine and eHealth, 2, 50–53.

Glasgow, R. E., and Chambers, D. (2012). Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science. Clinical and Translational Science, 5(1), 48–55.

Gleason, J. L., and Beck, K. H. (2017). Examining associations between relocation, continuity of care, and patient satisfaction in military spouses. Military Medicine, 182(5-6), e1657-e1664.

Goldberg, S. B., Del Re, A. C., Hoyt, W. T., and Davis, J. M. (2014). The secret ingredient in mindfulness interventions? A case for practice quality over quantity. Journal of Counseling Psychology, 61(3), 491.

Granville, K. (2018). Facebook and Cambridge Analytica: What you need to know as fallout widens. The New York Times, March 19. Retrieved from https://www.nytimes.com/2018/03/19/technology/facebook-cambridge-analytica-explained.html.

Gruen, R. L., Elliott, J. H., Nolan, M. L., Lawton, P. D., Parkhill, A., McLaren, C. J., and Lavis, J. N. (2008). Sustainability science: An integrated approach for health-programme planning. The Lancet, 372(9649), 1579–1589.

Gustafson, D. H., McTavish, F. M., Chih, M. Y., Atwood, A. K., Johnson, R. A., Boyle, M. G., Levy, M. S., Driscoll, H., Chisholm, S. M., Dillenburg, L. and Isham, A. (2014). A smartphone application to support recovery from alcoholism: A randomized clinical trial. Journal of the American Medical Association Psychiatry, 71(5), 566–572.

Haberer, J. E., Sabin, L., Amico, K. R., Orrell, C., Galárraga, O., Tsai, A. C., Vreeman, R. C., Wilson, I., Sam-Agudu, N. A., Blaschke, T. F., and Vrijens, B. (2017). Improving antiretroviral therapy adherence in resource-limited settings at scale: A discussion of interventions and recommendations. Journal of the International AIDS Society, 20(1), 21371.

Hallett, T. B., White, P. J., and Garnett, G. P. (2007). Appropriate evaluation of HIV prevention interventions: From experiment to full-scale implementation. Sexually Transmitted Infections, 83(suppl 1), i55–i60.

Hardy, D. F., Power, T. G., and Jaedicke, S. (1993). Examining the relation of parenting to children’s coping with everyday stress. Child Development, 64(6), 1829–1841.

Hashem, I. A. T., Yaqoob, I., Anuar, N. B., Mokhtar, S., Gani, A. and Khan, S. U. (2015). The rise of “big data” on cloud computing: Review and open research issues. Information Systems, 47, 98–115.

Hawkins, J. D., Catalano, R. F., and Arthur, M. W. (2002). Promoting science-based prevention in communities. Addictive Behaviors, 27, 951–976.

Hawkins, J. D., Catalano, R. F., and Miller, J. Y. (1992). Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance abuse prevention. Psychological Bulletin, 112(1), 64.

Heyman, R. E., and Smith Slep, A. M. (2001). Risk factors for family violence: Introduction to the special series. Aggression and Violent Behavior, 6(2–3), 115–119.

Heyman, R. E., Smith Slep, A. M., and Nelson, J. P. (2011). Empirically guided community intervention for partner abuse, child maltreatment, suicidality, and substance misuse. In S. M. Wadsworth and D. Riggs (Eds.), Risk and Resilience in U.S. Military Families (pp. 85–107). New York, NY: Springer.

Hoagwood, K. E., Serene Olin, S., Kerker, B. D., Kratochwill, T. R., Crowe, M., and Saka, N., (2007). Empirically based school interventions targeted at academic and mental health functioning. Journal of Emotional and Behavioral Disorders, 15(2), 66–92.

Ijadi-Maghsoodi, R., Marlotte, L., Garcia, E., Aralis, H., Lester, P., Escudero, P. and Kataoka, S., (2017). Adapting and implementing a school-based resilience-building curriculum among low-income racial and ethnic minority students. Contemporary School Psychology, 21(3), 223–239.

Institute of Medicine (IOM). (2000). From Neurons to Neighborhoods: The Science of Early Childhood Development. Washington, DC: National Academy Press.

________. (2011). Digital Infrastructure for the Learning Health System: The Foundation for Continuous Improvement in Health and Health Care: Workshop Series Summary. Washington, DC: The National Academies Press.

________. (2013). Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: The National Academies Press.

________. (2014). Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press.

Jai, T. M., McCool, B. N., and Reed, D. B. (2016). Military parents’ personal technology usage and interest in e-health information for obesity prevention. Telemedicine and e-Health, 22(3), 183–190.

Johnson, K., Hays, C., Center, H., and Daley, C. (2004). Building capacity and sustainable prevention innovations: A sustainability planning model. Evaluation Program Planning, 27, 135–149.

Jones, B. L., and Fiese, B. H. (2014). Parent routines, child routines, and family demographics associated with obesity in parents and preschool-aged children. Frontiers in Psychology, 5, 374.

Kardan-Souraki, M., Hamzehgardeshi, Z., Asadpour, I., Mohammadpour, R. A. and Khani, S., (2016). A review of marital intimacy-enhancing interventions among married individuals. Global Journal of Health Science, 8(8), 74.

Karnik, N. (2018). Memo prepared for the Committee on the Well-Being of Military Families.

Karre, J. K., Perkins, D. F., Aronson, K. R., DiNallo, J., Kyler, S. J., Olson, J., and Mentzer, C. E. (2017). A continuum of evidence on evidence-based programs: A new resource for use in military social service delivery. Military Behavioral Health, 5(4), 346–355.

Kazdin, A. E., and Blase, S. L. (2011). Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspectives on Psychological Science, 6(1), 21–37.

Kelly, B. (2012). Implementation science for psychology in education. In B. Kelly and D. F. Perkins (Eds.), Handbook of Implementation Science for Psychology in Education (pp. 3–12). London, UK: Cambridge Press.

________. (2013). Implementing implementation science: Reviewing the quest to develop methods and frameworks for effective implementation. Journal of Neurology and Psychology, 1.

King, A. C., Hekler, E. B., Grieco, L. A., Winter, S. J., Sheats, J. L., Buman, M. P., Banerjee, B., Robinson, T. N. and Cirimele, J. (2013). Harnessing different motivational frames via mobile phones to promote daily physical activity and reduce sedentary behavior in aging adults. PloS one, 8(4), e62613.

Kristensen, T. S. (1996). Job stress and cardiovascular disease: a theoretic critical review. Journal of occupational health psychology, 1(3), 246–260.

Kumar, S., Nilsen, W. J., Abernethy, A., Atienza, A., Patrick, K., Pavel, M., Riley, W. T., Shar, A., Spring, B., Spruijt-Metz, D., and Hedeker, D. (2013). Mobile health technology evaluation: The mHealth evidence workshop. American Journal of Preventive Medicine, 45(2), 228–236.

Kumpfer, K. L., Alvarado, R., Smith, P., and Bellamy, N. (2002). Cultural sensitivity and adaptation in family-based prevention interventions. Prevention Science, 3(3), 241–246.

Lam, C. Y., Businelle, M. S., Aigner, C. J., McClure, J. B., Cofta-Woerpel, L., Cinciripini, P. M., and Wetter, D. W. (2014). Individual and combined effects of multiple high-risk triggers on postcessation smoking urge and lapse. Nicotine and Tobacco Research, 16(5), 569–575.

Langley, G., Moen, R., Nolan, K., Nolan, T., Norman, C., and Provost, L. (2009). The Improvement Guide: A Practical Approach to Enhancing Organizational Performance (2nd ed.). San Francisco, CA: Jossey-Bass.

Lara-Cinisomo, S., Chandra, A., Burns, R. M., Jaycox, L. H., Tanielian, T., Ruder, T., and Han, B. (2012). A mixed-method approach to understanding the experiences of non-deployed military caregivers. Maternal and Child Health Journal, 16(2), 374–384.

Layne, C. M., Saltzman, W. R., Poppleton, L., Burlingame, G. M., Pasalic, A., Durakovic, E., Music, M., and Pynoos, R. S. (2008). Effectiveness of a school-based group psychotherapy program for war-exposed adolesecents: A randomized controlled trial. Journal of the American Academy of Child & Adolescent Psychiatry, 47, 1048–1062.

Lester P., Liang L. J., Milburn, N., Mogil, C., Woodward, K., Nash, W., and Beardslee, W. (2016). Evaluation of a family-centered preventive intervention for military families: Parent and child longitudinal outcomes. Journal of the American Academy of Child & Adolescent Psychiatry, 55(1), 14–24.

Lester, P., Mogil, C., Saltzman, W., Woodward, K., Nash, W., Leskin, G., Bursch, B., Green, S., Pynoos, R., and Beardslee, W. (2011). Families overcoming under stress: Implementing family-centered prevention for military families facing wartime deployments and combat operational stress. Military Medicine, 176(1), 19–25.

Lester, P., Peterson, K., Reeves, J., Knauss, L., Glover, D., Mogil, C., Duan, N., Saltzman, W., Pynoos, R., Wilt, K., and Beardslee, W. (2010). The long war and parental combat deployment: Effects on military children and at-home spouses. Journal of the American Academy of Child & Adolescent Psychiatry, 49(4), 310–320.

Lochman, J. E., and van den Steenhoven, A. (2002). Family-based approaches to substance abuse prevention. Journal of Primary Prevention, 23(1), 49–114.

Luxton, D. D., Pruitt, L. D., Wagner, A., Smolenski, D. J., Jenkins-Guarnieri, M. A., and Gahm, G. (2016). Home-based telebehavioral health for US military personnel and veterans with depression: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 84(11), 923.

MacDermid Wadsworth, S., and Riggs, D. (Eds.) (2010). Risk and Resilience in U.S. Military Families. New York, NY: Springer Science & Business Media.

Marmor, S. (2018). Well-being of Military Families and a Learning Health System: The Importance of Data-Driven Decision Making. Paper commissioned by the Committee on the Well-Being of Military Families. Washington, DC: The National Academies of Sciences, Engineering, and Medicine.

Marshall, R. C., Doperak, M., Milner, M., Motsinger, C., Newton, T., Padden, M., Pastoor, S., Hughes, C. L., LeFurgy, J., and Mun, S. K. (2011). Patient-centered medical home: An emerging primary care model and the military health system. Military Medicine, 176(11), 1253-1259.

Martire, L. M., Schulz, R., Helgeson, V. S., Small, B. J., and Saghafi, E. M. (2010). Review and meta-analysis of couple-oriented interventions for chronic illness. Annals of Behavioral Medicine, 40(3), 325–342.

McInnes, D. K., Sawh, L., Petrakis, B. A., Rao, S. R., Shimada, S. L., Eyrich-Garg, K. M., Gifford, A. L., Anaya, H. D., and Smelson, D. A. (2014). The potential for health-related uses of mobile phones and internet with homeless veterans: Results from a multisite survey. Telemedicine and e-Health, 20(9), 801–809.

McKay, J. R., Van Horn, D. H., Oslin, D. W., Lynch, K. G., Ivey, M., Ward, K., Drapkin, M. L., Becher, J. R., and Coviello, D. M. (2010). A randomized trial of extended telephone-based continuing care for alcohol dependence: Within-treatment substance use outcomes. Journal of Consulting and Clinical Psychology, 78(6), 912.

McKibbon, K. A., Lokker, C., Wilczynski, N. L., Ciliska, D., Dobbins, M., Davis, D. A., Haynes, R. B., and Straus, S. E. (2010). A cross-sectional study of the number and frequency of terms used to refer to knowledge translation in a body of health literature in 2006: A Tower of Babel? Implementation Science, 5(1), 16.

Means, S. N., Magura, S., Burkhardt, J. T., Schröter, D. C., and Coryn, C. L. (2015). Comparing rating paradigms for evidence-based program registers in behavioral health: Evidentiary criteria and implications for assessing programs. Evaluation and Program Planning, 48, 100–116.

Merolla, A. J. (2010). Relational maintenance during military deployment: Perspectives of wives of deployed U.S. soldiers. Journal of Applied Communication Research, 38(1), 4–26.

Mestre, J. M., Núñez-Lozano, J. M., Gómez-Molinero, R., Zayas, A., and Guil, R. (2017). Emotion regulation ability and resilience in a sample of adolescents from a suburban area. Frontiers in Psychology, 8, 1980.

Miller, C. J., McInnes, D. K., Stolzmann, K., and Bauer, M.S. (2016). Interest in use of technology for healthcare among veterans receiving treatment for mental health. Telemedicine and e-Health, 22(10), 847–854.

Mogil C., Aralis, H., Paley, B., Hajal, N., Aralis, H., Milburn, N., Beardslee, W., and Lester, P. (In Review). An in-home telehealth preventive intervention for military connected families with young children: 12 month outcomes from a randomized control trial.

Mohr, D. C., Schueller, S. M., Riley, W. T., Brown, C. H., Cuijpers, P., Duan, N., Kwasny, M. J., Stiles-Shields, C., and Cheung, K. (2015). Trials of intervention principles: Evaluation methods for evolving behavioral intervention technologies. Journal of Medical Internet Research, 17(7), e166.

Monasta, L., Batty, G. D., Cattaneo, A., Lutje, V., Ronfani, L., Van Lenthe, F. J., and Brug, J. (2010). Early-life determinants of overweight and obesity: A review of systematic reviews. Obesity Reviews, 11(10), 695–708.

Monk, J. K., Oseland, L. M., Nelson Goff, B. S., Ogolsky, B. G., and Summers, K. (2017). Integrative intensive retreats for veteran couples and families: A pilot study assessing change in relationship adjustment, posttraumatic growth, and trauma symptoms. Journal of Marital and Family Therapy, 43(3), 448–462.

Morgan, N. R., Davis, K. D., Richardson, C., and Perkins, D. F. (2018). Common components analysis: An adapted approach for evaluating programs. Evaluation and Program Planning, 67, 1–9.

Muessig, K. E., LeGrand, S., Horvath, K. J., Bauermeister, J. A., and Hightow-Weidman, L. B. (2017). Recent mobile health interventions to support medication adherence among HIV-positive MSM. Current Opinion in HIV and AIDS, 12(5), 432–441.

Nahum-Shani, I., Hekler, E. B., and Spruijt-Metz, D. (2015). Building health behavior models to guide the development of just-in-time adaptive interventions: A pragmatic framework. Health Psychology, 34(S), 1209.

Nahum-Shani, I., and Militello, L.K. (2018). Promoting Military Family Well-Being with Digitally-Supported Adaptive and Just-In-Time Adaptive Interventions: Opportunities and Challenges. Paper commissioned by the Committee on the Well-Being of Military Families. Washington, DC: The National Academies of Sciences, Engineering, and Medicine.

Nahum-Shani, I., Smith, S. N., Spring, B. J., Collins, L. M., Witkiewitz, K., Tewari, A., and Murphy, S. A. (2017). Just-in-time adaptive interventions (JITAIs) in mobile health: Key components and design principles for ongoing health behavior support. Annals of Behavioral Medicine, 52(6), 446–462.

Najera, E., Landoll, R. R., Pollock, L. D., Berman, M., Ellis, K., Knies, K. M., Seidler, D. A., Bartone, P. T., and Bowles, S. V. (2017). Stress and resilience in married military couples. In S. Bowles and P. Bartone (Eds.), Handbook of Military Psychology (pp. 157–175). Cham: Springer.

National Military Family Association. (2011). Finding Common Ground: A Toolkit for Communities Supporting Military Families. Alexandria, VA. Retrieved from https://www.militaryfamily.org/wp-content/uploads/Finding_Common_Ground__A_Toolkit_for_Communities_Serving.pdf.

National Research Council and Institute of Medicine (NRC and IOM). (2009a). Depression in Parents, Parenting And Children: Opportunities to Improve Identification, Treatment, and Prevention Efforts. Washington, DC: The National Academies Press.

National Research Council and Institute of Medicine (NRC and IOM). (2009b). Preventing Mental, Emotional, and Behavioral Disorders Among Young People: Progress and Possibilities. Washington, DC: The National Academies Press.

North Atlantic Treaty Organization. (2019). Impact of Military Life on Children from Military Families. Technical report STO-TR-HFM-258. S&T Organisation, NATO.

Nielsen, L., Riddle, M., King, J. W., Aklin, W. M., Chen, W., Clark, D., Collier, E., Czajkowski, S., Esposito, L., Ferrer, R. and Green, P. (2018). The NIH Science of Behavior Change Program: Transforming the science through a focus on mechanisms of change. Behaviour Research and Therapy, 101, 3–11.

Olds, D. L., Henderson Jr., C. R., Tatelbaum, R., and Chamberlin, R. (1988). Improving the life-course development of socially disadvantaged mothers: A randomized trial of nurse home visitation. American Journal of Public Health, 78(11), 1436–1445.

Padesky, C. A., and Mooney, K. A. (2012). Strengths-based cognitive-behavioural therapy: A four-step model to build resilience. Clinical Psychology Psychotherapy, 19(4), 283–290.

Pawson, R., Greenhalgh, T., Harvey, G., and Walshe, K. (2005). Realist review-A new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy, 10(1), 21–34.

Pellowski, J. A., and Kalichman, S. C. (2012). Recent advances (2011-2012) in technology-delivered interventions for people living with HIV. Current HIV/AIDS Reports, 9(4), 326–334.

Pentz, M. A. (2003). Evidence-based prevention: Characteristics, impact, and future direction. Journal of Psychoactive Drugs, 35(1), 143–152.

Perkins, D. F., Aronson, K. R., Karre, J., Kyler, S. J., and DiNallo, J. M. (2015). Reducing barriers to evidence-based practice with military families: Clearinghouse for Military Family Readiness. Military Behavioral Health, 4(1), 47–57.

Pew Research Center (2014). The Future of Privacy. Washington, DC.

Piehler, T. F., Ausherbauer, K., Gewirtz, A. and Gliske, K., 2018. Improving child peer adjustment in military families through parent training: The mediational role of parental locus of control. The Journal of Early Adolescence, 38(9), 1322–1343.

Posner, J., Russell, J. A., and Peterson, B. S. (2005). The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and Psychopathology, 17(3), 715–734.

Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C., Glass, J. E., and York, J. L. (2012). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69(2), 123–157.

Proctor, E. K., Powell, B. J., and McMillen, J. C. (2013). Implementation strategies: Recommendations for specifying and reporting. Implementation Science, 8(139).

Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., and Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38(2), 65–76.

Rabin, B. A., Brownson, R. C., Haire-Joshu, D., Kreuter, M. W., and Weaver, N. L. (2008). A glossary for dissemination and implementation research in health. Journal of Public Health Management and Practice, 14(2), 117–123.

Raghupathi, W., and Raghupathi, V. (2014). Big data analytics in healthcare: Promise and potential. Health Information Science and Systems, 2(1), 3.

Rea, J., Behnke, A., Huff, N., and Allen, K. (2015). The role of online communication in the lives of military spouses. Contemporary Family Therapy, 37(3), 329–339.

Rhoades, B. L., Bumbarger, B. K., and Moore, J. E. (2012). The role of a state-level prevention support system in promoting high-quality implementation and sustainability of evidence-based programs. American Journal of Community Psychology, 50(3–4), 386–401.

Riggs, N. R., Chou, C. P., and Pentz, M. A. (2009). Preventing growth in amphetamine use: Long-term effects of the Midwestern Prevention Project (MPP) from early adolescence to early adulthood. Addiction, 104(10), 1691–1699.

Riley, W., Obermayer, J., and Jean-Mary, J. (2008). Internet and mobile phone text messaging intervention for college smokers. Journal of American College Health, 57(2), 245–248.

Rogers, E. M. (2010). Diffusion of Innovations. New York: Simon and Schuster.

Rosenblum, K. L., and Muzik, M. (2014). STRoNG intervention for military families with young children. Psychiatric Services, 65(3), 399–399.

Rotheram-Borus, M. J., Lee, M., Lin, Y. Y., and Lester, P. (2004). Six year intervention outcomes for adolescent children of parents with HIV. Archives of Pediatrics & Adolescent Medicine, 158, 742–748.

Rotheram-Borus, M. J., Swendeman, D., and Becker, K. D. (2014). Adapting evidence-based interventions using a common theory, practices, and principles. Journal of Clinical Child & Adolescent Psychology, 43(2), 229–243.

Rotheram-Borus, M. J., Swendeman, D., and Chorpita, B. F. (2012). Disruptive innovations for designing and diffusing evidence-based interventions. American Psychologist, 67(6), 463.

Saltzman, W. R., Lester, P., Beardslee, W. R., Layne, C. M., Woodward, K., Nash, W.P. (2011). Mechanisms of risk and resilience in military families: theoretical and empirical basis of a family-focused resilience enhancement program. Clinical Child and Family Psychology Review, 14(3), 213–230.

Saltzman, W. R., Lester, P., Milburn, N., Woodward, K., and Stein, J. (2016). Pathways of risk and resilience: Impact of a family resilience program on active-duty military parents. Family Process, 55(4), 633–646.

Santucci, L. C., Thomassin, K., Petrovic, L., and Weisz, J. R. (2015). Building evidence-based interventions for the youth, providers, and contexts of real-world mental-health care. Child Development Perspectives, 9(2), 67–73.

Scheirer, M. A. (2005). Is sustainability possible? A review and commentary on empirical studies of program sustainability. American Journal of Evaluation, 26, 320–347.

Scheirer, M. A., and Dearing, J. W., (2011). An agenda for research on the sustainability of public health programs. American Journal of Public Health, 101(11), 2059–2067.

Shore, J. H., Aldag, M., McVeigh, F. L., Hoover, R. L., Ciulla, R., and Fisher, A. (2014). Review of mobile health technology for military mental health. Military Medicine, 179(8), 865–878.

Siegenthaler, E., Munder, T., and Egger, M. (2012). Effect of preventive interventions in mentally ill parents on the mental health of the offspring: Systematic review and meta-analysis. Journal of the American Academy of Child & Adolescent Psychiatry, 51(1), 8–17.

Sogomonyan, F., and Cooper, J. L. (2010). Trauma Faced by Children of Military Families: What Every Policymaker Should Know. National Center for Children in Poverty. Retrieved from http://www.nccp.org/publications/pub_938.html.

Spagnola, M., and Fiese, B. H. (2007). Family routines and rituals: A context for development in the lives of young children. Infants & Young Children, 20(4), 284–299.

Spoth, R., Greenberg, M., Bierman, K., and Redmond, C. (2004). PROSPER community-university partnership model for public education systems: Capacity-building for evidence-based, competence-building prevention. Prevention Science, 5, 31–39.

Spoth, R., Greenberg, M., and Turrisi, R. (2008). Preventive interventions addressing underage drinking: State of the evidence and steps toward public health impact. Pediatrics, 121(4), S311–S336.

Spoth, R. L., Kavanagh, K. A., and Dishion, T. J (2002). Family-centered preventive intervention science: Toward benefits to larger populations of children, youth, and families. Prevention Science, 3(3), 145–152.

Spoth, R., Redmond, C., Clair, S., Shin, C., Greenberg, M., and Feinberg, M. (2011). Preventing substance misuse through community-university partnerships: Randomized controlled trial outcomes 4(1/2) years past baseline. American Journal of Preventive Medicine, 40, 440–447.

Spring, B. (2017). Sense2Stop: Mobile Sensor Data to Knowledge. National Institutes of Health. Retrieved from https://clinicaltrials.gov/ct2/show/NCT03184389.

Spruijt-Metz, D., and Nilsen, W. (2014). Dynamic models of behavior for just-in-time adaptive interventions. IEEE Pervasive Computing, 13(3), 13–17.

Steenkamp, M. M., and Litz, B. T. (2014) One-size-fits-all approach to PTSD in the VA not supported by the evidence. American Psychology, 69(7), 706–707.

Stein, B. D., Adams, A. S., and Chambers, D. A. (2016). A learning behavioral health care system: Opportunities to enhance research. Psychiatric Services, 67(9), 1019–1022.

Stirman, S. W., Miller, C. J., Toder, K., and Calloway, A. (2013). Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implementation Science, 8, 65.

Tanielian, T., Batka, C., and Meredith, L. S. (2018). Bridging gaps in mental health care: Lessons learned from the Welcome Back Veterans Initiative. Rand Health Quarterly, 7(4).

Teubert, D., and Pinquart, M. (2010). The association between coparenting and child adjustment: A meta-analysis. Parenting: Science and Practice, 10(4), 286–307.

Thomas, J. G., and Bond, D. S. (2015). Behavioral response to a just-in-time adaptive intervention (JITAI) to reduce sedentary behavior in obese adults: Implications for JITAI optimization. Health Psychology, 34S, 1261–1267.

Thompson, B. (2018). Department of Defense Military Family Readiness System: Supporting Military Family Well-Being. Paper commissioned by the Committee on the Well-Being of Military Families, Washington, DC, The National Academies of Sciences, Engineering, and Medicine.

Tibbits, M. K., Bumbarger, B. K., Kyler, S. J., and Perkins, D. F. (2010). Sustaining evidence-based interventions under realworld conditions: Results from a large-scale diffusion project. Prevention Science, 11(3), 252–262

Turner, K. M., and Sanders, M. R. (2006). Dissemination of evidence-based parenting and family support strategies: Learning from the Triple P—Positive Parenting Program system approach. Aggression and Violent Behavior, 11(2), 176–193.

Verdeli, H., Baily, C., Vousoura, E., Belser, A., Singla, D., and Manos, G. (2011). The case for treating depression in military spouses. Journal Family Psychology, 25(4), 488–496.

Weist, M. D., and Murray, M (Eds.). (2008). Advances in mental health promotion. Advances in Mental Health Promotion 1(3). Stafford, UK: The Clifford Beers Foundation.

Weisz, J. R., Jensen-Doss, A., and Hawley, K. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. American Psychologist, 61(7), 671.

Weisz, J. R., Krumholz, L. S., Santucci, L., Thomassin, K., and Ng, M. Y. (2015). Shrinking the gap between research and practice: Tailoring and testing youth psychotherapies in clinical care contexts. Annual Review of Clinical Psychology, 11, 139–163.

Weisz, J. R., Ugueto, A. M., Cheron, D. M., and Herren, J. (2013). Evidence-based youth psychotherapy in the mental health ecosystem. Journal of Clinical Child & Adolescent Psychology, 42(2), 274–286.

Whitestone, Y. K., and Thompson, B. A. (2016). How do military family policies influence parenting resources available to families? In A. H. Gewirtz and A. M. Youssef (Eds.), Parenting and Children’s Resilience in Military Families. Risk and Resilience in Military and Veteran Families (pp. 283–297). Cham, Switzerland: Springer.

Wike, T. L., Bledsoe, S. E., Manuel, J. I., Despard, M., Johnson, L. V., Bellamy, J. L., and Killian-Farrell, C. (2014). Evidence-based practice in social work: Challenges and opportunities for clinicians and organizations. Clinical Social Work Journal, 42(2), 161–170.

Wilson, D. B., Gottfredson, D. C., and Najaka, S. S. (2001). School-based prevention of problem behaviors: A meta-analysis. Journal of Quantitative Criminology, 17(3), 247–272.

Wilson, S. J., and Lipsey, M. W. (2007). School-based interventions for aggressive and disruptive behavior: Update of a meta-analysis. American Journal of Preventive Medicine, 33(2), S130–S143.

Wilson, S. J., Lipsey, M. W., and Derzon, J. H. (2003). The effects of school-based intervention programs on aggressive behavior: A meta-analysis. Journal of Consulting and Clinical Psychology, 71(1), 136.

Witkiewitz, K., Desai, S. A., Bowen, S., Leigh, B.C., Kirouac, M., and Larimer, M. E. (2014). Development and evaluation of a mobile intervention for heavy drinking and smoking among college students. Psychology of Addictive Behaviors, 28(3), 639–650.

Yang, J. L., and Jayakumar, A. (2014). Target says up to 70 million more customers were hit by December data breach. Washington Post, January 10. Retrieved from https://www.washingtonpost.com/business/economy/target-says-70-million-customers-were-hit-by-dec-data-breach-more-than-first-reported/2014/01/10/0ada1026-79fe-11e3-8963-b4b654bcc9b2_story.html.

__________________

This page intentionally left blank.