WHERE AND HOW YOUNG PEOPLE LEARN STEM
Over the past decade, there has been a fundamental change in the way that learning is organized and supported. As family work patterns shift, children and youths are spending more time in supervised educational programs before and after school, on weekends, and during summers and other holidays.1 At the same time, more children and youths regularly access on-demand digital learning resources and opportunities, including online communities and resource collections. Thus, education can no longer be defined solely by what happens in a schoolroom. Indeed, a substantial body of research demonstrates that deep learning develops across multiple settings and timeframes.2 What happens outside the classroom directly affects what is possible inside the classroom and vice versa.3
This report, funded by the National Science Foundation, provides guidance for designing and implementing out-of-school science, technology, engineering, and mathematics (STEM) learning opportunities for all young people (ages 5-18). The intended audiences of the report are local, state, and federal policy makers, out-of-school STEM program developers, and both classroom educators and out-of-school educators. To address the statement of task for this study, the report describes the role that out-of-school programs play in deepening and broadening young people’s
Statement of Task
An ad hoc committee will plan and conduct a public workshop to explore criteria for identifying highly successful practices in the area of STEM education in out-of-school settings, with a focus on designed settings and programs targeted at children and youth, through examination of a select set of examples. The committee will determine some initial criteria for nominating successful practices to be considered at the workshop. The examples included in the workshop must have been studied in enough detail to provide evidence to support claims of success. Discussions at the workshop will focus on refining criteria for success, exploring models of “best practice,” and an analysis of factors that evidence indicates lead to success. The discussion from the workshop will be synthesized and combined with a literature review of peer-reviewed and gray-literature publications for a short, committee-authored consensus report that would outline criteria for identifying effective out-of-school STEM settings and programs and identify those criteria for which data are readily available and those where further work is needed to develop appropriate data sources.
access to multiple, high-quality STEM learning opportunities. Such programs are important in building a STEM-engaged and STEM-literate society and workforce. The report focuses on STEM learning that occurs in out-of-school programs that are designed and led by adults, and structured for youths.a Included are afterschool programs, summer and weekend classes, and apprenticeship opportunities.b We identify the features of productive out-of-school STEM programs, review the evidence of the effects of out-of-school STEM programs, discuss the capacity needs of program staff, and provide a framework for improving evaluations.
Our conclusions and recommendations are based on a review of the literature on out-of-school STEM learning programs and practices, and, more broadly, on STEM learning. We also hosted a national summit on out-of-school STEM programs and commissioned a set of research reviews to gather critical information for the report: see Appendix A for the summit agenda and Appendix B for the list of commissioned papers. Although the committee reviewed all the evidence on learning STEM in out-of-school programs that we could identify, this study does not include a detailed literature review of that work because it is beyond the scope of this study.
Thinking Systemically About STEM Learning
Over the past decade, many policy makers, funders, communities, and educators have come together to align resources to enrich what has been called the STEM learning ecosystem.4 This phrase refers to the dynamic interaction among individual learners, diverse settings where learning occurs, and the community and culture in which they are embedded.5 A STEM learning ecosystem6 includes all of a community’s STEM-rich assets, which include
- designed settings, such as schools, clubs, museums, and youth programs;
- naturalistic settings, such as city parks, waterways, and forests and deserts;
- people and networks of people, such as practicing STEM professionals, educators, enthusiasts, hobbyists, and business leaders who can serve as inspiration and role models; and
- everyday encounters with STEM, such as on the Internet, on television, on the playground, or during conversations with family members and other young people.c
In a STEM learning ecosystem, children are at the center of the model because children are influenced directly by other people (e.g., family, friends) and settings (e.g., schools, neighborhoods) and indirectly by their environment and culture. In turn, children themselves shape and influence
_____________
aLearning opportunities that take place outside of school have been referred to in many ways, including informal learning, nonformal learning, life-long learning, out-of-school time learning, and free-choice learning. We use the term “out-of-school programs” to focus on the particular set of learning opportunities in our charge.
bWe exclude designed, unsupervised youth learning opportunities, such as television, radio, Internet, and social media projects.
cEveryday encounters are outside the focus of this report but are an important part of the STEM learning ecosystem.
the environment through their interests, dispositions, and values. Time is included in this model to illustrate that there are constant changes in children themselves and in the surrounding context. For example, the cognitive, emotional, social, and motivational qualities that young people bring to learning experiences are constantly evolving as they mature and accumulate experiences. Each learning experience has the potential to augment and be augmented by these qualities, leading to a dynamic interplay over time between the qualities of young people and those of learning environments.7 Thus, from an ecosystems perspective of STEM learning, connections among learners, community assets, and the broader culture are critical for supporting young people’s learning.
STEM learning ecosystem model.
NOTE: This representation, of the learning ecosystem model is based on Bronfenbrenner’s ecological model of human development first published in 1977. The innermost circle represents interactions that directly involve both child and an embedding context (e.g., child ↔ school). The next level shows connections among the immediately embedding contexts themselves. These also affect the child’s experiences (e.g., quality of family ↔ school interactions affect child ↔ school interactions). Influences from the increasingly distant layers influence the child’s experiences indirectly. The inclusion of time indicates that both the child and the surrounding contexts are constantly changing, and thus that learning is always a dynamic process.
SOURCE: Adapted from Liben, L.S. (June 2014). An Ecological Framework for STEM Learning. Presentation at the National Summit on Successful Out-of-School STEM Learning, June, National Academy of Sciences, Washington DC.
A systemic approach to education policy that aligns with the ecosystem perspective considers the range of learning opportunities across settings and times.8 Such an approach would ensure that learners have access to learning experiences that reflect and respond to young people’s interests and prior experiences and connect to additional opportunities.
Examples of Connected STEM Learning Opportunities
Educational leaders in some communities are making concerted efforts to identify, diversify, connect, and broker young people’s STEM learning opportunities across the learning ecosystem.
In the BRIDGE project (1996-2000),* New Mexico State University researchers worked with teachers across the school district to document how young people’s home and community activities incorporated mathematical skills and knowledge. Educators used this documentation to design school programs that used young people’s home skills and resources as starting points for academic work.
The Urban Advantage Program,** launched in 2004, led by the American Museum of Natural History in New York, is a collaboration between school districts and cultural institutions that encourages programmatic connections among family events, research using collections from museums and other informal settings, and the district-mandated 8th-grade exit project.
The HIVE project in Chicago*** was launched in 2012 to identify and connect the broad range of out-of-school programs available for youths to help families and youths locate interesting programs and to help programs broker ongoing opportunities for youths.
A particular strength of such coordinated efforts is to engage a more inclusive range of children in STEM, and to sustain their interest, participation, and learning over time.
_____________
*See http://math.arizona.edu/~bridge/ [May 2015].
**See http://www.urbanadvantagenyc.org/ [May 2015].
***See http://hivechicago.org/ [May 2015].
The Importance of Out-of-School Programs for STEM Learning
Following Successful K-12 STEM Education,9 we identify three long-term, interrelated goals of STEM education: (1) increasing advanced training and careers in STEM fields; (2) expanding the STEM-capable workforce who serve as STEM educators, science communicators, medical assistants, computer technicians, and other STEM-related careers; and (3) increasing scientific literacy among all young people, supporting life-long interest and engagement with STEM. These long-term goals consist of many intermediate- and short-term goals, including learners’ participating in STEM practices, developing learners’ positive dispositions toward STEM, and creating social settings that promote life-long STEM learning. It is important to stress that STEM literacy is defined as involving far more than conceptual knowledge and skills: it also involves interest, reasoning, and understanding of real-world relevance.10 These aspects of STEM literacy are not secondary goals: they are intrinsic and intertwined with understanding and engaging with STEM.
Although the majority of reform efforts that address the three broad goals of STEM education have focused on schools, children of school age spend only 20 percent of their waking hours in schools; the other 80 percent is spent outside of school, including in supervised out-of-school programs that meet after school hours, on weekends, and during the summer.11 Strategies that support STEM learning, such as hands-on learning experiences, inquiry-based pedagogy, and connecting STEM to everyday life are widely applied in many out-of-school STEM programs.12 Furthermore, out-of-school STEM programs leverage common structural features of out-of-school settings (e.g., hands-on activities, ungraded or unassessed activities, multiage groupings, fluid uses of time) to spark, sustain, and extend young people’s interest, developing understanding, and commitment to STEM.13 These findings suggest that STEM in out-of-school programs can be an important lever for implementing comprehensive and lasting improvements in STEM education.
The committee’s review of current research and practice confirms that the evidence about learning in out-of-school programs, while promising, is not yet robust or consistent. This is not surprising for several reasons. First, many out-of-school experiences are short term and their effects will occur over time and across settings. Consequently, it is difficult or impossible to collect the downstream evidence of the program’s impact. For example, the interest and skill developed in a science and engineering summer camp may later manifest itself in increased interest and achievement in an autumn science class, or at a different program the following summer. Second, because designers of out-of-school programs seek to engage, inspire, and broaden learning for young people partly by differentiating the programs
from schooling, most avoid implementing tests and other familiar short-term ways of monitoring young people’s learning. Third, the existing data on out-of-school programs frequently focus at the program level rather than the individual level. The program measures tend to be as diverse, local, and nonstandardized as the programs themselves. This specificity allows local programs to understand the programs’ effect, but it simultaneously makes it difficult to aggregate the evidence across programs.
Historical Perspective on Evidence for Out-of-School STEM Learning
Out-of-school learning has a long history, dating back to the 18th century when institutions such as libraries, churches, and museums were seen as the main institutions concerned with public education.* However as an organized field, the out-of-school community is quite young. Recent years have seen an increase in research on how, when, where, and why children and youths learn across their days and over their lives. Although much has been learned, it is fair to say that much remains unknown.**
Our understanding of out-of-school STEM learning primarily comes from two forms of published knowledge—(1) studies that have been published in peer-reviewed academic journals, and (2) studies that are the result of internal or external evaluations of specific exhibitions or activities or other commissioned reports. Many research traditions and perspectives have contributed to what is known about out-of-school learning, including youth development, learning sciences, cognitive development, and informal learning.
Since 1980, research on informal STEM learning has increased dramatically. Investigations of STEM learning and engagement in out-of-school contexts have been published in many journals. There exists a substantial body of empirical work and scholarship that addresses the field of learning STEM in out-of-school contexts. However, there are notable gaps in the literature.
_____________
*Conn, S. (1998). Museums and American Intellectual Life: 1876-1926. Chicago, IL: University of Chicago Press.
**Peter, N. (2002). Outcomes and Research in Out-of-School Time Program Design. Philadelphia, PA: Best Practices Institute.
Although evidence of the effect of out-of-school programs is limited, a number of studies illustrate that out-of-school programs can contribute to young people’s understanding of and interest in STEM. Learning Science in Informal Environments: People, Places, and Pursuits14 details the many ways that learning STEM in out-of-school settings contributes to people’s engagement with and pursuit of science learning. For example, out-of-school programs are well positioned to broaden participation in STEM learning by providing inquiry-based STEM experiences not commonly available in underresourced schools typically located in low-income communities.15
Out-of-school programs are likely to be taught by adults in the local community, thus providing important role models and community connections that can encourage pursuit of STEM learning.16 In addition, the absence of high-stakes testing in out-of-school programs can allow for more flexibility and therefore inclusive approaches to STEM learning, which may encourage young people who do not yet see themselves as STEM learners.17 Consistent participation in out-of-school programs has also been linked to performance in school and career choice. For example, studies have found that consistent participation in out-of-school programs leads to a narrowing of the achievement gap between young people from low-income and high-income families, better attendance, and more enthusiastic participation in school.18 Retrospective and longitudinal studies of practicing scientists find that their experiences at home, in their community, or in other settings were at least as important as school for fueling their passion for and understanding of science.19
Out-of-school programs that contribute to the long-term, intermediate, and short-term goals of STEM education have three design features in common: they are engaging, responsive, and make connections across learning experiences. Engaging STEM learning experiences are an essential starting point. They attract children and their families by offering distinctive, well-designed activities that include interaction with STEM phenomena through visual media, the outdoors, hands-on explorations, exhibits, and other formats. Responsive programming taps into interests and understanding generated through prior experiences to optimize the relevance and accessibility of the program activities. Engaging and responsive programs can support inclusion, but their aggregate effects on young people require that they make connections across many learning opportunities. If a child deeply engages with engineering activities at home or in after-school programs, for example, but has no subsequent opportunity to build on those experiences in school or elsewhere, the longer-term value may be lost.20 These three design features of productive out-of-school STEM programs are the basis for the criteria for identifying productive programs, which is the subject of the second chapter of this report.
Meeting the Demand for Out-of-School STEM Programs
Achieving the three goals identified above is not simple, but there are many existing programs, settings, and opportunities that comprehensive improvement efforts could leverage. For example, the number of young people enrolled in after-school and summer programs has skyrocketed over the past decade: currently one in five children participate in such programs.21 Parents report that about 70 percent of the out-of-school programs available include STEM activities, and more than 50 percent of programs engage young people in STEM activities at least twice a week.22
Many organizations have been expanding STEM learning opportunities in out-of-school programs in recent years. For example, an increasing number of youth development organizations, such as 4-H, the Boy Scouts and Girls Scouts, and Boys and Girls Clubs embrace STEM as an important strategy for supporting youth in their intellectual, social, and emotional development. Expanded STEM learning opportunities can also be seen in the growth of citizen science programs and Makerspaces,d as well as an increased focus on STEM learning in public institutions, such as science centers, museums, and libraries.e Programs that focus on academic achievement and enrichment such as 21st Century Community Learning Centers also have begun to include STEM learning, with some 15 states currently having made STEM a priority focus.23 There has also been an increase in the number of environmental science, math, and engineering camps; habitat restoration projects; after-school hobbyist clubs on such topics as robotics and astronomy; and multiday expeditions—such as fossil-hunting trips—that provide STEM learning opportunities.
In addition to program-level expansion, there are now more than 40 statewide after-school networks that support coordinated approaches to after-school programs, including 17 with a specific focus on STEM.f Various governmental, private, and corporate funders have undertaken a range of efforts to build the capacity of youth organizations to provide more robust and inclusive STEM learning opportunities. They have done so by building a broad and overlapping infrastructure of elements to support coordinated and high-quality settings and programs and by encouraging greater coordination of learning opportunities among schools and across out-of-school settings.
Despite the increase in programs, only one-third of the national need for out-of-school programing is being met by existing programs.24 In addition, research has raised questions about the quality of STEM learning experiences in existing programs. A recent study of out-of-school programs in California found that most programs include STEM activities, but only a small proportion provide opportunities for youths to participate in inquiry-based STEM learning.25 Thus, there is a need to expand access to productive out-of-school STEM learning programs by improving existing programs and creating new ones.
_____________
dFor more information about the program, see http://www.makerspace.com/ [May 2015].
eFor more information, see the Institute for Museum and Library Service’s STEM efforts at http://www.imls.gov/about/stem.aspx [May 2015].
fFor more information, see http://www.statewideafterschoolnetworks.net/ [May 2015].
Developing an Infrastructure
Educators, funders, and governmental agencies have undertaken several notable efforts to create sustainable STEM learning infrastructure supports over the past two decades. Although none of these fully meet the national need, they illustrate a promising trend and foundation on which more comprehensive efforts can be built. The examples below illustrate those efforts.
Creating Statewide Coalitions for STEM Learning Opportunities Supported by the Charles Stewart Mott Foundation, over the past 12 years state representatives have been meeting annually to share strategies and undertake collective actions. These statewide after-school networks recently began to plan and build STEM-focused systems to provide more high-quality STEM learning opportunities that excite, engage, and inspire young people in their states. With additional support from the Noyce Foundation, the Mott state coalitions have worked to map STEM assets for after-school programs in their states, to leverage public and private funding, and to build good policies and practices to further after-school, summer, and expanded learning opportunities. For more information, see http://www.statewideafterschoolnetworks.net/ [May 2015].
Increasing Collaborations Between After-school Providers and Science Centers The Afterschool Alliance and the Association of Science-Technology Centers (ASTC) are working together to bring more high-quality STEM programs for young people to after-school programs. The initiative, announced in 2013 as a commitment to the Clinton Global Initiative, provides a series of conferences and meetings to create an “ASTC Community of Practice” that includes educators from science centers, museums, zoos, and planetariums and the providers of after-school programs to find ways to connect more ASTC members and after-school programs at the local level and to increase the quantity and quality of STEM in after-school programs nationwide. For more information, see http://www.astc.org/professional-development/communities-of-practice/ [May 2015].
Expanding the Reach of STEM Through Youth Organizations The Noyce Foundation in 2006 initiated a strategy to increase access to high-quality STEM learning opportunities through large national organizations whose leaders were already interested in providing science programs for children and youths but did not yet offer such programs on a large scale. For example, a series of grants to the National 4-H Council enabled 4-H to include substantial hands-on STEM programming for more than 1 million children and youths each year. Other grants to organizations, such as Girl Scouts, Girls Inc., and YMCA of the USA, have made it possible to reach millions more children and youths with high-quality STEM programs outside of school. For more information, see http://www.4-h.org/youth-development-programs/4-h-science-programs/ [May 2015], http://www.girlscouts.org/program/basics/science/ [May 2015], http://www.girlsinc.org/resources/programs/girls-inc-operation-smart.html [May 2015], and http://www.ymcanyc.org/association/pages/stem-science.-technology.-engineering.mathematics [May 2015].
Creating Measures of Out-of-School STEM Learning There are several initiatives under way to develop measures of learning in out-of-school programs. For example, with initial support from the Gordon and Betty Moore Foundation, a team at the Lawrence Hall of Science, SRI International, and the University of Pittsburgh have been working to identify the factors that distinguish children who lose interest in science when they get to middle school from those who go on to become active science learners in high school and beyond. In a related activity, the Science Learning Activation Lab has developed measures of interest, curiosity, motivation, reasoning, and persistence in science, as well as appreciation of the value of science, responsibility for learning, and identity as a science learner. Other efforts to develop measures are discussed in the third chapter of this report. For more information, see http://www.activationlab.org/ [May 2015].
Aligning Support for STEM Learning Opportunities The STEM Funders Network is composed of more than a dozen private foundations that support STEM in both schools and informal settings. Facilitated by the Teaching Institute for Excellence in STEM, representatives from each of the foundations meet periodically to share ideas and develop collaborative strategies so that together they can have a deeper and longer-lasting impact than any one foundation might have alone. For more information, see http://www.tiesteach.org/solutions/stem-network-design/ [May 2015].
Supporting the Field of Out-of-School STEM Learning Supported by the National Science Foundation, the Center for Advancement of Informal Science Education (CAISE) provides an infrastructure for the out-of-school STEM education field. CAISE provides resources for practitioners, researchers, evaluators, and STEM-based professionals. It also facilitates conversation, connection and collaboration, and hosts searchable repositories of programs, evaluation reports, peer-reviewed research, and unpublished reports. For more information, see http://informalscience.org/ [May 2015].
Building the Capacity of Science Centers The DeWitt-Wallace Reader’s Digest Fund, in collaboration with the Association of Science-Technology Centers, created Youth Alive! (Youth Achievement through Learning, Involvement, Volunteering, and Employment) to recruit teenagers from underserved local communities to work at science centers after school and during the summer. During its life from 1991 to 2001, 72 science centers received grants to start such programs, which soon became integral to the science centers’ missions. Ten years after funding ended, the number of science centers with such programs has grown to 163, with many positive effects for both science centers and teenagers.* Many science centers reported increased cultural sensitivity among staff and increased integration of the institution with the local community. Increased school attendance, academic aspirations, and interest in STEM careers were found among participating teenagers. Efforts to provide professional development, staff training, and community partnerships continue through the YouthAlive! Regional Networks, which were created in 2000. For information about the regional networks, see http://www.astc.org/professional-development/youth-program-networks/ [May 2015].
_____________
*Sneider, C.I., and Burke, M. (2011). The Legacy of YouthALIVE! Washington, DC: Center for the Advancement of Informal Science Education. Available: http://informalscience.org/images/research/SneiderandBurke_LegacyofYouthAlive.pdf [February 2015].
opportunities for youths to participate in inquiry-based STEM learning.25 Thus, there is a need to expand access to productive out-of-school STEM learning programs by improving existing programs and creating new ones.
Access to out-of-school STEM programs also remains a concern because STEM-rich out-of-school experiences are not evenly distributed.26 Many children have their out-of school time carefully orchestrated by parents and families, who enroll them in programs and lessons, take them to museums and parks, and involve them in hands-on activities at home. And many of these activities involve STEM learning, either directly or indirectly. Yet not all parents and families have the time, the resources, or the information needed to access community resources to strategically organize their children’s learning in activities outside of school. Policy makers can help address this basic inequity through policies that enrich, support, and expand high-quality STEM out-of-school learning programs.
