5
Learning Spaces Outside of School Time
Out-of-school time (OST) spaces comprise a vast range of environments and situations, including youth development programs, museums, libraries, zoos, botanical gardens, higher educational institutions, science centers, and community centers, as well as home-based and online communities. Given the substantial breadth and variability of these activities, this chapter focuses primarily on community-based organizations, museums, and public libraries, which are considered “educator-designed settings” of informal learning environments (National Research Council [NRC], 2009). This chapter discusses the potential roles and ways in which OST settings strive to provide authentic experiences that develop interest and competencies for computing to learners in grades K–12.
Community-based organizations often support a diverse set of initiatives that are core to their missions. Learning and engagement within these programs occur outside of school hours and may be held on school campuses, near school campuses, or elsewhere. The learning environments and experiences generated by these organizations are quite varied. Some of these programs are supported by a national umbrella organization or headquarters with a network of multiple sites (e.g., 4H, Boys & Girls Clubs, the Clubhouse Network, FIRST®, the SMASH Academy, and YWCA and YMCA to name a few). Other programs are single-site, unique, and standalone, sometimes tied to an overarching organization or funder (e.g., Mouse in New York City, DIY Girls in Los Angeles) and certain K–12 programs hosted by colleges and universities. Whether the programs have a national or state office or overseeing leadership structures, the programs on the ground take the shape of the region, community, and participants.
Whereas community-based organizations often provide a mixed suite of services for the community, it is not uncommon for their youth-serving efforts to be oriented toward science, technology, engineering, and mathematics (STEM), whether as a primary focus or integrated into their overall design. Less common and only recently introduced are youth programs from community-based organizations specifically focused on computing outcomes in their offerings. For example, Girl Scouts of the USA (GSUSA) released a large set of new badges and computing opportunities (e.g., Cybersecurity and Coding for Good).1 Another national organization, 4-H, created a 4-H CS Pathway that aims to support 4-H educators and facilitators, especially those in rural communities or areas with limited access to technology and computing resources, to teach computer science skills and coding projects.2
Museums—including science and technology centers, children’s museums, art museums, history museums, and even zoos, aquaria, arboretums, and botanical gardens—have evolved as institutions whose focus has shifted from curation or preservation to include a focus on learning and interactivity (Crowley, Pierroux, and Knutson, 2014; Falk and Dierking, 2018). Through educational programs, curricula, and exhibition topics, museums work to reach diverse communities, whether by geography, race and ethnicity, and/or socioeconomic level.3 Museums offerings are varied, including a combination of any of the following: collections and galleries, exhibits, school field trips, family workshops, teacher professional development, volunteer programs, summer camps, online content, etc. Although they are predominantly considered as informal/out-of-school space, museums have strong relationships with school institutions by aligning content with state and national standards (Center for the Future of Museums, 2014). Museums create a substantial amount of content as well, whether curricula, videos, games and apps, and/or other work related to research and evaluation efforts, often accessed and used globally by other museum settings, classrooms, professional development, and even directly by learners (Center for the Future of Museums, 2014).
In recent years, museums have begun to offer a range of computing-oriented programming, often embedded within their STEM portfolios and sometimes as a standalone effort. These programs may occur within a museum’s physical space or as part of community partnership work and
___________________
1 For more information, see https://blog.girlscouts.org/2019/07/big-news-42-new-girl-scoutbadges-to.html.
2 For more information about the initiative, see https://4-h.org/media/4-h-google-expandaccess-to-free-computer-science-education-to-one-million-youth-across-the-country. This includes efforts to introduce students to computer science: https://about.google/stories/google-4h as well as considering pathways: https://4hcspathways.extension.org.
3 For 2018 information, see https://www.aam-us.org/2018/01/20/museums-and-publicopinion.
include opportunities such as timed workshops, hands-on activities within interactive spaces on the museum floor (such as makerspaces or labs), classes coupled with field trips, outreach programs for specific youth, and/or collaborations with local institutions, including schools and after-school programs. For example, in San Francisco, the Exploratorium’s Tinkering Studio has developed activities and led workshops on Computational Tinkering.
Like museums, public libraries have evolved to meet the needs of their communities, including the K–12 population.4 In terms of programming5 and services for youth, two target populations are included: children (ages 11 and under) and young adults (ages 12–18). Public libraries offered 503,334 young adults’ programs and 2.85 million children’s programs in 2016 (Institute of Museum and Library Services [IMLS], 2019). IMLS (2019) also reports that the more rural the library is, the more children and young adult programs are offered. Libraries are deemed capable of filling the gap that schools are lacking in terms of providing opportunities related to computing, which may result in broadening the participation for youth of color in computing (Braun and Visser, 2017). It is important to note that public libraries vary tremendously in size, staffing structure, availability of resources, opportunities for professional development, and access to community partners, among other distinctions (IMLS, 2019; Real and Rose, 2017).
A variety of technology-enabled programs that leverage learner’s interest and promote equitable opportunities to use and experience technologies have been happening in public libraries for more than a decade (Braun et al., 2014; Subramaniam et al., 2018). A recent trend has been to offer programs tailored to the development of interest and competencies in STEM (Garmer, 2014; Martin, 2017) and to computing more specifically (ALA, 2019). The type of computing programs include topics such as robotics, gaming, making, learning specific coding languages, and app development, as well as other unplugged programs that are integrated into existing programs such as story times and reading programs. These programs may leverage design principles of the connected learning framework (Hoffman et al., 2016; Ito et al., 2013; Subramaniam et al., 2018) that strive to be authentic to youth, interest-driven, fun, and different from in-school settings. Based on available scholarship and data, some libraries (especially small and rural libraries) are likely contributing to cultivating interest and competencies for pursuing careers in computing by using authentic experiences (Phillips, Lee, and Recker, 2018); however, there is limited published empirical evidence.
___________________
4 It should be noted that although the committee is focusing on library programs outside of school settings, an important setting that could help with broader reach is computer labs in public schools.
5 The committee utilizes the phrase “library programs” or “library programming” to reflect the programs and activities offered in libraries. This is a common vernacular in library scholarship.
In its discussion of the ways in which these OST settings and their present and potential impact on interest and competencies for computing to learners in grades K–12, the chapter will emphasize the strengths and challenges that these settings face with respect to:
- access, equity, and cultural norms;
- duration and the format of programs;
- space and/or facility issues;
- learning facilitators; and
- measuring outcomes.
It is worth stating that the intention is not to address all types of computing programs (e.g., unplugged, game making) throughout the sections, but to use the available data and select examples to explain each of the above-mentioned factors by making connections to how these types of programs provide authentic experiences for computing. Moreover, when possible, the examples will focus on “authentic” experiences. However, a limitation of the available literature is understanding what is considered “authentic” in these contexts; in these cases, the committee draws from the broader literature.
ACCESS, EQUITY, AND CULTURAL NORMS
It is important to consider the degree to which the various settings afford opportunities to engage learners in authentic experiences for computing. However, it is worth noting that there are limitations to access and engagement beyond financial cost and access to resources (discussed below). For example, the learners’ home and family environments can have an impact, as when the presence or absence of supportive and/or knowledgeable family members can significantly impact learners’ experiences (Ito et al., 2020). The learner may have the resources to attend an OST program, but may have caregivers who are not willing to support the experience, or vice versa. Some learners must maintain an after-school job or care for their siblings. Moreover, informal experiences have affordances and constraints that play a specific role. What follows is a review of the ways in which issues related to access, equity, and cultural norms may facilitate or hinder the participation of learners in authentic experiences for computing and the ways in which OST institutions can potentially attend to these issues.
Distribution and Quality of Opportunities
Previous work has suggested that families from high socio-economic backgrounds spend nearly seven times more money on OST enrichment
activities than families from low socio-economic backgrounds (Duncan and Murnane, 2011; Lopez, Caspe, and McWilliams, 2016). It is clear that the distribution of these opportunities and the quality of opportunities are tied to various factors such as socio-economic status and geography. Although institutions such as public libraries (IMLS, 2019), 4-H, and others have substantial reach and offer quality programming for youth in rural areas, the strengths, challenges, and availability of resources in programs serving rural communities are distinct from those experienced by programs serving suburban or urban communities (Hartman, Hines-Bergmeier, and Klein, 2017; Ihrig et al., 2018). Differences in broadband access, physical space, and the socio-demographic characteristics of learners will undoubtedly affect the types of programs that facilitators can or choose to offer (Davis et al., 2018).
Moreover, across all settings, OST educational programs vary in terms of access and are inequitable, impacting minoritized communities who may need them most (Afterschool Alliance, 2014; 2016). For example, the Afterschool Alliance, a national non-profit organization that supports and advocates for quality after-school opportunities, notes in the 2014 (and 2016)6American After 3PM report that access to and supply of quality after-school programs is highly uneven for different communities and populations. There is a higher rate of participation and higher demand, though often unmet, for after-school opportunities among low-income households, African American families, and Hispanic/Latinx families, than compared with high-income households and White families. However, as suggested above, it is important to consider whether the learners’ home and family environments also serve as a potential barrier to participation.
Time, Cost, and Transportation
Significant barriers to accessing authentic experiences for computing include time availability, cost, and transportation. For example, Sirinides, Fink, and DuBois (2016) show that some public libraries’ hours of operations provide access challenges to families that work during the day and cannot get to the library until night—this is despite approximately 311 million people living with a public library service area (LSA) with free library programs. Some branches do not have regular evening hours during the week, and some smaller libraries change hours frequently. Some families opt to make a longer commute (if they can) to get to the central library, which may offer better resources, programs, and staff to work with or assist them than the local branch library (Sirinides, Fink, and DuBois, 2016).
___________________
6 See http://www.afterschoolalliance.org/press_archives/CCP-African-American-NR-083016.pdf.
In addition to the hours of operation, financial and family time commitments can be a major constraint in the opportunities afforded to learners. In a 2019 article, the National Conference of State Legislatures (NCSL) clearly states:
Only one dedicated federal funding stream, the 21st Century Community Learning Centers Program, is available for afterschool programs. Although 24 percent of the children in afterschool programs live in communities with concentrated poverty, federal funds cover only 11 percent of program costs. Therefore, the burden of funding such programs often falls on states, communities, and parents. (p. 1)
As an example, traditional Scouts programs require a substantial amount of voluntary time, and often money and resources, from families and caregivers. Troop leaders have been characterized as willing, impassioned caregivers, leading and supporting efforts on a volunteer basis, and troops may expect all families to contribute time, service, and resources.7 These norms and expectations, as well as the types of activities that troops engage in (e.g., campouts, community service, hiking, crafts, and sales), are reflective of the dominant culture, and access can be limited for those with different abilities, different schedules or availability (i.e., working an evening shift or weekend hours), different family units, and different values.
To offset some of the costs, institutions of interest, such as museums, often support complimentary admissions, reduced-price tickets, free school field trips, and a variety of other programs and services that support accessibility or learning needs. Organizations such as the Association of Science-Technology Centers (ASTC) have occasionally provided small grants to museums and science centers to subsidize the cost of providing transportation to enable youth to attend. Libraries have also found ways to confront this issue by obtaining bus passes or bringing programs to more accessible locations for learners such as community centers (Garmer, 2014; Zhou et al., 2019).
Technological Resources
The programs across the different settings are not always equipped with technologies that can facilitate authentic experiences for computing (e.g., computer or learning lab, a makerspace, a studio) and support (e.g., curriculum, materials) that may develop learners’ interest and competencies in computing (Hoffman et al., 2016). Some centers impose time limits on
___________________
7 See Pack 296 Cubs, Oakland, CA (2018). Available at http://www.pack296cubs.org/documents.
the use of tablets and computers. As a result, learners who do not have access to these technologies or only have mobile access at home may have to take a few days to complete their computing projects. These may result in learners falling behind in competency development or losing interest (Braun and Visser, 2017).
Families and learners living in poverty are more likely to visit a library than any other community venue, such as a museum (National Center for Education Statistics, 2015; Swan, 2014). This may be because nearly all public libraries offer free access to computers and wifi (American Library Association [ALA], 2019). Libraries also have ventured into unplugged programs to help learners understand computing concepts (see Box 5-1).
There are some potential approaches to offset lack of access to technological resources, beyond offering unplugged programs. A potential common strategy employed by libraries is to partner with organizations that
can provide the physical and intellectual access needed to design, develop, and implement authentic experiences. Collaborators can include local afterschool programs, higher educational institutions, state libraries that provide technical and staff assistance to libraries within the state, makers in the community, local schools, other informal learning spaces such as museums, businesses, non-profits, authors, and individual community members with technical expertise such as animation, coding, game design, and filmmaking (Subramaniam et al., 2018). Urban libraries may have a more extensive portfolio of partners to tap into than rural libraries, which often have trouble designating a partner (Davis et al., 2018).
Cultural Norms
OST settings also vary, as expected, in their norms and values, which are often drawn from the communities they serve and/or exist within. In some cases, OST programs may uphold cultural norms and values that are exclusionary, especially given the often private nature compared to the public nature of schools.8 However, in other instances as observed in the Clubhouse Network (originally known as the Intel Computer Clubhouse Network), these environments can “provide a creative, safe, and free OST learning environment where young people from underserved communities work with adult mentors to explore their own ideas, develop new skills, and build confidence in themselves through the use of technology.”9 As with any learning community, the members and actors of the clubhouse help to set the values, norms, and culture (Michalchik et al., 2008). Art—defined broadly, whether through music and rap, animation and video, poetry, drawing, or fashion design, just to name a few—is a way to express their own interests, identity, culture, and personality. Technology is specifically called out as a tool for expression, as the “tools provide an opportunity for learning of complex skills and, at the same time, the possibility of applying these skills in authentic situations across one’s life” (Michalchik et al., 2008, p. 32).
Museums and libraries have been engaged in debates for a long time around which communities and cultures they represent and how those are represented in their collection, programming, artifacts, and galleries showcased, and how they respond when there is a specific need in their communities (Gibson et al., 2017; 2020; Watson, 2007). Through new and refined initiatives addressing diversity, equity, accessibility, and inclusion in the museum and librarianship field overall, museums and libraries are examining their own practices, making clear the work needed for combating
___________________
8 In some instances, these programs may also have religious ties and uphold narrow views around topics such as gender (e.g., Boy Scouts).
9 For more information, see https://theclubhousenetwork.org.
implicit bias and enabling systemic change at all levels, and recognizing the need to remain relevant, responsive, and reflective of the communities in which they exist (see Box 5-2; ALA, 2019; American Alliance of Museums, 2018).
In addition, thinking about how communities are represented in their collections, museums and libraries also attend to how they can use their collections, spaces, and resources to serve members of those communities. Attending to the local communities can be achieved by leveraging the knowledge that program facilitators have about the learners in their communities gained through everyday interaction: for example, learners’ interests, their current challenges, the technologies they use, what they learn
in school, and challenges their families face. For example, the Los Angeles Public Library System offered Scratch programming workshops at 11 libraries serving underserved populations across the system (for a description of Scratch, see Box 7-7). Librarians reported learner interest in hip-hop, skateboarding, vampire and monster romance novels, Ninjago, Minecraft, etc. The fascination with hip-hop was incorporated into the design of the Scratch programming that the library offered. Learners used Scratch to combine and remix hip-hop dance moves by photographing themselves performing stop motion dance moves and remixing these moves with some of their friends’ movements in their Scratch projects (for another example, see Box 5-3).10 Another emerging way to design authentic experiences for computing is to incorporate “youth voice” in the design of such programs; this is a set of participatory design techniques derived from child-computer interaction research (Druin, 1999; Subramaniam, 2016; Yip, Lee, and Lee, 2019). Rubio (2017) uses the term “youth voice” to highlight and advocate for the partnership of younger learners and adults, built to facilitate deep participation in library programs and progressively build trust with underserved communities.
___________________
10 For more information, see https://scratch.mit.edu/codingforall.
DURATION AND PROGRAM FORMAT
Duration, frequency of visits, and overall accessibility to authentic experiences for computing have implications for how to design and implement these flexible and adaptable experiences across designed settings. As will be discussed in Chapter 7, it is important for designers and educators to take into consideration the quality and outcomes of an experience when learners may engage from 5 minutes to 60 minutes (or more), may engage independently or in a highly-social and collaborative manner, may be visiting the space for the first or fifteenth time, may come with deep knowledge and interest or little experience, and more. This can also include experiences across different settings. When there is variation in how much time is spent—and possible variation among learners as to the amount of time each spends—the facilitation, structures, and activities must remain flexible and adaptable. Moreover, the timeframe of a program—when in the year a learner might attend—may affect the learning or effectiveness as well. Some programs, such as the SMASH Academy program offered through the Kapor Center, are extensive in how long and consistently they engage with their participants (see Box 5-4).
The programs offered by community-based organizations, museums, and libraries can occur in various formats—such as in the form of summer or spring break camps, weekend programs, weekly programs that extend the whole year, season, or several weeks, and special one-time programs (see Box 5-5 for an example). Some of the programs may require mandatory attendance, though drop-in programming remains the most prevalent. For programs that have mandatory attendance, pre-registration is typically required, whereas for drop-in programming, educators may not necessarily know who will attend, how long they will stay, what prior knowledge they may bring, and how they will engage with other learners in the space. Drop-in programs may be able to expect a certain level of regularity, though regularity may look quite different from one program to another: twice a week for 2 hours each time, or every day for 30 minutes, or once a month on Saturdays for 3 hours with their families. This is important as learners have different reasons or motivations for engaging in authentic experiences.
Social interaction and peer learning are critical parts of authentic STEM experiences (NRC, 2009). For example, research focusing on museum experiences has suggested that socially engaging encounters are important in connecting the immediate experience to relevant interests and prior knowledge as well as transferring knowledge gained in the present to future experiences, regardless of how learners interact with others (Falk and Dierking, 2018). Research has also suggested that it is unlikely that meaningful social interaction can occur if learners are engaged in one-time classes or on a drop-in basis (Rubio, 2017), which may be a limitation
of some programs in particular settings (e.g., museums and libraries). To accommodate this, some libraries have begun to offer the same program multiple times a day during the week or extend it for longer hours so that learners and their families have opportunities to participate when they are available, especially when the curriculum involves authentic experiences where deep engagement with content is warranted (Lopez, Caspe, and McWilliams, 2016; Zhou et al., 2019).
Some libraries have proposed other efforts to provide opportunities for and maintain the much-needed deep engagement in authentic experiences. These include (1) providing meals and snacks; (2) encouraging regular participation while offering additional sessions such as “booster days,” which are drop-in sessions when learners and families who have attended the mandatory sessions can attend to extend their learning (Zhou et al., 2019); (3) offering bus passes to ensure that learners can attend a program
regularly (Garmer, 2014; Zhou et al., 2019); and (4) designing a program or a curriculum that allows for a particular participant to get up to speed when several sessions have been missed (Martin, 2017). Another important strategy promoted by an ecosystems approach may include brokering the connections across settings as it is important to not only engage learners for longer periods of time in a single setting, but also broker to and from different environments.
However, it is worth acknowledging that the research is mixed as to whether the amount of exposure (whether in duration or in frequency) or how that time is spent (the quality of the activities and engagement) has an impact on learning at all (Lauer et al., 2006). Specific programs do show that increased “dosage” is associated with increases in academic achievement and test scores (Wai et al., 2010) or other outcomes such as science interest (Noam et al., 2014). Ultimately, the goals of the learning experience are critical to consider when planning the expected duration or frequency of learning.
SPACE AND/OR FACILITY ISSUES
Physical space and facilities vary widely within and across the different designed settings.11 The National Clearinghouse for Educational Facilities notes that characteristics of academic facilities, such as ventilation and air quality, lighting, acoustics, temperature, and more, may have impacts on teacher and student performance (Schneider, 2002). These aspects are equally important to consider for OST spaces and facilities, though they may be more diverse and inconsistent in nature, from outdoor spaces in nature for environmental programs to computer labs and workshops. For authentic experiences that increase interest in computing, access to and availability of appropriate devices or tools, Internet and sufficient bandwidth, and electrical power capacity are critical factors.
The spaces for programs led by community-based organizations can vary in a number of ways: consistency (the same space is available every week); type of space (e.g., local fairgrounds, an enclosed community room at the community center, or shared multipurpose space when programs compete for time and access); and resources available in the space (e.g., spaces with working and maintained equipment and plentiful materials, compared with spaces with folding chairs and irregularly donated items). Moreover, some community-based programs may take place on school campuses in classrooms, gyms and cafeterias, higher-educational institutions, libraries, and makerspaces, such as FIRST® Lego League or FIRST® Robotics, as well as a number of multipurpose after-school programs that support academics and provide enrichment opportunities.
Museums also vary significantly in size and type of facility in part due to the ways in which they receive monetary support (i.e., government support at local, state, and federal levels; earned income; donations or endowments; and grants) and in part because they exist in and serve communities of every size and shape (IMLS, 2019). The physical footprint of museums ranges significantly. Physical space notwithstanding, museum facilities, resources, and capacity vary as well, whether related to dedicated access to tools and equipment for development of exhibits, or materials and supplies available to visitors, or opportunities to develop novel or innovative programs (see Box 5-6 for variations on makerspaces in museums).
Brick and mortar libraries continue to have a presence in communities, according to a survey conducted by the Pew Research Center (Zickuhr, Rainie, and Purcell, 2013). Libraries have evolved from quiet places to active learning places that encourage play-based learning, gaming, and interaction (Hassinger-Das et al., 2020). Some libraries have dedicated teens/children sections, while some have learning labs, computer labs, makerspaces, and
___________________
11 It is worth noting that much of this discussion predates COVID-19.
recording studios; these spaces vary tremendously as potential learning environments that support authentic experiences (see Box 5-7 on Chicago YOUMedia). While some libraries have received funding from IMLS and/or their states, many do not; hence the space for learners often varies based on funding, support, and the size of the libraries. Some libraries have used their limited spaces creatively for computing opportunities for youth, conducted programs at partner organizations’ sites, or brought programs to the places where youth already go, such as community centers, juvenile detention centers, etc. (ALA, 2019).
In 2012 and 2013, IMLS and the MacArthur Foundation funded the design, development, and implementation of Learning Labs in museum and library spaces around the United States.12 These dedicated spaces were
___________________
12 For more information on the Learning Labs program, see https://www.imls.gov/assets/1/AssetManager/LearningLabsReport.pdf.
focused on supporting a community of learners engaged in collaborative, hands-on, interest-driven learning across many types of media, all based in the principles of connected learning. The physical spaces that were created are representative of the thinking and repurposing that a lot of museums and libraries do anyway—renovations of meeting rooms, corners of bike shops and makerspaces, old computer labs developed into places for teen communities. Libraries are often used more as a physical space for experimentation or intervention rather than to develop an intellectual space offering authentic experiences (Garmer, 2014).
LEARNING FACILITATORS
Alongside with the learners and the setting, facilitators are critically important to any authentic learning experience for computing. These are the people who help to shape and foster learners’ interests and competencies for computing. What follows is a discussion of the facilitators for each designed setting separately.
Community-Based Organizations
Educators working at community-based organizations draw from a varied set of backgrounds, experiences, and goals. Already, they navigate many of the aforementioned challenges of time, access, technology, and
duration that influence the engagement of their learners, and their responsibilities may span a wide range of expectations, from homework help to enrichment to STEM programming. Due to the fragmented nature of the sector (Afterschool Alliance, 2014; Huang and Dietel, 2011; Lauer et al., 2006; Little, Wimer, and Weiss, 2008), the professional development, preparation, and support for instructors and facilitators in community-based organizations is varied and inconsistent as well, and educators do not always have related academic degrees, content-specific knowledge, or education-specific training (Bouffard and Little, 2004; Costley, 1998). Many of these positions are part-time, seasonal, or volunteer opportunities, and some are used as career growth or transition steps, which can result in educators staying only for short tenures (Bowie and Bronte-Tikew, 2006; Fleming, 2012).
In recent decades, marked by an increasingly important role played by OST programming, newer professional development and credentialing programs are being developed for after-school educators. Reports from the National Institute on Out-of-School Time (NIOST) share findings from multiple states on the development and impact of credentialing systems and core competency frameworks. In 2002, the Academy of Educational Development (AED), which had led the National Training Institute for Community Youth Work (NTI), developed the Advancing Youth Development: A Curriculum for Training Youth Workers (AYD) program. A summary report of an evaluation of its National BEST (Building Exemplary Systems for Training Youth Workers) Initiative, which created a citywide network and professional development systems to support youth workers, was releaased. The report noted that training affected both the pedagogical understandings and practices of facilitators, and that continued, ongoing professional support of facilitators may help to increase the overall positive perception and professionalization of the field. In turn, educators at these community-based organizations feel more connected, more valued, and more likely to remain in their positions with opportunities for professional growth and sustainment.13
Museums
Similarly, museum educators come to the profession from a diverse set of backgrounds, sometimes with educational training, sometimes with a degree in a specific field of study related to the museum’s focus (e.g., art, history, science), sometimes with a combination of the above. However, often many do not have museum-specific training nor computing-specific backgrounds. Across the wide variety of museum settings, educators vary
___________________
13 For more information, see http://scs.fhi360.org/publications/best.pdf.
widely in their practices, do not necessarily have the same set of responsibilities or roles, do not often have a shared pedagogical background or training, and may even conceive of learning differently from one another or from the educators—both inside and outside of the museum—with whom they work (Bevan and Xanthoudaki, 2008; Tran, 2006; Tran and King, 2007). Many studies show that museum educators are often basing their teaching and guidance on their own experiences, which are typically drawn from school settings (Crowley, Pierroux, and Knutson, 2014).
In addition to the complexity of educator roles in the museum profession, it is important to note who is involved in the delivery of such authentic experiences (Crowley, Pierroux, and Knutson, 2014; Hooper-Greenhill, 2013). Leaders and staff members of museums and cultural institutions are often unreflective of the demographics of the populations with respect to race, ethnicity, and socioeconomic status they serve or are striving to serve. Jacobs (2019) stated that people of color were far from being represented in internal positions (evidence by a 2018 study14 that surveyed New York City institutions, such as the Metropolitan Museum of Art) and that the city required city-funded museums to put plans in action to address the problem.
Without a doubt, museum education is its own field of research and study, and much work continues in understanding and clarifying its practices, interrogating its representation, and developing opportunities for professional development. That being said, there have long been opportunities and partnerships for museum educators to support and provide professional development opportunities to other after-school and in-school educators. Numerous museums around the country support curriculum integration and professional development at schools in their regions15 and more recently around computing (see Box 5-8).16 An interagency, cross-sector collaboration between IMLS and the Department of Education, which began in 2014 and expanded in 2019, supports science and children’s museums to provide curriculum, training, and resources to 21st Century Community Learning Centers (21st CCLC) across the country.17
Public Libraries
As highlighted earlier in this chapter, public libraries are venues whose potential for offering authentic experiences for computing could be fur-
___________________
14 The study was commissioned by the administration of Mayor Bill de Blasio.
15 For example, see https://makered.org/making-spaces.
16 For example, see https://www.hopkinsschools.org/district-news/news/science-museumembeds-computational-thinking-elementary-curriculum.
17 For more information on this initiative, see https://www.imls.gov/news/imls-announces-19-million-investment-stem-making-education-underserved-youth and https://y4y.ed.gov/stemchallenge/imls.
ther developed. The Re-envisioning the MLS report called for “facilitating learning in libraries through making, STEAM, coding, and a range of other activities. This not only promotes information organizations as essential to learning and education, but also enhances youth learning” (Bertot, Sarin, and Percell, 2015, p. v).
There are approximately 140,000 librarians in the United States. Many librarians have a master’s degree in library and information science (MLIS)18 from an ALA-accredited program. However, not all librarians hold a MLIS or hold a MLIS from an accredited institution. IMLS (2019) reports that just over two-thirds of full-time-equivalent librarians hold an ALA-accredited MLIS degree. Libraries serving larger populations had a higher percentage of ALA-accredited MLIS librarians (78.98%) than
___________________
18 There is a slight variation in the degree names depending on the institution, including Master’s in Library Science, Master’s in Information Science, etc.
libraries that serve smaller communities (10.21%) (IMLS, 2019). Library staff also may hold other degrees or, conversely, may not have any higher education.
When it comes to providing authentic experiences to learners in STEM or computing, librarians play a central role. In a nationwide interview and focus group study with 92 public library staff serving youth, Subramaniam and colleagues (2018) found that many librarians already leverage one or more of the connected learning design principles to create their programs because of the typical structure of library programs that often strive to be authentic to youth, interest-driven, fun, and different from “school.” There have been calls for librarian preparation programs to train future librarians in these areas. For example, Taylor et al. (2018) calls for the integration of computing in preservice librarian training and examines the nature of such integration in selected preparation courses. This requires a significant change in mindset about what librarians can contribute to the development of interest and competencies in STEM generally, and computing in particular.
Librarian preparation programs are providing training in connected learning, media mentorship, youth learning, and facilitation of STEM programs. This follows the recommendation of the two divisions within the ALA (Association for Library Services for Children [ALSC] and Young Adult Library Services Association [YALSA]) that have developed new competencies for library staff serving learners (see ALSC, 2015; YALSA, 2017). These divisions also provide professional development webinars, short courses, conferences, training, and published guidelines for librarians serving learners in public libraries—some free and some for a fee. But while some librarian preparation programs have included such content and some institutions are in the process of doing this, such integration is not prevalent.
Another primary challenge is that library staff members who interact with learners may not have received any training in working with these populations. Another pervasive problem is where libraries provide access to technologies (Davis et al., 2018; Subramaniam et al., 2018) but are not used to provide or facilitate authentic experiences for computing that promote the success of learners typically underrepresented in computing. This can happen when library staff are not aware of the connected learning principles to create technology-enabled learning environments or are unable to connect with other networks that support such endeavors (Ito et al., 2020).
In the face of these challenges, library staff often assume the following approaches when they lack the training or background needed to provide authentic experiences to learners: (1) developing partnerships with institutions and individuals that have expertise, such as science centers, universities, and industries (Garmer, 2014; Zhou et al., 2019); (2) developing skills in mentoring, rather than being an expert who teaches STEM or computing skills by embracing roles such as facilitator and co-learner (Clegg
and Subramaniam, 2018; Ito et al., 2020; Tripp, 2011); and (3) training and encouraging peer mentorship and near-peer mentors (Martin, 2017; Vickery, 2014).
MEASURING OUTCOMES
Recently developed efforts focused on evaluation and assessment of learning outcomes in and across museums, libraries, and makerspaces have been pushing research to address many of the evolving questions and needs emerging in these spaces. These projects include development of frameworks (Cun, Abramovich, and Smith, 2019; Wardrip et al., 2016) and observational tools (Martin et al., 2019), as well as investigation of documentation (Byrne and Louw, 2020) and portfolio practices (Peppler, Keune, and Chang, 2018).
Community-based organizations with large national networks and headquarters may have evaluators and program designers on staff, but for educators and facilitators, measuring outcomes is not often a priority, nor do on-site staff have sufficient training or support to gather data and interpret it for their own use (Murchison et al., 2019). Programs also vary widely in their focus (STEM and computing versus academic support) and the rigor of their methods, varying from program to program and location to location, so determining effectiveness is not consistent across the whole field (Lauer et al., 2006; McComb and Scott-Little, 2003; Scott-Little, Hamann, and Jurs, 2002). A RAND Corporation analysis on The Value of Out-of-School Time Programs reports that examining program effectiveness writ large can be flawed because “programs are often grouped together without regard for their goals (e.g., improve academic performance, promote positive social skills, or decrease substance use), content, or the measurable outcomes programming might produce” (McCombs, Whitaker, and Yoo, 2017, p. 2).
High-quality research and evaluation studies are limited. Specifically with regard to STEM and computing programs based in community organizations, the quantity and quality of peer-reviewed research is small, especially compared to the evidence base within the formal K–12 system, in part because the OST sector is fragmented. Significantly, there is also far less funding available for research and evaluation, compared to that available for the K–12 school settings (Fredricks and Eccles, 2006; Krishnamurthi, Ballard, and Noam, 2014; Lauer et al., 2006; Weinberg, Basile, and Albright, 2011). The body of research and evaluation on STEM outcomes is continuing to grow, as community-based organizations and the national networks that connect them (e.g., Afterschool Alliance, National Institute for Out-Of-School Time, National Afterschool Association, among others) support development of frameworks, review and report more con-
sistently on program impacts, and offer guidance and tools to train and elevate evaluation for program providers. For example, FIRST®, in partnership with Brandeis University, is in its fifth year of a longitudinal study to measure and understand outcomes related to STEM, including interests, attitudes, career trajectory, and workforce skills (Melchoir et al., 2019).
The programmatic efforts, outcomes, and research on community-based organizations focused specifically on providing authentic experiences in computing are expanding and evolving. As noted in the K–12 Computer Science Framework, released in 2016, “informal education organizations are essential to the CS education ecosystem and should be included as critical stakeholders in state and district implementation efforts” (pp. 167–168). But a resource guide from the Afterschool Alliance,19 does not mince words when it says
Computer science education of course requires access to technology, internet connectivity, and funding. But beyond that, we know that it’s especially challenging to find qualified staff and curriculum specifically designed for the out-of-school time environment. The same levels of professional development and variety of quality curriculum just don’t exist for computer science in the same way it does for the other STEM topics like science and engineering. (p. 1)
Clarified definitions and outcomes, reports and guidance, curriculum and activities, professional development and coaching, and funding are being developed by researchers, OST staff, and professional organizations to support the design, implementation, and evaluation of such efforts, ultimately ensuring computing experiences are readily available, accessible, and of high quality.20
Museums and libraries take different approaches to establishing and measuring learning outcomes. There is little consistent evidence and research in museum settings specific to interest, outcomes, and trajectories related to computing. Research on museums over the past few decades still seeks to better understand and clarify the complex nature and outcomes of learning that may happen within and in connection with museums (Falk and Dierking, 2018). Though there have been museum studies in the past decade looking broadly at STEM learning experiences and learner outcomes, they are often focused on short-term programming, commonly related to museum visits and field trips, and evidence generally related to STEM, such as attitudes, content knowledge, and skills (Chi, Dorph, and Reisman, 2015).
___________________
19 Available at http://afterschoolalliance.org//documents/AfterschoolCS_ResourceGuide_2017.pdf.
20 For more information, see http://afterschoolalliance.org/documents/Growing_Computer_Science_Education_2016.pdf.
Some museums have their own internal research and evaluation teams or departments (e.g., The Exploratorium, Children’s Museum of Pittsburgh) and lead research or research-practice partnerships to better investigate key questions in the field and the impact of their own programming and designs (Association of Science and Technology Centers, 2020; IMLS, 2008). The informal science education field is also supported by the Center for the Advancement of Informal Science Education (CAISE), which originated in 2007 and is funded through the NSF Advancing Informal STEM Learning (AISL) Program. With regard to evaluation, CAISE provides a wealth of resources and guidance for evaluating informal science education efforts.21
Libraries have a compounding problem of measuring outcomes, due to the lack of standardization of duration and format, the complexity of assessment of connected learning in the design of authentic experiences (Ito et al., 2020), and the skills that librarians need to design evaluation plans to capture these much-needed outcomes. Information shared by librarians in practitioner literature attests to the importance of libraries as sites where authentic experiences in STEM and computing occur. Libraries continue to count activities and use attendance as a measure of their use and effectiveness in this digital era. However, the Aspen Institute (Garmer, 2014) calls for librarians to think about the long-term public library sustainability by “becoming more skilled at measuring outcomes rather than counting activities.” Libraries use surveys and focus groups to assess participants about the content and goals of specific programs. The Capturing Connected Learning in Libraries team has developed some sample case studies of evaluation of connected learning programs with large public library systems and with small and rural library branches, and have created tools such as observations, talk-back boards, and staff surveys (Allen et al., 2020; Penuel, Chang-Order, and Michalchik, 2018). These evaluation case studies are a start toward capturing outcomes resulting from authentic experiences; however, these evaluation studies are not relatable to small and rural libraries that often serve communities that lack resources.
EXPERIENCES THAT CUT ACROSS SETTINGS
The committee recognizes that learners exist within an ecosystem of relationships, opportunities, and contexts; no learning experience is completely isolated from others. It is important to consider some of the genres that can occur in and outside of school spaces within this ecosystem. This section is intended to signal the committee’s recognition of topics that were beyond the scope of the charge to review in depth and in some cases lacking a robust research base. The committee calls attention to the fact that
___________________
educational research, particularly studies of learning outcomes in home and online youth-driven settings, is sparse. The research that does exist tends to center on social and cultural studies of media, Internet, and youth culture, rather than educational research. What follows is a brief review of online gaming and creative communities and STEM competitions.
Online Gaming and Creative Communities
Even as engineering, coding, and computer science have made inroads into the K–12 curriculum and after-school and summer programming, children’s exposure to technology is dominated by their recreational experiences playing computer games and engaging in digital platforms such as YouTube, Instagram, TikTok, and Snapchat.22 Studies have documented how youth have developed coding and digital creation skills and interests in authentic communities of practice by tinkering MySpace profiles (Perkel, 2010), modding games (Kow and Nardi, 2010; Kow, Young, and Tekinbas, 2014), creating online videos (Lange, 2014), and “geeking out” in varied communities of interest online (Ito et al., 2018; 2019). Exposure to and sustained interaction with augmented reality through apps such as Pokémon Go may also provide opportunities to develop interest in computing; however, Layland et al. (2018) point out that White women as well as Black men and women have limited opportunities to participate. Given the limited lifespan of mobile apps, there continues to be a need for social justice in digital leisure activities (Layland et al., 2018).
Many entertainment-oriented games are explicitly designed to encourage digital making, algorithmic thinking, coding, and remixing. This genre of game was piloted by games such as SimCity, Rollercoaster Tycoon, Civilization, and the Sims, where players can tinker with worlds and algorithms to “grow” cities, theme parks, and families. Studies have documented how commercial games in these genres have fostered technical interests and skills (Ito, 2009; Squire, 2006). More recently, games such as Little Big Planet, Minecraft, and Roblox offer both tools and platforms for communities to make, code, and play together (Dezuanni, O’Mara, and Beavis 2015; Rafalow and Tekinbas, 2014; Ringland et al., 2017). In recent years, these gaming platforms that support player-level making, coding, and tinkering have become increasingly central to young people’s growing up.
Many digital games and platforms have engaged youth who may feel excluded or disengaged from STEM or computing in school or other educator-led settings. This genre provides contexts that foster personally authentic computing experiences for youth who may not connect to pro-
___________________
22 For more information about teen use of media, see https://www.commonsensemedia.org/research/the-common-sense-census-media-use-by-tweens-and-teens-2019.
fessionally authentic experiences in school. At the same time, youth-led settings of play often reproduce existing cultural and gender divisions and status hierarchies in young people’s peer networks. It is widely recognized that tech-savvy gaming communities are more often unwelcoming to women and girls.
These large-scale digital creation, play, and coding platforms and communities are dominated by commercial tools, with at least one notable exception. The Scratch platform (see Box 7-7), launched at the MIT Media Lab and now stewarded by a non-profit with more than 50 million registered users, has been used to connect from homes, schools, and community organizations. Learning institutions have also adopted and adapted commercial tools and platfoms such as SimCity, Civilization, Minecraft, and Roblox, seeking to build bridges between recreational pursuits and the interests and skills that young people are developing in their home (Takeuchi and Vaala, 2014).
STEM Competitions
Competitions for young learners are a prominent feature within STEM and computing, coming in a great variety of forms, with substantial differences in intended audience, goals, organizational structure, rules, and atmosphere. Competitions are an important part of many learning ecosystems, crossing as they do between various physical and performative domains, and mediating between different disciplines and modes of learning. Some research exists on a number of different competitions (Hendricks, Alemdar, and Ogletree, 2012; Nugent et al., 2016; Veety et al., 2018; Woszczynski and Green, 2017). The committee offers here a brief overview of the competition landscape for learners, with a particular focus on how competitions engage with the challenges of authenticity, equity, and learning outcomes.
The concept of “competition” is not uniform, even among contest organizers; the lack of consensus and the differences in conceptual framing and terminology have implications for equitable access, inclusivity, and authentic experience. Psychologists, sociologists, and educational theorists—among others—have long explored whether and how competition may foster or hinder learning and other positive youth development outcomes, with findings that support various suppositions based at least in part on the nature and setting of a given competition (Dillenbourg, 1999; Johnson and Johnson, 2009; Kochanek et al., 2019; Torres and Hager, 2007).
Miller, Sonnert, and Sadler (2018) concluded that “students who participate in STEM competitions are more likely to express interest in a STEM-related career at the end of high school than are students who do not participate,” and, as a result, “competitions are an effective way to foster career interest in specific STEM careers” (p. 95). However, it
is worth pointing out that this may not be surprising as competitions typically include learners who self-select and have the resources to opt into these experiences. One concern observers have raised is that formal competitions may serve as gatekeepers, privileging those who have deeper educational and financial resources and perpetuating dominant forms of authenticity and STEM/computing culture (Mabey, Lande, and Jordan, 2016; Schank, 2015; Zimmer, 2016).23 By contrast, a study of the Air Force’s Cyberpatriot program indicated active engagement by learners whose families have lower incomes (Brough, 2016), suggesting opportunities to design competitions that attract a more diverse set of youth.24 There does not appear to be much research to determine how best to meet such concerns, especially with respect to learners from minoritized communities.
It is important to note that there may be distinct differences relating to inclusivity and authenticity even within a given competition, for instance, between its various levels and cohorts. Suggesting what might need to be changed, one recent study of robotics competitions by Witherspoon et al. (2016) found that
in the youngest groups/entry-level competitions, girls were heavily involved in programming. Unfortunately, in older/more advanced competitions, girls were generally less involved in programming, even after controlling for prior programming experience . . . While robotics competition experiences may motivate students to learn more programming, gender gaps in programming involvement persist in these learning environments and appear to widen as students grow older and enter more advanced competitions. Therefore, addressing gender imbalances in programming will likely require greater attention to particular curricular and pedagogical characteristics of robotics competitions that support girls’ interest and involvement in programming. (p. 18)
Partly as a response to perceptions that traditional science fairs may be too rigidly structured and too tightly knit to existing STEM norms, in the past two decades a host of other competitive formats have emerged, including hackathons, game jams, maker faires, and tech showcases. Examples include Games for Change Student Challenge, Emoti-Con run by The Hive NYC Learning Network, hackathons run by Black Girls Code, S.T.E.A.M.
___________________
23 For more information by Education Development Center on Taking Science Fairs to the Test, see https://www.edc.org/putting-science-fairs-test.
24 It should be cautioned that some of these programs have been compared to school-based pathways such as career and technical education, in that these programs may reach more diverse learners but may lead to lower-wage pathways into computing. Moreover, it should be acknowledged that there are criticisms surrounding the military’s role in STEM education, particularly when targeting minoritized communities (i.e., learners of color and those with lower incomes) (Vossoughi and Vakil, 2018).
Achievers Purpose Hackathons, Regeneron International Science & Engineering Competition, the Academy of Applied Science’s Young Inventor competitions in northern New England, and Technovation’s global competitions for families and for girls.
The newer formats emphasize collaboration, learner- and/or community-based identification of the challenge to be addressed, and open-ended, design- and engineering-oriented processes for meeting the challenge. Many of the newer formats include young learners generating ideas for the competition and serving as competition (co-)organizers, peer mentors and judges; many actively reach out to families whose members have never or only very seldom participated in competitions of these sorts. The newer formats do not eschew competition but stress the team element of the contest and emphasize the shared competition against the external challenge (e.g., create and program together a sensing device that will help your school’s physical plant use less energy) and against the clock (e.g., do so within this specified 8 hours).
Given the novelty and wide variety of game jams, hackathons, and the like, relatively little research has yet been done to discover what works best and why, and how best practices might be replicated, specifically for young learners and with a focus on authenticity and equity issues. Some initial findings from small sample sizes (including from collegiate settings that may nevertheless apply to secondary environments) include increased interest and retention in computing and tech-related subjects, resulting from students’ increased sense of community and their opportunity to craft for themselves an identity within STEM and/or computer science (Arya et al., 2019; Fowler et al., 2016; Lara, Lockwood, and Tao, 2015; Munro, 2015; Pe-Than et al., 2018; Porter et al., 2017; Trainer and Herbsleb, 2014).
SUMMARY
Exposure to authentic experiences for computing occurs in a variety of settings, including youth development programs, museums, libraries, STEM competitions, and home-based and online communities. This chapter presented what is known about the role of educator-designed OST settings—particularly community-based programs, museums, and libraries—in fostering access to authentic experiences for computing.
Although OST experiences have the potential to provide broad access to authentic experiences for computing, there are a number of different barriers and constraints to participation. For example, there are inequities associated with the distribution and quality of OST experiences, which include time availability, cost, and transportation, in addition to lack of resources. A number of programs have sought ways to offset costs, offer programs over multiple sessions, and include low-technology or unplugged
opportunities. Differences are also observed across the settings in terms of the duration and format of programming, the preparation of program facilitators, and the outcomes being measured. Moreover, although there has been increasing emphasis on access to authentic programs, the links to computing have been less emphasized and are limited.
Overall, programs are beginning to be more intentional in the outcomes being measured in an effort to ensure that programs are effective and that equity-oriented goals are being met. In an effort to reach learners, a number of programs (e.g., Beam Camp, The Clubhouse Network) have attempted to provide authentic experiences for computing that are developed to capitalize upon the shared and personally relevant interests of learners. Some of these activities occur in settings such as online gaming and creative communities as well as STEM competitions. Unfortunately, across all of the settings discussed throughout this chapter, problems remain with reaching learners who have been excluded based on gender, race, ethnicity, or perceived ability. The most prominent limitation to understanding what is happening in these spaces and how these programs are attending to reaching learners from the aforementioned marginalized communities is the lack of programs measuring and publishing outcomes.
