National Academies Press: OpenBook

Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors (2021)

Chapter: Appendix A: Search Strategy and Data Coding

« Previous: References
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×

Appendix A

Search Strategy and Data Coding

The committee sought to assemble a set of studies that represented the extent of available evidence. The review focused on literature and programs centered around three categories of outcomes: (1) affective (such as interest, identity/belonging, motivation, and self-efficacy); (2) cognitive (such as knowledge and skills); and (3) behavioral (such as engagement, persistence, and retention). A search was conducted through Scopus requesting studies from the past two decades (2000–2020) and limited to English. Table A-1 shows the syntax used to identify studies. The search parameters were set to be broad and inclusive, which yielded a large number of studies (n = 2,974).

The list was culled down by removing studies that did not meet committee-determined standards for quality and relevance. Specifically, studies were classified based on the following attributes: goal, intervention type, study type, findings reported, outcome category, STEM field, methods, setting, authentic, and the age/grade range. To determine the type of study, the committee used the common guidelines for education research and development (Earle et al., 2013). Box A-1 briefly documents the guide and summarizes the different study types. From this analysis, 309 studies were identified as potentially relevant and requiring additional review.

The committee then reviewed the abstracts for each of the 309 studies. Table A-2 illustrates the systematic culling down of studies that were reviewed at each stage, whereas Table A-3 highlights the ones described in the chapter on outcomes. In Table A-2, studies with measurable impact estimates are those studies that can be characterized as Impact and Effectiveness Research based on Box A-1.

Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×

TABLE A-1 Search Syntax for Locating Studies

Criteria and Syntax

Computing focus
(STEM
STEM OR science OR technology OR engineering OR mathematic* OR computing OR “computer science” OR “computer program*”)
AND Personalized learning focus
(“hands-on” OR “design based” OR “maker ed” OR “maker space” OR “place based” OR authentic OR “technology education” OR “project based” OR “problem based” OR “activity design” OR “maker movement*” OR “problem solving” OR “game design”)
AND Setting
(“in school” OR “out of school” OR formal OR informal OR librar* OR museum* OR “community setting*” OR YMCA OR “boy scout*” OR “girl scout*” OR club* OR “summer camp”)
AND Target youth
(“K–12” OR “elementary school*” OR “middle school*” OR “high school*” OR child* OR youth* OR adolescent* OR teenager* OR “young student*”)
(student* AND NOT {college* OR undergraduate* OR graduate* OR university})
AND Focal outcomes
(engagement OR {time on task} OR “future course*” OR “future class*” OR “future club*” OR persistence OR “choice major” OR “curriculum choice*” OR “class choice” OR “career choice”)
(interest OR awareness OR attitude*)
(knowledge OR skill* OR disposition* OR network*)
AND Recency
PUBYEAR AFT 2000
Citations returned = 2,974

*Focal outcomes were aligned with commonly stated program goals.

Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Search Syntax Strategies and Yield
1 2 3 4 5 6 7 8 9
X X X
X X X X X X
X X X X X X X X X
X X X X X X X X X
X X X
X X X
X X X
X X X
X X X
X X X X X X X X X
172 74 154 382 179 331 655 225 802
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×

TABLE A-2 Search Summary

Outcome Domain Abstracts Retrieved & Reviewed Studies Retrieved & Reviewed Studies Referenced in the Text Studies with Credible Impact Estimate
Affective 107 76 24 3
Cognitive 149 102 37 1
Behavioral 53 40 16 3
Total 309 218 77 7
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×

TABLE A-3 Evaluation Studies Identified Through the Literature Search

Author (Date) Study Type Intervention/Practice
Affective Outcomes
Barker et al. (2018) Exploratory Wearable technologies
Bugallo and Kelly (2014) Exploratory Participatory STEM activities
Bugallo et al. (2015) Exploratory Participatory STEM activities
Choudhury et al. (2010) Exploratory Participatory STEM activities
Denault et al. (2008) Exploratory Participatory CS activities
Doerschuk et al. (2013) Exploratory Participatory CS activities
Evans and Schares (2017) Exploratory Makerspaces/activities
Gardner-McCune et al. (2013) Exploratory Participatory CS activities
Harriger et al. (2012) Exploratory Participatory CS activities
Jagiela et al. (2018) Exploratory Participatory STEAM activities
Krayem et al. (2019) Exploratory Participatory STEM activities
Ladeji-Osias et al. (2018) Exploratory Participatory STEM activities
Monterastelli et al. (2008) Exploratory Participatory STEM activities
Nugent et al. (2016) Exploratory Participatory STEM activities
Nugent et al. (2019) Exploratory Wearable technologies
Stapleton et al. (2019) Exploratory Participatory STEM activities
Ahn et al. (2014) Design & Development Participatory STEM activities
Erete et al. (2016) Design & Development Participatory STEM activities
Jin et al. (2018) Design & Development Participatory CS activities
Lau et al. (2009) Design & Development Wearable technologies
Pinkard et al. (2017) Design & Development Participatory STEM activities
Scott and White (2013) Design & Development Participatory CS activities
Klopfer et al. (2004) Impact & Effectiveness Participatory STEM activities
Loksa et al. (2016) Impact & Effectiveness Participatory CS activities
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Setting Grade/Age Group Outcomes for Impact Estimates
After-school, on site Elementary; Middle school Self-efficacy
Summer program/camp High school Identity; Self-efficacy
Summer program/camp High school Interest; Motivation; Self-efficacy
Summer program/camp High school Identity
Summer program/camp High school Interest
Summer program/camp High school Interest
University K–12 Interest
University High school Interest
Summer program/camp High school Interest
University High school; Middle school Interest; Self-efficacy
University High school Interest
Summer & Saturday Program Middle school Engagement; Interest
University High school Interest
Camps, Clubs & Competitions Middle school Self-efficacy
Classroom—traditional & after-school Elementary school (K–5) Self-efficacy; Programming design & knowledge
University; Summer camp Middle school Self-efficacy
After-school, on site Middle school Identity
Community—other Middle school Identity
Summer program/camp High school Knowledge & Skills; Motivation
Summer program/camp Middle school Knowledge & Skills; Motivation; Interest
Multiple OST Elementary school (K–5) Identity
Multiple OST High school Motivation
Classroom—traditional Middle school; High school Motivation
Summer camp High school Self-efficacy
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Author (Date) Study Type Intervention/Practice
Cognitive Outcomes
Bicer et al. (2017) Exploratory Makerspaces & activities
Brooks and Sjöberg (2019) Exploratory Games
Bugallo et al. (2015) Exploratory Participatory STEM activities
Dasgupta et al. (2019) Exploratory Makerspaces & activities
de Paula et al. (2018) Exploratory Participatory STEM activities
Denault et al. (2008) Exploratory Participatory STEM activities
Folk et al. (2015) Exploratory Unplugged STEM activities
Freina et al. (2019) Exploratory Games
Gardeli and Vosinakis (2019) Exploratory Mobile apps
Nugent et al. (2013) Exploratory Participatory STEM activities
Nugent et al. (2016) Exploratory Participatory STEM activities
Nugent et al. (2019) Exploratory Wearable technologies
Ouyang et al. (2018) Exploratory Participatory CS activities
Sharma and Ali (2018) Exploratory Participatory STEM activities
Weitze (2017) Exploratory Making games/sims
Jenson et al. (2018) Design & Development Course
Yin et al. (2019) Design & Development Makerspaces/activities
Garneli et al. (2015) Impact—Efficacy Games
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Setting Grade/Age Group Outcomes for Impact Estimates
Summer program High school Perception of skills & creativity
Research Lab setting Elementary school (K–5) Interactions with technology, design, decision making
Summer program High school Engineering knowledge
Classroom—traditional Middle school Technology & science knowledge; Experimental strategies & graph interpretation
After School Program—at a school Middle school Computational & game development skills
Summer program/camp High school Computer programming
Classroom—traditional K–12 (across grade bands) CT concepts & terms; Recognition of CT
Classroom—traditional Elementary school (K–5) Programming & documentation skills
Classroom—traditional Elementary school (K–5) Problem-solving, algorithmic concepts & game design
After school, on-site Middle school Computer programming, engineering & design, GPS & science, math, robotics & workplace skills
Camps, clubs & competitions Middle school Computer programming, engineering & design, GPS & science, math, robotics & workplace skills
Classroom—traditional Elementary school (K–5) Knowledge of circuity & programming; engineering design
Classroom—traditional Elementary school (K–5) CT concepts
Classroom—traditional High school Cartesian coordinate system; Applied geometry
Classroom—traditional High school Disciplinary content; Game design & making
Classroom—traditional Middle school Computer programming; Academic achievement
Summer academy Middle & High school Physics & engineering, CT definition & CT skills
ns Middle school Problem-solving, programming & use of CS constructs
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Author (Date) Study Type Intervention/Practice
Behavioral Outcomes
Amo et al. (2019)—study 2 Exploratory Hands-on cyber security workshops
Freudenthal et al. (2011) Exploratory Media-propelled CT
Gardeli and Vosinakis (2019) Exploratory Game: Mobile augmented reality
Klein (2013) Exploratory Casual mobile game
Simpson et al. (2017) Exploratory Computer-supported collaborative learning
Vickery (2014) Exploratory Web design workshop
Folk et al. (2015) Exploratory Deliberate instruction in CT & algorithm design (unplugged)
Weston et al. (2019) Exploratory AP and computing courses
Charlton and Poslad (2016) Exploratory (Descriptive) Maker events: Shareable wearables
Mystakidis et al. (2014) Development Immersive multi-media learning experience
Amo et al. (2019)—study 1 Impact & Effectiveness (RCT) Cyber-networking workshop
Klopfer et al. (2004) Impact & Effectiveness (RCT) Palm versus wearable supported participatory simulations
Wanzer et al. (2020) Impact & Effectiveness Web-based introductory computing experience

NOTE: ns means not specified. For each outcome category, the articles are organized by study type and then alphabetized.

Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Setting Grade/Age Group Outcomes for Impact Estimates
Summer camp 13–17-year-olds Cyber security engagement & self-efficacy
High school math & college calculus class High school students First year college students Enrollment in CT courses, engagement in problem-solving, math course enrollment& college grades
Primary school classrooms Age 9–10 Enjoyment & collaboration
Play Club 10–15-year-olds Level of emersion & reflection
Summer camp Rising high school freshmen Positions of power
Public library 8–16-year-olds Sustained enjoyment
K–12 STEM classes Middle & High school instructors & students. Recognition & definition of algorithms; engagement in CT
High school High school students Persistence in computing post high school & technology majors
Community setting Age 14–15 Engagement in design and development & product application
Library Grades 2–6 Interest in books & positive mentality toward reading
Museum Middle school (low-income) General problem solving, science engagement, cyber & digital awareness
Science classes Study 1: 14–16-year-olds Study 2: 11–13-year-olds Engagement in collaborative problem-solving, motivation to participate in problem-solving & self-reported learning
Workshops; summer camps, traditional classes Middle school; High school Intent to persist in computing
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×

This page intentionally left blank.

Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 199
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 200
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 201
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 202
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 203
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 204
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 205
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 206
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 207
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 208
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 209
Suggested Citation:"Appendix A: Search Strategy and Data Coding." National Academies of Sciences, Engineering, and Medicine. 2021. Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors. Washington, DC: The National Academies Press. doi: 10.17226/25912.
×
Page 210
Next: Appendix B: Committee and Staff Biographies »
Cultivating Interest and Competencies in Computing: Authentic Experiences and Design Factors Get This Book
×
Buy Paperback | $55.00 Buy Ebook | $44.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Computing in some form touches nearly every aspect of day to day life and is reflected in the ubiquitous use of cell phones, the expansion of automation into many industries, and the vast amounts of data that are routinely gathered about people's health, education, and buying habits. Computing is now a part of nearly every occupation, not only those in the technology industry. Given the ubiquity of computing in both personal and professional life, there are increasing calls for all learners to participate in learning experiences related to computing including more formal experiences offered in schools, opportunities in youth development programs and after-school clubs, or self-initiated hands-on experiences at home. At the same time, the lack of diversity in the computing workforce and in programs that engage learners in computing is well-documented.

It is important to consider how to increase access and design experiences for a wide range of learners. Authentic experiences in STEM - that is, experiences that reflect professional practice and also connect learners to real-world problems that they care about - are one possible approach for reaching a broader range of learners. These experiences can be designed for learners of all ages and implemented in a wide range of settings. However, the role they play in developing youths' interests, capacities, and productive learning identities for computing is unclear. There is a need to better understand the role of authentic STEM experiences in supporting the development of interests, competencies, and skills related to computing.

Cultivating Interest and Competencies in Computing examines the evidence on learning and teaching using authentic, open-ended pedagogical approaches and learning experiences for children and youth in grades K-12 in both formal and informal settings. This report gives particular attention to approaches and experiences that promote the success of children and youth from groups that are typically underrepresented in computing fields. Cultivating Interest and Competencies in Computing provides guidance for educators and facilitators, program designers, and other key stakeholders on how to support learners as they engage in authentic learning experiences.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!