National Academies Press: OpenBook
« Previous: Foundation for This Study
Suggested Citation:"Study Overview." National Research Council. 2013. Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing?. Washington, DC: The National Academies Press. doi: 10.17226/13509.
×

STUDY OVERVIEW

Because education in the United States is decentralized, it is likely that schools, districts, and states will implement the recommendations in Successful K-12 STEM Education in varied ways and to varying degrees. And although some data that could provide indications of progress in meeting the recommendations of the 2011 report are currently collected at the state and national levels, these data are not collected or analyzed expressly for that purpose, nor is there any attempt to coordinate data collection. As a result, educators, researchers, and policy makers lack a clear picture of the extent to which the nation as a whole is implementing the recommended improvements to STEM education, or making progress toward them.

To address this lack, and in response to a request from Congress, the National Science Foundation (NSF) requested that the National Research Council’s Board on Science Education and Board on Testing and Assessment convene an ad hoc committee (see Box 1 for the formal charge to the committee). This report of the Committee on the Evaluation Framework for Successful K-12 STEM Education presents the committee’s proposed indicators and framework for creating a monitoring and reporting system with those indicators at its core.

Scope of the Study

The committee’s charge addresses only the recommendations in Successful K-12 STEM Education. However, those recommendations were developed to support improvements that would enable progress toward the broader goals of U.S. STEM education (see Figure 1). Thus, the committee decided that it is important for this proposed monitoring and reporting system to include information related to the goals for STEM education. By including indicators of conditions that lead to improved performance of students and the education system alongside key performance measures that can be tracked over time to evaluate whether the United States is making progress toward its STEM education goals, the committee’s proposed monitoring and reporting system has the potential to support better decision making and contribute to the desired improvements (Walpole and Noeth, 2002).

Reflecting the emphasis in Successful K-12 STEM Education, this report primarily addresses science and mathematics education rather than all of the STEM disciplines. Because the research base that underpins Successful K-12 STEM Education was the strongest for mathematics and science, the recommendations in that report relate mostly to those subjects, with some of the recommendations addressing only science. Research in technology and engineering education is less mature because those subjects are not as commonly taught in K-12 education (National Research Council, 2011, p. 2). However, engineering and technology are included in A Framework for K-12 Science Education (National Research Council, 2012), which is forming the basis for the Next Generation Science Standards (being developed by Achieve, Inc.). Thus, any of the committee’s proposed indicators that relate to the 2012 report include engineering and technology. In addition, the scheduled 2014 National Assessment of Educational Progress (NAEP) technology

Suggested Citation:"Study Overview." National Research Council. 2013. Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing?. Washington, DC: The National Academies Press. doi: 10.17226/13509.
×

BOX 1
Charge to the Committee

An ad hoc committee will conduct a fast-track, targeted consensus study to identify methods for tracking and evaluating the implementation of the improvements for K-12 education recommended in the 2011 NRC report, Successful K-12 STEM Education. At the school and district levels, these recommended improvements include adequate instructional time and resources for science, coherent standards and curriculum, greater teacher capacity, and supportive school conditions that have been identified in the research. At the state and national levels, these recommended improvements include greater attention to science, including assessments for science, investment in support for STEM teachers, and increased support for research programs that can identify instructional practices that improve student outcomes in STEM.

The committee was charged with authoring a short report that includes a plan for evaluating progress toward the 2011 report’s recommendations. The short report is to provide guidance to enable NSF and Congress to consider how to support a full-scale evaluation in the future. To that end, the report will

•     identify existing and additional measures needed for tracking the improvements recommended in 2011,

•     discuss the feasibility of including such measures in existing programs of data collection and linking them to existing and improved measures of science learning outcomes,

•     identify additional research that would be needed to inform a full-scale evaluation of progress toward the 2011 report’s recommendations, and

•     make new recommendations regarding the roles of various federal and state institutions in supporting the needed research and data collection for an evaluation of progress.

and engineering framework is increasing the emphasis on these subjects by measuring students’ literacy in them. As these subjects become more central to K-12 education in the future, they can be more fully incorporated into the proposed monitoring system, with additional research to identify the most important aspects to measure.

Suggested Citation:"Study Overview." National Research Council. 2013. Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing?. Washington, DC: The National Academies Press. doi: 10.17226/13509.
×

Study Approach and Sources of Evidence

The committee worked through an iterative process of gathering information, deliberating, identifying gaps and questions, gathering further information to fill those gaps, and further deliberations. The recommendations in Successful K-12 STEM Education (National Research Council, 2011) were the starting point for our deliberations: we took the previous committee’s work and its recommendations as given and did not debate their merit. To identify indicators of progress that would be important to measure for each recommendation, we also drew on a long series of research reports related to K-12 science and mathematics instructional practices (National Research Council, 1999, 2001, 2006, 2007, 2009, 2012; National Academy of Engineering, 2010); school conditions, leadership, and teacher capacity building (e.g., Bryk et al., 2010; Gamoran et al., 2003; McLaughlin and Talbert, 2006); learning science in informal environments (National Research Council, 2009); and general principles of monitoring and evaluation (Shadish, Cook, and Campbell, 2001; U.S. General Accounting Office, 2003). Finally, we consulted relevant experts at agencies that manage large-scale data sets (e.g., National Center for Education Statistics, National Science Foundation) and other organizations (e.g., Council of Chief State School Officers) about the availability of certain types of data.

The rest of this report identifies key indicators that could provide evidence of progress toward the recommendations in Successful K-12 STEM Education. We set four criteria to guide the selection of indicators. Indicators must

•     be shown by research to be related to a given recommendation,

•     generate information that clearly enhances understanding of progress toward the recommendation for all student groups,

•     have the potential to be used for continuous improvement, and

•     be feasible to measure on a large scale.

Some of the proposed indicators require new kinds of data collection, and all would benefit from additional research and conceptual development to enhance understanding of what to measure. We included indicators as long as they met the above criteria, but we did not undertake a systematic analysis of the costs involved in measuring the proposed indicators. Instead, we drew on the committee’s expertise with federal research and data collection systems and on consultations with relevant experts to determine whether collecting data on the proposed indicators would generally be feasible.

The monitoring and reporting system proposed in this report is not the first of its kind. In the late 1980s similar indicator systems were developed for K-12 science and mathematics education (National Research Council, 1988; Shavelson, McDonnell, and Oakes, 1989), and the committee consulted those reports. The indicators in those reports were comprehensive, were

Suggested Citation:"Study Overview." National Research Council. 2013. Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing?. Washington, DC: The National Academies Press. doi: 10.17226/13509.
×

carefully developed, and had the potential to generate valuable information about the condition of mathematics and science education in the nation’s schools and how individual components of the education system interact to affect the system as a whole (National Research Council, 1988; Shavelson, McDonnell, and Oakes, 1989). Although they were never fully implemented, these indicator systems provided a base for our work.

As we note at the beginning of this report, the committee’s proposed indicators are being offered at a time of promising changes to what is taught in K-12 science and mathematics, how students are assessed in those subjects, and how teachers are prepared to meet those changing demands. Although the proposed indicators overlap to some degree with previously developed indicators, they reflect current national priorities and recent research on teaching and learning in STEM. They also reflect the continuing interest of Congress and NSF in putting the recommendations of Successful K-12 STEM Education into action. By proposing a comparatively small set of focused, specific, and actionable indicators that are driven by research on impact, the committee hopes that the current political will and momentum will be harnessed to build and sustain a valuable monitoring and reporting system.

Suggested Citation:"Study Overview." National Research Council. 2013. Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing?. Washington, DC: The National Academies Press. doi: 10.17226/13509.
×
Page8
Suggested Citation:"Study Overview." National Research Council. 2013. Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing?. Washington, DC: The National Academies Press. doi: 10.17226/13509.
×
Page9
Suggested Citation:"Study Overview." National Research Council. 2013. Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing?. Washington, DC: The National Academies Press. doi: 10.17226/13509.
×
Page10
Suggested Citation:"Study Overview." National Research Council. 2013. Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing?. Washington, DC: The National Academies Press. doi: 10.17226/13509.
×
Page11
Next: Indicators for Measuring Improvements to the U.S. K-12 STEM Education System »
Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing? Get This Book
×
Buy Paperback | $27.00 Buy Ebook | $21.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Following a 2011 report by the National Research Council (NRC) on successful K-12 education in science, technology, engineering, and mathematics (STEM), Congress asked the National Science Foundation to identify methods for tracking progress toward the report's recommendations. In response, the NRC convened the Committee on an Evaluation Framework for Successful K-12 STEM Education to take on this assignment. The committee developed 14 indicators linked to the 2011 report's recommendations. By providing a focused set of key indicators related to students' access to quality learning, educator's capacity, and policy and funding initiatives in STEM, the committee addresses the need for research and data that can be used to monitor progress in K-12 STEM education and make informed decisions about improving it.

The recommended indicators provide a framework for Congress and relevant deferral agencies to create and implement a national-level monitoring and reporting system that: assesses progress toward key improvements recommended by a previous National Research Council (2011) committee; measures student knowledge, interest, and participation in the STEM disciplines and STEM-related activities; tracks financial, human capital, and material investments in K-12 STEM education at the federal, state, and local levels; provides information about the capabilities of the STEM education workforce, including teachers and principals; and facilitates strategic planning for federal investments in STEM education and workforce development when used with labor force projections. All 14 indicators explained in this report are intended to form the core of this system. Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing? summarizes the 14 indicators and tracks progress towards the initial report's recommendations.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!