National Academies Press: OpenBook

Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs (2013)

Chapter: Appendix D: Program Evaluation Information Provided by the Agencies

« Previous: Appendix C: Workshop Participants
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Program Goals Evaluation Criteria for Success Successes
USGS National Cooperative Geologic Mapping Program (EdMap) • Train students in geologic mapping
• Help fund academic research
• Prepare students for real-world careers in the geosciences
• Create collaborations between academic institutions, state geological surveys, and the USGS
• Contribute to national geologic mapping efforts
• Survey of participants and providers • Number of students educated in geologic mapping techniques • Program benefits participants
USGS/NAGT Cooperative Summer Field Training Program • Provide an opportunity to partner a highly able intern with a quality science mentor to work on a meaningful research project
• Present research results at professional meetings or in refereed publications
• Surveys of participants and providers • Number of participants
• Success of participants in earth science careers
• Excellent interns are placed in research units
• Program quality is recognized by USGS and earth science academic community
USGS Youth Internship Program • Stimulate ongoing interest in science among college undergraduates
• Build a pool of well-prepared new college graduates who could fill vacancies in the USGS
• Reports by participants and providers • Fraction of participants that obtained permanent jobs
• Program satisfaction
• Most available slots are filled
• Many interns accept positions at USGS
USGS Hydrologic Technician Internship Program • Stimulate ongoing interest in water science among college undergraduates
• Build a pool of well-prepared new college graduates who could fill vacancies in the USGS
• Reports by participants and providers • Fraction of participants that obtained permanent jobs
• Program satisfaction
• Most available slots are filled
• Many interns accept positions at USGS
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
NSF Earth Sciences Research Experience for Undergraduates (REU) Program • Support research participation by undergraduate students • External, formal evaluation in 2006
• Reports by providers
• Number of applicants to REU sites
• Number of former participants in an earth science career
• Interest of undergraduate students
NSF Opportunities for Enhancing Diversity in the Geosciences (OEDG) Program • Increase participation in the geosciences by African Americans, Hispanic Americans, Native Americans, Native Pacific Islanders, and persons with disabilities
• Increase the perceived relevance of the geosciences among broad and diverse segments of the population
• External evaluation by an expert consulting firm using a logic model
• External reviews every 3 years by a visiting committee
• Reports by providers
• None specified • New degree programs at minority-serving institutions (MSIs)
• New partnerships between MSIs and 4-year and research universities
• Better understanding how to recruit and retain minorities in earth science
• Slight increase in number of undergraduate majors
NSF Earth Sciences Postdoctoral Fellowships • Recognize beginning investigators of significant potential
• Provide these individuals with research and education experience that will establish them in positions of leadership in the scientific community
• External reviews every 3 years by a visiting committee
• Reports by participants on research advances
• Information on career development of former fellows
• None stated • A high fraction of participants are in tenure-track positions
• Participants are active scientifically
• Some participants have become mentors
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Program Goals Evaluation Criteria for Success Successes
NSF Geoscience Education (GeoEd) Program • Improve the quality and effectiveness of formal and informal geoscience education at all educational levels
• Increase the number of students pursuing geoscience education and career paths
• Broaden participation of traditionally underrepresented groups in the geosciences
• Promote public engagement in Earth system science
• External reviews every 3 years by a visiting committee
• Reports by providers
• Widespread use of models for understanding geoscience content and strategies for developing spatial and systems-thinking skills
• Number of peer-reviewed publications
• Continued research support
• Incubation of the Digital Library for Earth System Education
• Incorporation of big ideas that all citizens should know into the next-generation science standards
• Creation of geoscience educators who use best practices from STEM education research to strengthen formal and informal education
NSF/NOAA/NASA Global Learning and Observations to Benefit the Environment (GLOBE) Program • Improve student understanding of environmental and Earth system science across the curriculum
• Contribute to scientific understanding of Earth as a system
• Build and sustain a global community of students, teachers, scientists, and citizens
• Engage the next generation of scientists and global citizens in activities to benefit the environment
• Internal evaluation (not specified) • None specified • A large number of K–16 students are engaged worldwide
• A large community of teachers with improved pedagogy and content knowledge
NSF Geoscience Teacher Training (GEO-Teach) Program • Improve the quality of geoscience instruction, primarily at middle and high school levels ESSEA project:
• Internal evaluation based on data from course site, participant surveys and products, and provider surveys and products
ESSEA project:
• Resources developed and how much they have been used
• Effect of courses on learner’s knowledge of Earth system science and how to
ESSEA project:
• Institutionalization of effective professional development and training methods over many universities
• Funding to create more modules
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
teach it
• Effect of the program on faculty and their institutions
TESSE project:
• External evaluation planned based on participant surveys, focus groups, phone interviews, observation of activities, and standardized teacher assessment
TESSE project:
• Extent to which project goals, activities, and outcomes are understood by providers and participants
• Extent to which program activities met stated goals
• Extent to which teacher opportunities improved their understanding of earth science and ability to teach it
• Success of processes and structures to improve collaboration
TESSE project:
• Identification of challenges in extending a professional development model from research institutions to HBCUs
DOE Office of Science Graduate Fellowship (SCGF) Program • Encourage the development of the next generation of scientific and technical talent in the United States who will pursue careers in research critical to the Office of Science mission • Program goals, outputs, and outcomes defined in a logic model
• Reports by providers and participants
• Longitudinal tracking of participants
• External review of WDTS portfolio in 2010
• External reviews every 3 years by a visiting committee
Long-term outcomes include
• Pursuit of a career in DOE Office of Science-related research in academia, a DOE laboratory, or industry
• Program is too young to determine. The first cohort is nearing the end of the 3-year support period
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Program Goals Evaluation Criteria for Success Successes
DOE Summer of Applied Geophysical Experience (SAGE) Program • Introduce students in geophysics and related fields to “hands-on” geophysical exploration and research
• Apply research and geophysical exploration methods to solve specific problems
• Reports by participants and providers • Rate of continuation to graduate school and careers in geophysics
• Support and feedback from industry participants and visitors
• Presentations at professional meetings and publications
• None specified
DOE Science Undergraduate Laboratory Internships (SULI) Program • Encourage undergraduate students to pursue STEM careers • Program goals, outputs, and outcomes defined in a logic model
• Pre- and postparticipation surveys of participants
• Longitudinal tracking of participants
• External review of DOE laboratories’ execution of the program
• External review of WDTS portfolio in 2010
• External reviews every 3 years by a visiting committee
Near-term outcomes include
• Completion of STEM degree
• Pursuit of advanced degree in STEM
• Preparedness for STEM career
• Knowledge of DOE mission science and technology Long-term outcomes include
• Pursuit of a career in DOE Office of Science-related research in academia, a DOE laboratory, or industry
• More than 90 percent of participants report that the internship helped prepare them for a STEM career
• More than three-quarters of participants (2003-2011) are pursuing a career in STEM
DOE Community College Internship (CCI) Program • Encourage community college students to enter technical careers relevant to the DOE mission • Program goals, outputs, and outcomes defined in a logic model
• Pre- and postparticipation surveys of participants
• Longitudinal tracking of participants
• External review of DOE laboratories’ execution of the program
Near-term outcomes include
• Completion of STEM degree
• Pursuit of certification in STEM
• Preparedness for STEM career
• Knowledge of DOE mission science and technology
• None specified
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
• External review of WDTS portfolio in 2010
• External reviews every 3 years by a visiting committee
Long-term outcomes include
• Pursuit of a career in DOE Office of Science-related research in academia, a DOE laboratory, or industry
NASA Gravity Recovery and Climate Experiment (GRACE) programs None specified. Proposed outcomes are:
• Conduct GRACE activities with 500 teachers with priority given to educators from underrepresented areas or with underrepresented audiences
• Increase the knowledge of 500 secondary teachers about the Earth’s history, Earth system science, and global climate change
• Increase the knowledge of 500 students about earth science and global climate change
• Make available selected activities of “changing mass = changing earth” programs through the University of Texas Center for Space Research Web site
• Incorporate NASA satellite data and resources into the activities
• Reports by providers
• External evaluations are planned
• Number of teacher and student participants
• Field trip opportunities for students
• Increase in knowledge of global climate change
• Student interest in STEM careers
• None specified
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Program Goals Evaluation Criteria for Success Successes
USDA 4-H Environmental Education/Earth Science programs • Increase awareness of science among youth
• Improve science skills and knowledge among youth
• Increase awareness of opportunities to contribute to society using science skills
• Increase life skills among youth
• Internal evaluation (not specified) • None specified • 4-H enrollment
• Professional development has positive impact on performance and satisfaction
• High-quality programs
• Creation of a new research database
• Participants excel beyond their peers
USDA Agriculture and Food Research Initiative (AFRI) projects • Increase technical competency in priority area(s) to ensure that the United States remains globally competitive in the knowledge age • None yet • Number of participants • Participants complete their degree
USDA National Institute of Food and Agriculture Fellowships Grant Program • Strengthen the ability of the nation’s scientific community to meet the current and future challenges facing agriculture, forestry and food systems
• Develop the technical and academic competence of doctoral candidates
• Develop the research independence and teaching credentials of postdoctoral scientists
• Reports by providers • None specified (project dependent) • None specified (project dependent)
EPA Greater Research Opportunities (GRO) Undergraduate Fellowship Program • Support undergraduate study in environmentally related fields • External review by a visiting committee in 2009
• Reports by participants
• Number of awards
• Number of completed degrees
• Topic area distribution
• Professional success of former fellows
• None specified
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
EPA Science to Achieve Results (STAR) Graduate Fellowship Program • Defray the cost of study leading to advanced degrees in environmental science fields • External review by a visiting committee in 2009
• Reports by participants
• NRC review of STAR in 2003
• Number of awards
• Number of completed degrees
• Topic area distribution
• Professional success of former fellows
• None specified
NOAA Educational Partnership Program (EPP) With Minority Serving Institutions Cooperative Science Centers:
• Increase the number of educated, trained, and graduated students, particularly from underrepresented communities in STEM fields that directly support NOAA’s mission
• Increase collaborative research efforts between NOAA scientists and researchers at NOAA EPP cooperative science centers
Cooperative Science Centers:
• External review of cooperative science centers in their third year
• NRC review of education program in 2010
• Surveys of providers and participants
Cooperative Science Centers:
• Annual number of students from underrepresented communities who are trained and graduate in NOAA mission sciences
• Annual number of students who are trained and graduate in NOAA mission sciences
• Number of students completing experiential opportunities at NOAA facilities
• Number of students who are hired by NOAA; NOAA contractors; other environmental, natural resource, and science agencies; academia; and the private sector
• Number of collaborative research projects undertaken between NOAA and EPP cooperative science centers in support of NOAA operations
• Number of students and faculty who participate in and complete postdoctoral-level research programs in support of the NOAA mission
Cooperative Science Centers:
• None specified
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Program Goals Evaluation Criteria for Success Successes
• Number of peer-reviewed papers published in NOAA mission sciences by faculty, postdoctoral fellows, and students sponsored by NOAA EPP
• Funds leveraged with NOAA EPP funds
• Number of participants engaged in NOAA mission-relevant learning opportunities
Graduate Sciences Program:
• Increase the number of educated, trained, and graduated students, particularly from underrepresented communities in STEM fields that directly support NOAA’s mission
Graduate Sciences Program:
• NRC review of education program in 2010
• Surveys of providers and participants
Graduate Sciences Program:
• Number of graduate sciences program students hired by NOAA
Graduate Sciences Program:
• None specified
Undergraduate Scholarship Program:
• Increase the number of educated, trained, and graduated students, particularly from underrepresented communities in STEM fields that directly support NOAA’s mission
Undergraduate Scholarship Program:
• NRC review of education program in 2010
• Surveys of providers and participants
Undergraduate Scholarship Program:
• Number of undergraduate scholarship students who are educated, trained, and graduate in NOAA mission sciences
• Number of undergraduate scholarship students who pursue graduate work in NOAA mission sciences
Undergraduate Scholarship Program:
• None specified
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
• Number of undergraduate scholarship students who are hired by NOAA, NOAA contractors, and other natural resources and science agencies
NPS/GSA Geoscientists-inthe-Parks Program • Provide on-the-job earth science training for America’s youth
• Build technical capacity for parks
• Enhance the public’s understanding of earth science
• Reports by providers • Completion of unmet earth science projects in NPS units
• Communication of earth science to the public and park staff who are not geologists
• Program growth
• Program has grown from 20 positions per year in 2007 to more than 100 positions per year in 2012
NPS/NAGT Geoscience-Teachers-in-Parks Program • Provide cooperative exchange of learning and scientific research between the park, local earth science teachers, and communities
• Advance educational and interpretive opportunities at the park
• Develop a lifelong network with local communities, schools, and the park
• Participant feedback • None specified • Teachers gain career enhancement opportunities
• Program increases the recognition of NAGT as a leader in promoting earth science education
• Teachers sign up as park volunteers and return to the park after completing their internship
• Products developed during the internship can be used by park staff for other educational purposes
• Teachers continue using the park as an educational tool or destination
• Students become more familiar with the park as an educational resource and more aware of the need to protect park resources
• Teachers and students become stewards of parks and the environment
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Program Goals Evaluation Criteria for Success Successes
NPS National Fossil Day • Promote public awareness and stewardship of fossils
• Foster a greater appreciation of the scientific and educational value of fossils
• Survey of participants • None specified • Rapid expansion of partners embracing the mission
• Positive feedback from partners and the media
Smithsonian Leadership and Assistance for Science Education Reform (LASER) Program • Improve K–12 science education • External reviews of the program and its components are planned • Increased student achievement • Statewide implementation of program (e.g., Washington state LASER)

NOTE: DOE = Department of Energy; EPA = Environmental Protection Agency; ESSEA = Earth System Science Education Alliance; GSA = Geological Society of America; HBCUs = historically Black colleges and universities; MSI = minority-serving institution; NAGT = National Association of Geoscience Teachers; NASA = National Aeronautics and Space Administration; NOAA = National Oceanic and Atmospheric Administration; NPS = National Park Service; NSF = National Science Foundation; STEM = science, technology, engineering, and mathematics; TESSE = Transforming Earth Systems Science Education; USDA = U.S. Department of Agriculture; USGS = U.S. Geological Survey; WDTS = Office of Workforce Development for Teachers and Scientists.

Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 67
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 68
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 69
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 70
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 71
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 72
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 73
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 74
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 75
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 76
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 77
Suggested Citation:"Appendix D: Program Evaluation Information Provided by the Agencies." National Research Council. 2013. Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs. Washington, DC: The National Academies Press. doi: 10.17226/18369.
×
Page 78
Next: Appendix E: Biographical Sketches of Committee Members »
Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs Get This Book
×
Buy Paperback | $43.00 Buy Ebook | $34.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Earth science, which in this context does not include oceanic, atmospheric, and space sciences, is vital to the wellbeing of the United States and many of its issues, such as water resources, are expected to grow in importance. An earth science workforce will be needed to deal with this issues and it's important that this workforce draw on the talents of all citizens. Thus, federal education programs can be implemented to help attract and retain students on an earth science pathway; however, tight funding means agencies need to invest in programs that actually work.

As a result, the U.S. Geological Survey (USGS) Office of Science Quality and Integrity asked the National Research Council (NRC) to establish a committee to carry out a study, organized around a workshop, to address several tasks including: examining recent earth science education programs with a research or training component, both formal and informal, in these federal agencies; indentifying criteria and the results of previous federal program evaluations, and summarizing the knowledge and skills identified in recent NRC workforce reports that are needed by earth scientists in their careers.

Preparing the Next Generation of Earth Scientists: An Examination of Federal Education and Training Programs presents the committee's finding. The investigation was completed through information provided by federal agency managers and published articles and reports. A 2-day workshop was also held to examine federal earth science education programs and efforts to leverage resources. The report includes the workshop agenda, a glossary of abbreviated terms, and more.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!