Indicators for Monitoring Undergraduate STEM Education (2018) / Chapter Skim
Currently Skimming:
7 Implementing the Indicator System
7 Implementing the Indicator System
Pages 177-200
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 177... ...
The chapter ends with the committee's conclusion about moving forward, including a caution about the intended use of the proposed indicator system. OPTION 1: CREATE A NATIONAL STUDENT UNIT RECORD DATA SYSTEM A national student unit record data system incorporating administrative data on individual students would provide reliable and usable data for many of the proposed indicators focused on students' progress through 177
|
From page 178... ...
A student unit record data system would allow longitudinal analyses of trends over time for 8 of the 21 proposed indicators: • Indicator 2.1.2: Entrance to and persistence in STEM academic programs • Indicator 2.2.1: Diversity of STEM degree and certificate earners in comparison with diversity of degree and certificate earners in all fields • Indicator 2.2.2: Diversity of students who transfer from 2-year to 4-year STEM programs in comparison with diversity of students in 2-year STEM programs • Indicator 2.2.3 Time to degree for students in STEM academic programs • Indicator 3.1.1 Completion of foundational courses, including de velopmental education courses, to ensure STEM program readiness • Indicator 3.2.1: Retention in STEM programs, course to course and year to year • Indicator 3.2.2: Transfers from 2-year to 4-year STEM programs in comparison with transfers to all 4-year programs • Indicator 3.3.1: Number of students who attain STEM credentials over time, disaggregated by institute type, transfer status, and de mographic characteristics In this option, the federal government would require institutions to collect and provide to the national system standardized unit record data on student educational experiences and activities. Currently, the creation of such a system is prohibited by the 2008 amendments to the Higher Educa 1 The National Center for Education Statistics has added new survey components to begin capturing information on part-time and transfer students: see Chapter 6.
|
From page 179... ...
that would amend the Higher Education Act to repeal the current ban on a n ational student unit record data system and direct the NCES to create such a system. If the bills became law, NCES, when creating the new system, could take advantage of the lessons learned from the many state higher education systems and multi-institution education reform consortia that have successfully collected and used unit record student data to monitor undergraduate education.
|
From page 180... ...
analyzed various approaches to developing a national student unit record data system to be overseen by the Department of Education. Like Cunningham and Milam (2005)
|
From page 181... ...
estimated that moving to a student unit record system would reduce reporting time by about 60 percent, saving around 88.6 hours per institution per year. The current lack of a student unit record data system makes it difficult to develop a national picture of and to monitor trends over time among the nation's postsecondary students and institutions; this difficulty applies to undergraduate STEM education.
|
From page 182... ...
182 INDICATORS FOR MONITORING UNDERGRADUATE STEM EDUCATION TABLE 7-1 Continued Objective Indicator Proposed Data Source 1.3 An institutional climate 1.3.1 Use of valid measures Renewed and expanded that values undergraduate of teaching effectiveness NSOPF STEM instruction 1.3.2 Consideration of Renewed and expanded evidence-based teaching NSOPF in personnel decisions by departments and institutions 1.4 Continuous No indicators: see improvement in STEM "Challenges of Measuring teaching and learning Continuous Improvement" in Chapter 3. 2.1 Equity of access to 2.1.1 Institutional structures, Extended and expanded BPS high-quality undergraduate policies, and practices that STEM educational strengthen STEM readiness programs and experiences for entering and enrolled college students 2.1.2 Entrance to and Unit record data system persistence in STEM educational programs 2.1.3 Equitable student Extended and expanded BPS participation in evidence based STEM educational practices 2.2 Representational equity 2.2.1 Diversity of STEM Unit record data system among STEM credential degree and certificate earners earners in comparison with diversity degree and certificate earners in all fields 2.2.2 Diversity of students Unit record data system who transfer from 2-year to 4-year STEM programs in comparison with diversity in 2-year STEM programs 2.2.3 Time to degree for Unit record data system students in STEM academic programs 2.3 Representational 2.3.1 Diversity of STEM Revised IPEDS Human diversity among STEM instructors in comparison Resources Survey instructors with diversity of STEM graduate degree holders
|
From page 183... ...
IMPLEMENTING THE INDICATOR SYSTEM 183 TABLE 7-1 Continued Objective Indicator Proposed Data Source 2.3.2 Diversity of STEM Revised IPEDS Human graduate student instructors Resources Survey in comparison with diversity of STEM graduate students 2.4 Inclusive environments 2.4.1 Students pursuing Extended and expanded BPS in institutions and STEM STEM credentials feel departments included and supported in their academic programs and departments 2.4.2 Instructors teaching Renewed and expanded courses in STEM disciplines NSOPF feel supported and included in their departments 2.4.3 Institutional Renewed and expanded practices that are culturally NSOPF responsive, inclusive, and consistent across the institution 3.1 Foundational 3.1.1 Completion of Unit record data system preparation for STEM for foundational courses, all students including developmental education courses, to ensure STEM program readiness 3.2 Successful navigation 3.2.1 Retention in STEM Unit record data system into and through STEM programs, course to course programs of study and year to year 3.2.2 Transfers from 2-year Unit record data system to 4-year STEM programs in comparison with transfers to all 4-year programs 3.3 STEM credential 3.3.1 Number of students Unit record data system attainment who attain STEM credentials over time, disaggregated by institution type, transfer status, and demographic characteristics NOTES: BPS, Beginning Postsecondary Students Longitudinal Study; IPEDS, Integrated Postsecondary Education Data System; NSOPF, National Study of Postsecondary Faculty.
|
From page 184... ...
In comparison with the first option, this option would place a greater burden on institutions of higher education. In Option 1, institutional research staff would only have to upload student unit record data to the national student unit record system.
|
From page 185... ...
IMPLEMENTING THE INDICATOR SYSTEM 185 TABLE 7-2 Data for Indicators in Option 2 Objective Indicator Proposed Data Source 1.1 Use of evidence- 1.1.1 Use of evidence-based Renewed and expanded based STEM educational STEM educational practices NSOPF practices both in and in course development and outside of classrooms delivery 1.2.1 Use of evidence-based Renewed and expanded STEM educational practices NSOPF outside the classroom 1.2 Existence and use of 1.2.1 Extent of instructors' Renewed and expanded supports that help STEM involvement in professional NSOPF instructors use evidence- development based learning experiences 1.2.2 Availability of support Renewed and expanded or incentives for evidence- NSOPF based course development or course redesign 1.3 An institutional climate 1.3.1 Use of valid measures Renewed and expanded that values undergraduate of teaching effectiveness NSOPF STEM instruction 1.3.2 Consideration of Renewed and expanded evidence-based teaching NSOPF in personnel decisions by departments and institutions 1.4 Continuous No indicators: see improvement in STEM "Challenges of Measuring teaching and learning Continuous Improvement" in Chapter 3. 2.1 Equity of access to 2.1.1 Institutional structures, Extended and expanded BPS high-quality undergraduate policies, and practices that STEM educational strengthen STEM readiness programs and experiences for entering and enrolled college students 2.1.2 Entrance to and Extended and expanded BPS persistence in STEM educational programs 2.1.3 Equitable student Extended and expanded BPS participation in evidence based STEM educational practices continued
|
From page 186... ...
186 INDICATORS FOR MONITORING UNDERGRADUATE STEM EDUCATION TABLE 7-2 Continued Objective Indicator Proposed Data Source 2.2 Representational equity 2.2.1 Diversity of STEM Revised and expanded IPEDS among STEM credential degree and certificate earners earners in comparison with diversity of degree and certificate earners in all fields 2.2.2 Diversity of students Revised and expanded IPEDS who transfer from 2-year to 4-year STEM programs in comparison with diversity in 2-year STEM programs 2.2.3 Time to degree for Revised and expanded IPEDS students in STEM academic programs 2.3 Representational 2.3.1 Diversity of STEM Revised IPEDS Human diversity among STEM instructors in comparison Resources Survey instructors with diversity of STEM graduate degree holders 2.3.2 Diversity of STEM Revised IPEDS Human graduate student instructors Resources Survey in comparison with diversity of STEM graduate students 2.4 Inclusive environments 2.4.1 Students pursuing Extended and expanded BPS in institutions and STEM STEM credentials feel departments included and supported in their academic programs and departments 2.4.2 Instructors teaching Renewed and expanded courses in STEM disciplines NSOPF feel supported and included in their departments 2.4.3 Institutional Renewed and expanded practices that are culturally NSOPF responsive, inclusive, and consistent across the institution
|
From page 187... ...
Expanding IPEDS Under this option, IPEDS surveys would be expanded to require institutions to report on several measures that have been developed and tested in voluntary data collections by states and higher education reform consortia: see Box 7-1. These new measures, like the committee's proposed indicators, are designed to represent important dimensions of undergraduate education in a readily understandable form.
|
From page 188... ...
The proposed measures include 20 measures of students' progression and completion that are relevant to the committee's proposed indicators, and they have been used by institutions, states, and education reform consortia to collect and interpret data.
|
From page 189... ...
These additional measures would not only support the proposed indicator system, but would also improve the capacity of IPEDS, allowing policy makers and higher education leaders to track students' progress and completion generally, across all fields or majors. Different combinations of the proposed additional student measures have been used in voluntary data-sharing programs among institutions participating in higher education reform consortia, such as Achieving the Dream, Completion by Design, and Complete College America, and by some state higher education data systems (see Chapter 6)
|
From page 190... ...
Since 2014, postsecondary institutions have been required to report to NSLDS unit student record data on students' program of study selection for those programs with enrolled students who receive Title IV aid.4 These reports do not capture students' intentions as early as the measure of program of study selection outlined above, and they miss the 30 percent of students who do not receive or apply for some type of federal aid (Miller, 2016)
|
From page 191... ...
. The participating institutions shared their student unit record data, including all full-time, part-time, and transfer students, and analyzed the data to report on measures similar to those that would be added to IPEDS in this option.
|
From page 192... ...
In this option, the federal government or a private foundation would seek to tap these existing data resources by commissioning research on the feasibility of creating a nationally representative sample of postsecondary institutions that are already reporting the measures outlined in Option 2. Specifically, the Department of Education, NSF, or a private foundation would commission research to first identify institutions that are already submitting student unit record data to a consortia or their state data systems.
|
From page 193... ...
To encourage participation and reduce the burden of additional data collection, federal agencies or foundations might offer to reimburse institutions for the costs of additional staff time required to share and analyze existing student unit record data and gather additional data. Assuming that a nationally representative sample could be assembled, one potential limitation is that Completion by Design (CBD)
|
From page 194... ...
delivery sample of all types of 2-year and 4-year institutions 1.2.1 Use of evidence-based Same as above STEM educational practices outside the classroom 1.2 Existence and use of 1.2.1 Extent of instructors' Revised and expanded supports that help STEM involvement in professional proprietary surveys to include use evidence-based STEM development a nationally representative learning experiences sample of all types of 2-year and 4-year institutions 1.2.2 Availability of support Same as above or incentives for evidence based course development or course redesign 1.3 An institutional climate 1.3.1 Use of valid measures Revised and expanded that values undergraduate of teaching effectiveness proprietary surveys to include STEM instruction a nationally representative sample of all types of 2-year and 4-year institutions 1.3.2 Consideration of Same as above evidence-based teaching in personnel decisions by departments and institutions 1.4 Continuous No indicators: see improvement in STEM "Challenges of Measuring teaching and learning Continuous Improvement" in Chapter 3.
|
From page 195... ...
IMPLEMENTING THE INDICATOR SYSTEM 195 TABLE 7-3 Continued Objective Indicators Data Source 2.3 Representational 2.3.1 Diversity of STEM Nationally representative diversity among STEM instructors in comparison sample of institutions drawn instructors with diversity of STEM from appropriate voluntary graduate degree holders reform initiatives 2.3.2 Diversity of STEM Same as above graduate student instructors in comparison with diversity of STEM graduate students 2.4 Inclusive environments 2.4.1 Students pursuing Revised and expanded in institutions and STEM STEM credentials feel proprietary surveys to include departments included and supported in a nationally representative their academic programs and sample of all types of 2-year departments and 4-year institutions 2.4.2 Faculty teaching Same as above courses in STEM disciplines feel supported and included in their departments 2.4.3 Institutional practices Same as above that are culturally responsive, inclusive, and consistent across the institution 3.1 Foundational 3.1.1 Completion of Nationally representative preparation for STEM foundational courses, sample of institutions drawn for all students including developmental from appropriate voluntary education courses, to ensure reform initiatives STEM program readiness 3.2 Successful navigation 3.2.1 Retention in STEM Nationally representative into and through STEM programs, course to course sample of institutions drawn programs of study and year to year from appropriate voluntary reform initiatives 3.2.2 Transfers from 2-year Same as above to 4-year STEM programs in comparison with transfers to all 4-year programs 3.3 STEM credential 3.3.1 Number of students Nationally representative attainment who attain STEM credentials sample of institutions drawn over time (disaggregated from appropriate voluntary by institution type, transfer reform initiatives status, and demographic characteristics)
|
From page 196... ...
As noted above, legislation has been introduced in Congress to repeal the current ban on a student unit record data system and direct NCES to create it. Although creating a national student unit record data system would require investment of federal resources, the system would provide valuable information to policy makers about the status and quality of undergraduate education generally, not only in STEM fields.
|
From page 197... ...
As in option 1 and 2, additional data from surveys would be needed to support the indicators. Research, Evaluation, and Updating of the Proposed Indicator System Many of the indicators proposed by the committee represent new conceptions of key elements of undergraduate STEM education to be monitored over time.
|
From page 198... ...
A Note of Caution The proposed indicator system would create a picture of the current status of undergraduate STEM education and allow policy makers to monitor change over time, including movement toward the three goals that underlie the indicator system. Although individual institutions or consortia of institutions may wish to adopt some or all of these indicators to monitor their own STEM educational programs, the indicator system is not intended to support ranking systems or inter-institutional comparisons.
|
From page 199... ...
. Feasibility of a Student Unit Record System Within the Integrated Postsecondary Education Data System.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.