Furthering America's Research Enterprise (2014) / Chapter Skim
Currently Skimming:
Appendix A: An Evaluation of STAR METRICS
Appendix A: An Evaluation of STAR METRICS
Pages 139-154
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 139... ...
Data collection is planned in two phases. The first, Level 1, is drawing a limited number of data fields from existing university administrative databases and includes no personally identifiable information.
|
From page 140... ...
, and the relevant performing institution account number for each record. The three files are as follows: • Individual file, which tracks salary and wage payments to uni versity employees (faculty, staff, students, and trainees)
|
From page 141... ...
POTENTIAL VALUE OF THE DATA While we believe that the value of these data might be increased by the addition of new data fields, we focus our attention in this section on potential uses of the Level I data alone. We first address the descriptive and comparative possibilities we see in the STAR METRICS data and then summarize possibilities for linking these data to existing sources of information that might expand their reach.
|
From page 142... ...
was a scientific supply company in Hampton, New Hampshire. While we had access to raw vendor file data from only a single campus, such information might provide new and useful insight into the direct local and national economic impacts of federal R&D spending on campus.
|
From page 143... ...
Such comparative and descriptive data are potentially valuable, but both their scientific utility and usefulness for public policy are limited if they cannot be linked to other data sources to enable more rigorous analyses. Linkages to Other Relevant Sources of Information We briefly consider three possible means of linking STAR METRICS data to other sources of information that could prove useful in assessing the value of research for achieving national goals.
|
From page 144... ...
FIGURE A-1 STAR METRICS collaborative network for two universities in a single quarter. NOTE: Red = research staff (research scientists, technicians, coordinators, etc)
|
From page 145... ...
Census Bureau, including the Longitudinal Business Database and the Longitudinal Employee Household Dynamics dataset, could potentially be matched to STAR METRICS using DUNS numbers. Such linkages might enable new and more accurate estimates of the direct and indirect economic impacts of federal R&D spending on university campuses, but substantial work would be required to match the data, and a quick examination of vendor and subcontractor files for a single university suggests that data quality issues may pose substantial obstacles to this linkage (see the next section on data quality)
|
From page 146... ...
; Yellow = faculty; Green = trainees (grad students, post-docs) ; Blue = clinicians.
|
From page 147... ...
A pilot effort to accomplish these very linkages supports discussion below of the extent to which current STAR METRICs data cover federal R&D expenditures and STEM degree completions. While there are other sources of funding information, including, for example, National Patterns of R&D Resources, maintained and collected by NSF's National Center for Science and Engineering Statistics (NCSES)
|
From page 148... ...
Fourteen of the institutions in Table A-2 are enrolled in TABLE A-2 Top Twenty-Five Universities by Federal R&D Expenditures, 2011 University (NSF Standardized) Federal Expenditures Johns Hopkins University 1,884,025 University of Washington, Seattle 950,293 University of Michigan All Campuses 824,752 University of Pennsylvania 707,051 University of Pittsburgh All Campuses 662,471 Stanford University 656,114 Columbia University in the City of New York 645,233 University of California, San Diego 636,879 University of Colorado All Campuses 636,278 University of Wisconsin–Madison 593,633 Duke University 585,262 University of California, San Francisco 570,116 University of California, Los Angeles 563,560 University of North Carolina at Chapel Hill 561,708 Harvard University 543,097 Yale University 519,844 University of Minnesota All Campuses 498,488 Ohio State University All Campuses 493,130 Massachusetts Institute of Technology 489,080 Cornell University All Campuses 476,583 Pennsylvania State University All Campuses 472,693 Washington University 469,490 Vanderbilt University 458,173 University of Southern California 453,283 Georgia Institute of Technology All Campuses 427,867 Total 15,779,103 NOTES: Expenditure data in thousands of constant 2000 dollars; STAR METRICS campuses in italics; R&D = Research and Development SOURCE: National Science Foundation (2009)
|
From page 149... ...
Number of Graduates University of Phoenix 21,050 Community College of the Air Force 7,223 University of Florida 6,348 Ivy Tech State College Central Office 5,801 Pennsylvania State University Main Campus 5,728 Ohio State University Main Campus 5,370 University of Michigan, Ann Arbor 5,310 Purdue University Main Campus 5,241 Texas A&M University Main Campus 5,053 University of Washington-Seattle 4,965 University of Minnesota-Twin Cities 4,956 University of Illinois at Urbana–Champaign 4,594 University of Wisconsin–Madison 4,412 ECPI College of Technology 4,403 University of California, Berkeley 4,329 University of Southern California 4,262 University of California, Los Angeles 4,209 University of California, San Diego 4,195 University of Texas at Austin 4,165 University of Central Florida 4,091 Michigan State University 3,974 North Carolina State University at Raleigh 3,918 Georgia Institute of Technology Main Campus 3,888 Arizona State University Main Campus 3,747 University of California, Davis 3,696 Total 134,928 NOTES: STAR METRICS campuses in italics; STEM = Science, Technology, Engineering, and Mathematics. Source: National Center for Education Statistics (2014)
|
From page 150... ...
While a lack of access to STAR METRICS data and a lack of resources precluded our conducting a thorough analysis of data quality, our preliminary work provides some useful hints. We focus here on key variables for data linkage in the files of a single public university.
|
From page 151... ...
Similarly, Owen-Smith's pilot effort to link STAR METRICS data for a single campus to NIH RePORTER information using NIH grant numbers suggests that accomplishing similar link ages at scale would be challenging but possible. Of 2,270 records of NIH grants that paid wages, 383 (16.8 percent)
|
From page 152... ...
They suggest that difficulties with data access and the need to reach agreements with individual reporting units, together with the narrow impact focus, greatly limit the usefulness of the database. More generally, FFRDC users often need to think of the impact of research funding from different perspectives, something that is difficult to do with the STAR METRICS database at this time.
|
From page 153... ...
Yet these questions of causal involvement arise often. The FFRDC users we consulted are not naïve and fully understand that the questions they ask are complex and perhaps even ill posed.
|
From page 154... ...
154 FURTHERING AMERICA'S RESEARCH ENTERPRISE REFERENCES National Center for Education Statistics.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.