Skip to main content

Currently Skimming:

8 EMERGING METRICS AND MODELS
Pages 69-84

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 69...
... Two separate sessions at the workshop included seven speakers who examined specific tools and approaches, from the creation of a science policy infrastructure at NSF to visual analytics that can probe data sets for unexpected findings. ASSESSING RESEARCH AT NSF Traditional measures of research outputs provide only a partial picture of the state of scientific research in the United States, said NSF Director Subra Suresh during his keynote address at the workshop.
From page 70...
... The social sciences research being sponsored by NSF offers many similar opportunities to leverage natural sciences research. In the context of clean energy, for example, Suresh has been talking with officials at the Department of Energy on how social, behavioral, and economic research sponsored by NSF can contribute to research supported by the department.
From page 71...
... New computer technologies could gather information to help answer some of these questions and shape human capital policies within the financial constraints expected in the future." A fifth theme was the need to measure the impacts of NSF funded research intelligently and over a long period of time. Although a good deal of the research NSF funds has purely scientific motivations, some of it has helped generate entirely new industries making significant contributions to the economy, Suresh observed.
From page 72...
... THE STAR METRICS PROJECT In 2005, OSTP Director John Marburger observed at a AAAS policy forum that he found it very difficult to provide an evidence-based answer to the question, "How can the federal government optimize its investments in science? " An interagency working group under the title of Science of Science Policy came to a similar conclusion in 2008, noting that no solid theoretical and empirical basis exists for deciding the level or allocation of scientific investments.
From page 73...
... This represents what Lane termed a "sea change" from the current data infrastructure on public science. For 50 years, the science agencies have essentially been proposal processing and award administration factories, she said.
From page 74...
... The data also make it possible to calculate the total number of individuals supported by research funding, along with the number of positions supported outside universities through vendor and subcontractor funding. "Not a single PI lifted a pen or typed a keyboard to enable us to pull this information, yet the information is very powerful and can be used to inform federal and state lawmakers." Future Plans The next step in STAR Metrics' development is to develop the main features of the phase II platform that will compile information from individual researchers, commercial publication databases, administrative data, and other sources to capture as much information about scientific activities as possible.
From page 75...
... But the evidence behind such claims is patchy." Building an Empirical Framework Continuing the discussion of STAR Metrics, Bertuzzi described it as a way of combining and linking input measures with economic, scientific, and social outcomes. For example, when a new discovery or technology is licensed to a company, the license represents a return on research investments.
From page 76...
... STAR Metrics will make it possible to "disentangle and unpack all the complexity of the network that eventually led to that particular discovery." A potential practical application would be to look for the common features of successful discovery processes and then try to replicate them. CREATING KNOWLEDGE FROM DATA The outputs of research historically have been viewed as consisting of papers, patents and human resources, noted Ian Foster, Arthur Holly Compton Distinguished Service Professor and Chan Soon-Shiong Scholar at the University of Chicago.
From page 77...
... For example, the MyExperiment project seeks to make the sharing of computational procedures, data, and software as easy as sharing images on a social networking site. The site also makes it possible to share workflows and reports on how often they are used and for what purpose.
From page 78...
... Scientific breakthroughs led to hundreds of new firms. Consolidation occurred when scientific advances slowed, with some firms growing and others failing.
From page 79...
... NSF funding for nanotechnology has had a large impact in the field, Zucker observed, contributing to large increases in published nanoscale articles and significant growth in nanoscale patenting. The impacts of star scientists vary across S and T areas in proportion to technological opportunity, said Zucker.
From page 80...
... Through such processes as text mining and entity identification, it produces multiple interactive visualizations of the content of the documents for exploration. Finally, Stasko mentioned a system called Ploceus (named after a weaver bird that creates elaborate nests)
From page 81...
... "It has been a long time in coming, and we've talked about it for a long time, but we are now at a point where we can glimpse that it may actually be happening." The only thing that can protect science funding, he said, is demonstrating the long-term and diffuse but tremendously important impacts of science, "and that requires very extensive and complicated data." One way to build such a database will be to take advantage of automated data capture. Once the framework for the system has been created, huge amounts of data can be collected automatically by searching the web.
From page 82...
... "I know NIH is the 800-pound funding gorilla, but there are other sciences and other industries out there." The indirect effects of research funding can be very difficult to track. Things like the accumulation of human capital or the spillover effects from research have very long lags and diffuse impacts.
From page 83...
... Researchers are understandably confused, he said, about how many of these considerations to incorporate into their research proposals, how much of the burden to place on the individual versus the department versus the school versus the institution, and how to consider such factors as economic impact and workforce development. "This is very much a work in progress." A number of groups are working in parallel and in conversation with one another, he said, ideally leading to clarity rather than confusion on this issue.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.