Automated Research Workflows for Accelerated Discovery Closing the Knowledge Discovery Loop (2022) / Chapter Skim
Currently Skimming:
4 Automatic Research Workflows and Implications for Advancing Research Integrity, Reproducibility, and Dissemination
4 Automatic Research Workflows and Implications for Advancing Research Integrity, Reproducibility, and Dissemination
Pages 63-72
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 63... ...
. These challenges range from detrimental research practices, such as authorship misrepresentation and inappropriate use of statistical analysis, to research misconduct in the form of data falsification and fabrication.
|
From page 64... ...
. Interestingly, a report by the European Commission's Open Science Policy Platform reveals that while some research enterprise stakeholders such as research institutions and learned societies believe that significant progress has been made in addressing integrity issues, stakeholders such as research funders, libraries, publishers, and other organizations that disseminate research believe that much still needs to be done (EC, 2020)
|
From page 65... ...
It involves assessment of which data sets to prioritize for the considerable effort involved in curation, as well as training, incentives to prioritize the effort above other tasks such as conducting further experiments that might lead to further publications, and funding to support the extra effort and cost involved. Several studies have suggested that data stewards can provide research teams with this expertise (e.g., Scholtens et al., 2019)
|
From page 66... ...
The growing ubiquity and complexity of computation in the research process across many disciplines presents additional challenges to independently reproducing results. Examples of these challenges include the use of nonpublic data and code in research, the costs of retrofitting long-standing research projects with tools that automatically capture logs of computational decisions, and incomplete information about the computing environment where the research was originally performed (NASEM, 2019b)
|
From page 67... ...
One important step toward openness was the memorandum on Increasing Access to the Results of Federally Funded Scientific Research from the Office of Science and Technology Policy (Holdren, 2013) , which directed federal agencies with over $100 million in annual conduct of research and development expenditures to develop a plan to support increased public access to the results of research funded by the federal 3 For example, CRediT (Contributor Roles Taxonomy)
|
From page 68... ...
have developed services that use algorithms and text mining and/or natural language processing to support authors, editors, funders, and others in identifying relevant peer reviewers, guest editors, contributors to special issues, and others involved in the process. Some AI-based tools aim to accelerate the publication process as well as streamline the effort required to validate article submissions, which can be especially important to make high-priority areas of research available sooner.
|
From page 69... ...
A shift in how research is conducted and produced is allowing the community to rethink publishing approaches beyond simply automating existing practices, but rather to better utilize technologies aligned with the way research outputs are being produced. For example, models are now available to rapidly publish (typically within a few days)
|
From page 70... ...
As the speed of research updates created by automated research is likely to increase, we need to consider how to adequately review these outputs, especially given that peer reviewers are already overwhelmed. Publishers are developing article transfer mechanisms of various forms to minimize subsequent review as a manuscript passes between journals looking for acceptance.
|
From page 71... ...
are relevant to developing methods and platforms for disseminating ARWs as expressions of methods underlying reported work. Specific issues include standardizing formats for unstructured data and accommodating AI black boxes -- models created from data by an algorithm that are "inherently uninterpretable and complicated" (L'Heureux et al., 2017; Rudin and Radin, 2019)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.