Experimental Approaches to Improving Research Funding Programs Proceedings of a Workshop (2024) / Chapter Skim
Currently Skimming:

6 Structuring Support for Experimentation in Government
Pages 53-62

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 53...
... , have direct experience with the issue, in that they have been connecting scientists and researchers on the outside of government with people on the inside. FOSTERING AN INTERDISCIPLINARY EFFORT The creation of innovative science policy is a classic interdisciplinary effort in which all parties benefit from a flow of ideas, people, and conversations across economics, policy, academia, industry, science, and government, said Jordan Dworkin (Federation of American Scientists [FAS]
From page 54...
... A major focus of policy entrepreneurship has been exploring ideas to build out the structural diversity of the science and technology ecosystem, he said. For example, FAS has been involved in supporting innovative institutional designs and funding models across government, with a particular focus on testing a wider range of portfolios and funding systems.
From page 55...
... Documents are posted on agency websites and are linked centrally at the website evaluation.gov. The Biden administration reinforced the Evidence Act through a presidential memorandum on restoring trust in government through scientific integrity and evidence-based policymaking, which stated that "scientific and technological information, data, and evidence are central to the development and iterative improvement of sound policies, and to the delivery of equitable programs, across every area of government." OMB also released guidance that integrates, clarifies, and enhances prior guidance on these topics; reiterates the expectation that agencies use evidence whenever possible to further both mission and operations and build evidence where it is lacking; and emphasizes the importance of building a culture of evidence to improve decision making.
From page 56...
... In response, Epstein explained, OMB issued guidance on evaluation standards and practices that articulates five standards -- relevance and utility, rigor, independence and objectivity, transparency, and ethics -- that agencies must apply to all evaluation activities. The guidance should be implemented by evaluation officers, agency evaluators, and staff in related functions; it also includes ten leading practices that can support implementation of the standards.
From page 57...
... 91-648) , which temporarily assigns workers to tasks across governmental boundaries, Manoli and his colleagues randomly selected some individuals to receive simplified notices while others received unchanged, baseline notices.
From page 58...
... External feedback from peer reviewers has driven scientific innovation, implementation, and policy building at the agency, and the IRS is now collaborating with other academic researchers on the customer experience, tax compliance, and audits. Through the IPA, Manoli, who had previously worked with the Volunteer Income Tax Assistance program, was allowed to work at the IRS as if he were an employee.
From page 59...
... Some agencies are also experimenting with ways of getting students and scholars involved, such as the summer data challenge posed by the Department of Labor to use existing data to answer specific research questions. Similarly, a new Analytics for Equity pilot is a partnership among federal agencies and the National Science Foundation to provide small grants for projects aligned with agencies' equity-focused learning agendas.3 "Those seem like great opportunities for students to get a foot in the door," Epstein stated.
From page 60...
... But the infrastructure needed to collect that kind of information is not discussed enough, he said, and both public and private support need to be marshalled to build it. In response to a question about how collaborators can compensate for federal staffing shortages, Dworkin said that FAS is eager to support agencies with shortages in finding fellows or external collaborators who can help pursue evaluations or experiments.
From page 61...
... This process would resemble a natural experiment much more than a randomized controlled trial, but Manoli said it is still likely to be "really helpful." Eva Guinan (Harvard Medical School) , who had spoken earlier in the workshop, raised the issue of validating data collection instruments, based on her own experience with the problems encountered when using the World Management Survey with faculty members, who respond "in very individualized ways." Manoli responded to this issue by pointing out the possibility of using administrative data, which generally does go through checks and validations as part of compliance measures.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.