Experimental Approaches to Improving Research Funding Programs Proceedings of a Workshop (2024) / Chapter Skim
Currently Skimming:

2 Experiments by U.S. Federal Agencies
Pages 19-26

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 19...
... In this case, "if you had to pick between those two, cash was better," Handel said. With the workforce development program, the cash transfer did more than the traditional program to improve consumption, livestock wealth, income, productive assets, and subjective well-being, while the traditional workforce program did more to improve business knowledge.
From page 20...
... "No amount of cash to individuals is going to harmonize the software at the Rwandan and Ugandan border posts," Handel pointed out. But cash has been shown to have widespread positive effects, including reductions in child labor, improvements in dietary diversity, enhanced female empowerment, better use of health services, greater household income and savings, increased labor force participation, reduced malnutrition, improved school attendance, and reduction in risky sexual behaviors.
From page 21...
... face many challenges in reviewing the 600,000 or so patent applications that arrive at the office each year, including limited time to examine the applications, limited experience with the subject matter fields, and difficulty finding scientific publications to help them ascertain whether an invention is new and nonobvious. As a result, Ouellette and other researchers have asked whether scientific peer review would be a way to bring in outside expertise in judging applications.
From page 22...
... These results changed Ouellette's mind about the potential of expert peer review in the patent system: "As someone who had previously advocated for this as a way to improve patent examination, I no longer think this is the most costeffective way to help improve patents." Nevertheless, running the RCT was important, she said, because it provides a measure of the impact of this intervention and suggests that other reforms may be more cost-effective. For example, Ouellette hypothesized that simply giving patent examiners more time may be a better use of resources.
From page 23...
... Peter-Anthony Pappas was one of a group of agency employees who were tasked with addressing this request, and they developed what was essentially an RCT. Fifteen senior patent examiners were charged with examining pro se applications from various technical areas; these examiners were given specialized training that enabled them to work closely with applicants.
From page 24...
... Ouellette noted that the size of the cultural gap between patent examiners and scientific reviewers created unexpected problems in measuring the costs and benefits of the intervention. She said, "Thinking about how you translate to a different group; what is useful in a particular context; and how, as part of your experimental design, you're going to measure these costs and incorporate them into your assessment of how valuable the intervention is -- that's useful to think about on the front end." Pappas reflected on the value of RCTs, even though he had little awareness when he was creating his two groups of patent applications that he was actually creating an RCT.
From page 25...
... Highlighting such results helps fight against inertia and also facilitates setting up future RCTs. Handel pointed out that staff people at USAID have lots of ideas about potential program improvements, and the organization has taken advantage of the situation by establishing a competition to apply for funds to conduct experiments, which is how the cash benchmarking experiment got started.
From page 26...
... 26 IMPROVING RESEARCH FUNDING PROGRAMS some ways, it takes a village." Although a single academic cannot do all the work of justifying a program, groups of people can, she said, including people who can translate evidence for policymakers and enlist support at relevant government agencies.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.