Skip to main content

An Assessment of ARPA-E (2017) / Chapter Skim
Currently Skimming:

Summary
Pages 1-14

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 1...
... The ad hoc committee convened to conduct this assessment relied on quantitative and qualitative analyses of data on ARPA-E's creation of technology-focused funding programs, its decision making processes for granting awards, its management of projects and awardees, the patenting and publishing activities of awardees, and further investments made in awardee projects following ARPA-E funding. There are clear indicators that ARPA-E is making progress toward achieving its statutory mission and goals, and it cannot reasonably be expected to have completely fulfilled those goals given so few years of operation and the size of its budget.
From page 2...
... accelerating transformational technological advances in areas that industry by itself is not likely to undertake because of technical and financial uncertainty." 2 The committee's operational assessment considers how ARPA-E is organized, how it selects projects to support, how it partners with performers to manage those projects, how it actively manages projects, and what nontechnical support it provides to projects. The technical assessment outlines how ARPA-E, through its project selection and management, has made progress toward producing commercial products with the potential to transform the energy sector.
From page 3...
... , ARPA-E can be distinguished by its culture, methods, and focused mission and goals. Through the course of its deliberations and analyses of the evidence gathered for this assessment, the committee found that ARPA-E benefits from three defining organizational features: • The director provides technical and leadership skills that enable and sustain a culture of empowerment.
From page 4...
... This process is distinguishable from those of some other agencies that make funding decisions based principally or solely on numerical reviewer scores, often utilizing a strict cutoff that does not allow for discretion on the part of the agency or program directors. Strong and consistent evidence indicates that projects selected through ARPA-E's process have the potential to yield measurable outcomes at least as good as, if not better than, those of projects that would have been selected had less discretion been allowed.
From page 5...
... Finding 3-9: ARPA-E program directors actively manage projects through technical research guidance and feedback, regular and frequent assessments of progress made toward stated technical milestones, and revision of milestones in response to new findings and research discoveries. Throughout a program's life cycle -- from review of applications, through award negotiations, to completion of individual projects -- program directors engage in active project management.
From page 6...
... undertaken by entities other than the agency rather than undertaking its own R&D activities. Because the agency is tasked with seeking out transformational technological advances, it has necessarily utilized novel operational benchmarks to try to accomplish its goals.
From page 7...
... The committee recommends that ARPA-E strive to preserve this management approach that gives its program directors wide authority to develop new focused technology programs with potential to be transformative and enables them to manage projects actively through technical research guidance and feedback, regular and frequent assessments of progress toward stated milestones, and revision of milestones in response to new findings and research discoveries. This management approach is a defining organizational feature that can contribute to the agency's ability to achieve its statutory mission and goals, and helps to distinguish ARPA-E from other public funding initiatives for energy R&D.
From page 8...
... The agency could link data from its robust internal database of project-level metrics to program-level goals, including indicators of commercial and noncommercial outcomes over the short and long terms; connect those goals to standard, observable innovation metrics; and then translate those metrics into progress toward achieving the agency-level mission and goals. Such a framework would need to include a system for tracking performers postfunding for at least 10 years, and very likely longer, to capture technologies that are transferred in arms-length transactions along with other ways of observing technology deployment.
From page 9...
... Finding 4-4 describes the importance for ARPA-E of seeking high-risk, potentially transformative technologies and overlooked, "off-roadmap" opportunities pursued by neither private firms nor other funding agencies, including other programs and offices within DOE, as a way to position itself to accomplish its mission. ARPA-E's underlying organizational features include encouraging its program directors to seek potentially high-impact projects and recognizing that many of its projects will produce only valuable knowledge, including knowledge of research pathways that should not be pursued further, instead of commercialized products.
From page 10...
... While some elements of ARPA-E's approach may be difficult to scale or translate to other programs and offices, there is great benefit in exploring their adaptability and suitability. Of particularly high value would be finding suitable ways to incorporate such key features as term-limited program managers, use of constructive engagement among program directors to sharpen the focus of programs, the degree of operational freedom accorded to program managers, and the risk-taking orientation of programs.
From page 11...
... Finding 3-2: Program director discretion enables ARPA-E to fund relatively risky projects, with no indication that average project performance in the short term is reduced. Finding 3-3: Program directors are continuously engaged in ongoing projects, as reflected by the altering of project milestones, budgets, and timelines, to help ensure that projects support ARPA-E's mission and goals.
From page 12...
... Finding 4-3: While 6 years is not long enough to produce observable evidence of widespread deployment of funded technologies, there are clear indications that ARPA-E is making progress toward its statutory mission and goals. Finding 4-4: One of ARPA-E's strengths is its focus on funding high-risk, potentially transformative technologies and overlooked, "off-roadmap" opportunities pursued by neither private firms nor other funding agencies, including other programs and offices within DOE.
From page 13...
... Recommendation 3-1: ARPA-E should preserve its distinctive and flexible management approach that empowers program directors and stresses active project management. Recommendation 3-2: ARPA-E should continue to hire exceptional program directors and empower them to create programs and manage projects.
From page 14...
... Recommendation 4-3: ARPA-E should continue to use processes designed to identify and support unexplored opportunities that hold promise for resulting in transformational technological advances. Recommendation 4-4: ARPA-E should be careful not to misinterpret or extend its interpretation of its original authorizing statute, whose careful language is appropriate to the agency's mission and goals.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.