An Assessment of ARPA-E (2017) / Chapter Skim
Currently Skimming:
Pages 91-136
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 91... ...
An Assessment of ARPA-E 4 Technical Assessment The statement of task for this study (Box 1-1 in Chapter 1) directs the committee to conduct a retrospective assessment and technical evaluation of ARPA-E, including the most significant accomplishments and impacts of the ARPA-E program and any unique features of the program that may have contributed to these accomplishments.
|
From page 92... ...
An Assessment of ARPA-E 92 AN ASSESSMENT OF ARPA-E ARPA-E TECHNICAL OBJECTIVES: TRANSFORMATIONAL TECHNOLOGIES AND WHITE SPACE Chapter 1 describes how ARPA-E was created to turn revolutionary advances and out-of-the-box ideas into transformational energy technologies to reduce energy imports and emissions, improve efficiency, and ensure that the United States maintains a technological lead in advanced energy technologies. The agency also describes itself as looking to fund projects that fill "white space" -- energy technologies not being fully addressed by the private sector or other federal research programs (ARPA-E, 2013b; Rohlfing, 2015)
|
From page 93... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 93 enough to disrupt its market by being more widely adopted than incumbent technologies, or by creating an entirely new market. Often the specific details -- cost, performance, application, size and scope of market, specific impact on society -- look different from what was imagined when the technology is in its early development stages.
|
From page 94... ...
An Assessment of ARPA-E 94 AN ASSESSMENT OF ARPA-E Overall, all of the technologies appeared to have disruptive potential when introduced. But only optical disks outperformed printed material when first introduced.
|
From page 95... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 95 White Space In developing and reviewing proposed programs, ARPA-E personnel make it clear that the agency seeks to fill white space as it develops and reviews its proposed programs (e.g., Rohlfing, 2015)
|
From page 96... ...
An Assessment of ARPA-E 96 AN ASSESSMENT OF ARPA-E While the distinction is not explicitly described by ARPA-E, the committee notes that funding white space technologies may or may not necessarily be the same as funding transformational technologies. This distinction is apparent in reviewing the breadth of ARPA-E's programs.
|
From page 97... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 97 director of DARPA, Arati Prabhakar, offered a cautionary note that there are "no viable metrics for judging success" for an agency seeking to bring about transformational innovations (Prabhakar, 2015)
|
From page 98... ...
An Assessment of ARPA-E 98 AN ASSESSMENT OF ARPA-E BOX 4-1 Continued point that others pick up and take over. Alternatively, such follow-on funding might be to fund DARPA-like scale demonstrations.
|
From page 99... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 99 the development and deployment of advanced energy technologies. Nonetheless, ARPA-E has completed more than 200 projects, and these projects provide a record of early accomplishments.
|
From page 100... ...
An Assessment of ARPA-E 100 AN ASSESSMENT OF ARPA-E Energy (EERE) (Goldstein, 2016)
|
From page 101... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 101 were three times as likely to publish as EERE awardees. Importantly, this latter finding is robust to the inclusion of control factors, such as the amount of funding awarded and project duration.
|
From page 102... ...
An Assessment of ARPA-E 102 AN ASSESSMENT OF ARPA-E duration of the research grant and how much time had elapsed since the beginning of the project period. During the time period investigated, ARPA-E's portfolio of projects resulted in more patents per project than the portfolios of either the Office of Science or EERE.
|
From page 103... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 103 of activity -- follow-on public funding, follow-on private funding, and new firm formation. Overall, there is evidence that market engagement metrics are particularly sensitive to organization type.
|
From page 104... ...
An Assessment of ARPA-E 104 AN ASSESSMENT OF ARPA-E Processes for Advanced Carbon Capture Technologies (IMPACCT) and Electrofuels programs have resulted in a publication.
|
From page 105... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 105 BOX 4-2 Summary of External Metrics Analysis Publications • ARPA-E awardees have published at a higher rate than EERE awardees and at a rate similar to that of Office of Science awardees. • ARPA-E awardees have produced more energy journal publications than Office of Science awardees.
|
From page 106... ...
An Assessment of ARPA-E 106 AN ASSESSMENT OF ARPA-E storage projects. In both instances, the committee aimed to select cases with a broad set of potential impacts.
|
From page 107... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 107 Program Objectives and Impacts Within the SWITCHES program, ARPA-E recognized the substantial potential for WBG semiconductor materials to yield energy savings and new designs for these applications relative to existing technologies. The goal of this program is to enable the development of high-voltage (approximately 200– 2,000 V)
|
From page 108... ...
An Assessment of ARPA-E 108 AN ASSESSMENT OF ARPA-E voltage. The right panel shows materials with better power device performance (high breakdown voltage and low loss)
|
From page 109... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 109 highly competent, dedicated, and responsive. Each project is subject to 10 or more "touches" annually by the program director and the staff front office team, including quarterly reviews, annual reviews, teleconferences, site visits, and other events.
|
From page 110... ...
An Assessment of ARPA-E 110 AN ASSESSMENT OF ARPA-E • examine whether the narrative ARPA-E portrays is reflected in reality; • compare data for this group of projects against the quantitative analyses applied to all ARPA-E projects; and • provide the other committee subgroups with additional data. Note that for this analysis, the committee examined only projects focused on storage of electricity.
|
From page 111... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 111 projects designed to explore improvements to battery management systems, sensors, or testing. Even with the analysis limited to energy, the important unknown remains of the extent to which ARPA-E considers prior electricity storage funding decisions and technical content in weighing the relative merits of new programs or projects focused on storage, particularly projects awarded through the OPEN solicitations.
|
From page 112... ...
An Assessment of ARPA-E 112 AN ASSESSMENT OF ARPA-E FIGURE 4-4 Focus of ARPA-E electricity storage projects funded 2009-2014. FIGURE 4-5 Degree of novelty and prior support for ARPA-E electricity storage projects.
|
From page 113... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 113 The committee also looked at the storyline of the projects, meaning the combination of the type of project lead and type of project. The majority of the projects (47 of 63)
|
From page 114... ...
An Assessment of ARPA-E 114 AN ASSESSMENT OF ARPA-E This is a worthy investment given the critical need for improved electricity storage in both the transportation and electricity sectors. The committee considers much of the electricity storage portfolio to have a medium or high degree of technical risk, meaning that its projects have goals that go beyond current technical capabilities and are in many cases difficult to achieve.
|
From page 115... ...
An Assessment of ARPA-E TABLE 4-1 ARPA-E Individual Project Case Studies Company Company Name Short Project Title OPEN/Focused Type Successes -- projects that the committee or the agency considers likely to have or that have had market success in the energy sector and that have received follow-on funding 1366 Technologies Cost-Effective Silicon Wafers for Solar Cells OPEN 2009 Small Co. Foro Energy Laser-Mechanical Drilling for Geothermal Energy OPEN 2009 Small Co.
|
From page 116... ...
An Assessment of ARPA-E 116 AN ASSESSMENT OF ARPA-E • "Cancelled" are projects that were ended before the original end date because they were not meeting their goals and appeared likely not to do so eventually. • "Other" are projects that were completed and that to date have resulted in little or no direct energy market success, but still advanced the state of knowledge.
|
From page 117... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 117 FIGURE 4-6 Funding history for 1366 Technologies. NOTES: The figure shows the timing and amounts of funding for the company's inception, for ARPA-E funding (purple)
|
From page 118... ...
An Assessment of ARPA-E 118 AN ASSESSMENT OF ARPA-E technology and select those with the greatest promise. After a 6-month exploratory effort, the team, together with ARPA-E, chose to develop a SLIPS coating for refrigeration coils that would reduce defrost cycles by enabling faster shedding of frost and water from the surface of the coils.
|
From page 119... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 119 identify such markets. Focusing on early market adoption made funding available to continue the research in more complex areas.
|
From page 120... ...
An Assessment of ARPA-E 120 AN ASSESSMENT OF ARPA-E FIGURE 4-8 Funding history for Smart Wires. NOTES: The figure shows the timing of the company's start-up, the ARPA-E funding (purple)
|
From page 121... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 121 ARPA-E's productivity and how its unique operations can contribute to its success. It should be noted that the committee heard presentations from current and former ARPA-E directors and program directors about other projects and programs, including others that could qualify as successes (e.g., Branz, 2015; Gur, 2015; Heidel and Gould, 2015; Majumdar, 2015; McGrath and Umstattd, 2015; Schiff and Zahlar, 2015; Williams, 2015b)
|
From page 122... ...
An Assessment of ARPA-E 122 AN ASSESSMENT OF ARPA-E number of benefits apart from avoiding duplication of existing research. It challenges established attitudes and entrenched constituencies and may alter pathways, barriers, and risks in key technology areas, and it builds community knowledge and expertise in new areas as well.
|
From page 123... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 123 The DARPA Model for Orchestrating Communities of Scientists The desire of ARPA-E to foster new communities of researchers in energy follows DARPA's lead in creating such communities. Fuchs (2010)
|
From page 124... ...
An Assessment of ARPA-E 124 AN ASSESSMENT OF ARPA-E birds-eye system-integrating perspective that is critical to integrating disparate activities across the nation's innovation ecosystem. Finally, throughout these activities -- whether bringing together members of research communities that may not normally interact or funding an entire suite of technologies necessary to achieve an integrated outcome -- DARPA program managers contribute a system-level perspective to organizing national R&D.
|
From page 125... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 125 management systems for strategic investors (ARPA-E, 2015c)
|
From page 126... ...
An Assessment of ARPA-E 126 AN ASSESSMENT OF ARPA-E their projects, the compelling nature of those projects and how they could be transformational often was unclear. The committee is concerned that the inability to deliver compelling messaging consistently in the language of the general public limits ARPA-E's ability to describe its programs and projects to the broader set of policy makers and the public at a time when energy issues often are at the forefront of public debates.
|
From page 127... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 127 • demonstrating the viability of existing scientific theory that is critical to energy transformation; • demonstrating that particular technological pathways should not be pursued; and • funding individual components of the energy system that may be underfunded and could hold back advancement throughout the system because of interdependencies with other components, including funding white space technologies, funding white space where other agencies or companies are not working, funding a portfolio of projects to ensure compatibility across the energy system, and funding activities for an existing technology to drive down its costs or the cost-to-performance ratio and enable its deployment to be transformational. Other metrics that should be considered for a longer term assessment include metrics of commercialization, technological or environmental impacts, lessons learned, and the creation of novel collaborations or new research communities.
|
From page 128... ...
An Assessment of ARPA-E 128 AN ASSESSMENT OF ARPA-E including disclosures and patents. Moreover, these traditional innovation metrics are poorly suited to evaluating ARPA-E's activities.
|
From page 129... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 129 technologies, as most of them originated before ARPA-E funding was forthcoming. In a number of cases, however, ARPA-E provided crucial early stage funding and frequent technical and market analysis expertise that enabled a company or idea to pass at least partially through the "valley of death," 4 as ARPA-E was established to do.
|
From page 130... ...
An Assessment of ARPA-E 130 AN ASSESSMENT OF ARPA-E have received patents. However, most transformative energy technologies require many years, often several decades, to go from nascent research to first marketable product.
|
From page 131... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 131 ARPA-E may eventually be assessable by the impacts on energy and environment of the technologies and products it has enabled, much as the Defense Advanced Research Projects Agency (DARPA) is judged.
|
From page 132... ...
An Assessment of ARPA-E 132 AN ASSESSMENT OF ARPA-E its overall portfolio between innovations that appear to have the potential to be transformative and other important opportunities that are being ignored despite their potential for impact. Recommendation 4-2: The director of ARPA-E should continue to promote and maintain a high-risk culture within the agency.
|
From page 133... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 133 existing technology to enable its adoption could be transformational to the energy system. ARPA-E's success also should not be measured based on whether a funded project has reached the market.
|
From page 134... ...
An Assessment of ARPA-E 134 AN ASSESSMENT OF ARPA-E scientific community, to aid in the review of existing programs, and to serve as a reference for future programs. In this last case, ARPA-E may need to consider producing both an internal document with confidential information available only to its program directors and a sanitized public document with no performer confidential information.
|
From page 135... ...
An Assessment of ARPA-E TECHNICAL ASSESSMENT 135 Recommendation 4-7: ARPA-E should increase and improve its communication for non-technical audiences, including the impact of its activities, the diversity of appropriate metrics to judge the success of individual projects and programs, and the fact that no single metric is appropriate for this purpose. Improving ARPA-E's Measurement and Assessment of Its Impact Finding 4-9: ARPA-E is not yet able to assess the full extent to which it has achieved its statutory mission and goals.
|
From page 136... ...
An Assessment of ARPA-E 136 AN ASSESSMENT OF ARPA-E empowered to develop and implement such a framework in a way that best serves its mission and goals. Recommendation 4-8: The ARPA-E director and program directors should develop and implement a framework for measuring and assessing the agency's impact in achieving its mission and goals.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.