An Assessment of ARPA-E (2017) / Chapter Skim
Currently Skimming:
Pages 161-182
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 161... ...
An Assessment of ARPA-E Appendix C Methods Used to Assess ARPA-E This chapter explains the methods and data used in this report's assessment of ARPA-E; further detail is in Appendixes D and G The methods used for the committee's operational and technical assessments were determined largely by the specific charges in the statement of task for this study (Box 1-1 in Chapter 1)
|
From page 162... ...
An Assessment of ARPA-E 162 AN ASSESSMENT OF ARPA-E TABLE C-1 Statement of Task and Corresponding Research and Evaluative Questions for the Operational Assessment Statement of Task Corresponding Research Question(s) Research/Descriptive: 1.
|
From page 163... ...
An Assessment of ARPA-E APPENDIX C 163 TABLE C-1 Continued Statement of Task Corresponding Research Question(s)
|
From page 164... ...
An Assessment of ARPA-E 164 AN ASSESSMENT OF ARPA-E TABLE C-1 Continued Statement of Task Corresponding Research Question(s) Research/Descriptive: 1.
|
From page 165... ...
An Assessment of ARPA-E APPENDIX C 165 TABLE C-2 Methods and Data Sources for Questions under Statement of Task Charge 1 (Evaluate ARPA-E's methods and procedures for developing and evaluating its portfolio of activities.) Research/Evaluation Methodological Data Sources Question Approach What are ARPA-E's • Review of program ARPA-E: methods and procedures for documentation, • Archival ARPA-E developing and evaluating including strategic planning documents its portfolio of activities?
|
From page 166... ...
An Assessment of ARPA-E 166 AN ASSESSMENT OF ARPA-E TABLE C-3 Methods and Data Sources for Statement of Task Charge 6 (Evaluate the success of the focused technology programs in spurring the formation of new communities of researchers in specific fields.) Research/Evaluatio Methodological Approach Data Sources n Question 1.
|
From page 167... ...
An Assessment of ARPA-E APPENDIX C 167 TABLE C-4 Methods and Data Sources for Statement of Task Charge 9 (Reflect, as appropriate, on the role of ARPA-E, DOE, and others in facilitating the culture necessary for ARPA-E to achieve its mission.) Research/Evaluation Methodological Data Sources Question Approach 1.
|
From page 168... ...
An Assessment of ARPA-E 168 AN ASSESSMENT OF ARPA-E C-4, the data for the research questions were to be obtained primarily from such sources as literature reviews, question-and-answer sessions with ARPA-E staff during panel presentations at committee meetings, conversations with ARPA-E awardees at agency events, interviews with former agency staff, case studies, and event observations. The evaluative questions were to be addressed through the expert assessment of committee members.
|
From page 169... ...
An Assessment of ARPA-E APPENDIX C 169 TABLE C-5 Consultations Conducted for ARPA-E Assessment Consulted Title Date Cadieux, Gena DOE, Deputy General Counsel for Transactions, 1/15/2016 Technology, & Contractor Human Resources Chalk, Steven Deputy Assistant Secretary for Renewable Energy 1/5/2016 Chu, Steven Secretary of Energy 2/15/2016 Danielson, David DOE, Assistant Secretary of Energy Efficiency & 12/3/2015 Renewable Energy (EERE) , and former ARPA-E Program Director Friedmann, Julio DOE, Principal Deputy Assistant Secretary for the 1/6/2016 Office of Fossil Energy Gur, Ilan DOE, former ARPA-E Program Director, currently 9/22/2015 founder of Cyclotron Road Hoffman, Patricia DOE Assistant Secretary for the Office of Electricity 1/6/2016 (OE)
|
From page 170... ...
An Assessment of ARPA-E 170 AN ASSESSMENT OF ARPA-E to the question of important factors related to ARPA-E's success, and was used for the initial outline of the chapter of this report on ARPA-E's internal operations (Chapter 3)
|
From page 171... ...
An Assessment of ARPA-E APPENDIX C 171 FIGURE C-1 Relationship between statement of task charges and themes that arose during consultations. NOTES: SoT = statement of task.
|
From page 172... ...
An Assessment of ARPA-E 172 AN ASSESSMENT OF ARPA-E The energy sector section was designed to capture more substantive information on projects and their potential impact. Committee members were asked to comment on and rate the following: • Coherent statement of the energy problem to be solved -- Was the energy problem easily understood and well researched?
|
From page 173... ...
An Assessment of ARPA-E APPENDIX C 173 projects are funded for roughly 3 years, so that at the time of this assessment, there were at most two cohorts of cases to examine. Therefore the focus was on evaluating ARPA-E's operations to assess whether the projects selected (1)
|
From page 174... ...
An Assessment of ARPA-E 174 AN ASSESSMENT OF ARPA-E FIGURE C-2 Statement of task charges and topics covered by case studies. NOTE: SoT = statement of task; T2M = technology-to-market.
|
From page 175... ...
An Assessment of ARPA-E APPENDIX C 175 directors and the T2M team; and other thoughts or insights the performer wished to provide. Box C-1 presents the list of questions used as a guide for the conversations with performers across the case studies to ensure coverage of a consistent set of topics.
|
From page 176... ...
An Assessment of ARPA-E 176 AN ASSESSMENT OF ARPA-E addition to determining whether ARPA-E used active management practices, the committee sought to assess whether these practices had an impact on internal and external measures of success of the projects. To address these questions, the committee worked with external consultants to investigate ARPA-E's practices for project selection and management and their impact on project outcomes through a detailed analysis of internal data provided by the agency.
|
From page 177... ...
An Assessment of ARPA-E APPENDIX C 177 milestones to determine whether program managers were using active program management. Program managers can adjust project milestones, budgets, and timelines.
|
From page 178... ...
An Assessment of ARPA-E 178 AN ASSESSMENT OF ARPA-E to ARPA-E support as of February 2016. They were used to create dummy variables for each of the six market engagement metrics as well as a dummy for at least one market engagement metric having been met.
|
From page 179... ...
An Assessment of ARPA-E APPENDIX C 179 cooperative agreement is different from a grant in that it entails "substantial involvement" between the agency and the recipient. Given that ARPA-E uses cooperative agreements as its primary mechanism for distributing funds, grants and cooperative agreements were the most relevant choice for this comparison.
|
From page 180... ...
An Assessment of ARPA-E 180 AN ASSESSMENT OF ARPA-E To determine whether a publication was highly cited, additional information was downloaded from Thomson Reuters to supplement the publication analysis: (1) a list of all indexed journals assigned to one of 22 categories, (2)
|
From page 181... ...
An Assessment of ARPA-E APPENDIX C 181 government affiliates (including national laboratories and other state/federal agencies)
|
From page 182... ...
An Assessment of ARPA-E 182 AN ASSESSMENT OF ARPA-E SUMMARY This appendix has provided details of the committee's framing of its task as well as specific methods used in this assessment. This discussion is intended to help the reader understand the operational and technical assessments that appear in Chapters 3 and 4, respectively.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.