Evaluation of the Congressionally Directed Medical Research Programs Review Process (2016) / Chapter Skim
Currently Skimming:
5 Peer Review
5 Peer Review
Pages 77-90
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 77... ...
The peer review process is the standard by which the most meritorious science is developed, assessed, and distributed. This chapter describes the process of assigning peer reviewers to panels and applications; the criteria and scoring used by peer reviewers; activities that occur prior to, during, and after the peer review meeting; and the quality assurance procedures used by CDMRP to ensure that the applications are scored and critiqued correctly.
|
From page 78... ...
The peer review support contractor works with the CDMRP program manager to determine the number of peer review panels that will be required and the types of expertise that will be necessary to review the anticipated applications. Panel and Application Assignments Depending on the program and the number of award mechanisms and anticipated applications, multiple peer review panels -- tailored to the specific expertise required by the research program and award mechanism -- may be needed for a single funding opportunity or topic area (for example, if several applications propose to investigate a specific protein group or metabolic pathway)
|
From page 79... ...
. Because the review criteria differ to some extent by mechanism, having multiple panels allows peer reviewers to focus on the specific review criteria and programmatic intent of an award mechanism (Kaime et al., 2010)
|
From page 80... ...
Applicants also assign primary and secondary research classification codes to their applications; the codes can be used to assign applications to peer review panels, inform recruitment of peer reviewers, and balance panel workloads. Panel membership is confidential until the end of the review process for that funding year, when the names and affiliations of all peer reviewers for each research program are posted to the CDMRP website (CDMRP, 2016g)
|
From page 81... ...
In response to the committee's solicitation of input, several peer reviewers stated that they would like more time to review and critique their assigned applications. Preliminary critiques and scores are submitted to the electronic biomedical research application portal (eBRAP)
|
From page 82... ...
The overall and criterion scores are not combined or otherwise mathematically manipulated to connect them, but the overall score is expected to correspond to the individual criterion scores (for example, if the individual criteria receive scores in the excellent range, the overall score should also be in the excellent range)
|
From page 83... ...
The committee finds that because CDMRP funding announcements dictate the importance of each criterion for the overall score, it would be easier for reviewers to appropriately consider each criterion if the same scale were used for both overall and criterion scores. The committee notes that in 2009 the NIH, which reviews tens of thousands of applications per year, moved to a 9-point, whole-number scale (see Table 5-3)
|
From page 84... ...
However, the committee recognizes that the preliminary peer review scores, including the overall scores, can be revised by a reviewer during the full panel discussion of an application. THE PEER REVIEW MEETING There are five possible formats for conducting scientific peer review: onsite, in-person peer review panels; online/virtual peer review panels; teleconference peer review panels; video teleconference peer review
|
From page 85... ...
At the start of peer review panel meetings, the program manager presents a plenary briefing for all reviewers which includes an overview of CDMRP, the history and background of the program, the award mechanisms and their intent, the goals and expectations of the review process, and a summary of how the peer review deliverables inform the programmatic review panel. Program managers observe the performance of the panel as a whole as well as the performance of the individual peer reviewers, the panel chair, and the scientific review officers.
|
From page 86... ...
Applications designated for expedited review are not discussed at the plenary peer review meeting unless the application is championed. An expedited application may be championed by any member of the peer review panel and will immediately be added to the docket for full panel discussion (Salzer, 2016c)
|
From page 87... ...
. Program managers review the summary statements and a final scoring report and may request rewrites from reviewers or the contractor if, for example, the critiques and summary do not match the scores or a summary statement is inadequate.
|
From page 88... ...
. Summary statements are reviewed to ensure that there is concordance among the evaluation criteria, overall scores, and the written critiques; that key issues of the panel discussion were captured; and that the critiques are appropriate for each criterion.
|
From page 89... ...
and the technological interfaces used for review; and whether reviewers have recently submitted an application to another CDMRP research program or to another award mechanism in the same research program. Scores and comments are compiled and used by the peer review contractor and CDMRP program managers to assess the peer review process and to evaluate the program announcement for future modifications -- for example, to clarify overall intent of the award, focus areas, or peer review criteria.
|
From page 90... ...
The consumer reviewers meet separately as a group at the end of the meeting to debrief on their experience and provide feedback to CDMRP through the consumer review administrator on how the experience could be improved (CDMRP, 2016f)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.