Evaluation of the Congressionally Directed Medical Research Programs Review Process (2016) / Chapter Skim
Currently Skimming:
Summary
Summary
Pages 1-16
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
It also recommended that CDMRP adopt a scientific peer review process modeled on the process used by NIH. In 1997, the IOM reviewed the DoD's progress in implementing the 1993 recommendations and provided additional guidance on enhancing the program's areas of research and its review processes.
|
From page 2... ...
The committee focused on the CDMRP processes used to select applications for funding by the research programs, specifically, those programs that receive funding only from appropriations to CDMRP. The review process for those research programs that are administered by CDMRP on behalf of the DoD Defense Health Agency's Joint Program Committees was not considered in detail as they have different research prioritization, planning activities, and approval processes from those of other CDMRP programs.
|
From page 3... ...
Although CDMRP leadership and staff responded to the committee's series of questions, the committee was constrained by a lack of some documentation from CDMRP staff and the almost complete lack of information from its two contractors, who provide administrative and managerial support for the programmatic and peer review panels. This was a substantive omission because the contractors are responsible for selecting, recruiting, training, and compensating peer and programmatic reviewers, and ensuring they have no conflicts of interest.
|
From page 4... ...
The CDMRP review cycle takes about 2 years from the annual appropriation of funds through award negotiation and implementation; however, the committee's evaluation focused on the application review cycle, the left side in Figure S-1. All CDMRP research programs follow five major steps for soliciting, reviewing, and funding applications (in yellow in Figure S-1)
|
From page 5... ...
Applications that are approved for funding move into the award negotiation and man agement phases. PARTICIPANTS IN THE PROCESS CDMRP uses an integrated program team to manage each research program and facilitate the communication and coordination of program activities.
|
From page 6... ...
Programmatic panel members typically serve 2-year terms and panel chairs typically serve for 1–3 years, but there is no limit on the number of terms that members or chairs can serve or on the consecutive terms they can serve. The committee appreciates that for some health conditions there may be relatively few organizations that are able to provide consumer representatives for programmatic panels, but it encourages CDMRP to strive to recruit more consumer reviewers from a variety of organizations in order to improve turnover and expand participation on the panels.
|
From page 7... ...
CDMRP uses mentors for consumer reviewers on peer review panels to supplement the reviewers' training, explain the review process to them, and assist them with their initial scoring and critiquing of applications. The committee finds this to be a good approach for new consumer peer reviewers, but it notes that mentors are not provided to either consumer reviewers on programmatic panels or to scientist reviewers on either panel.
|
From page 8... ...
The committee understands that having 1-year funding makes it difficult to guarantee that long-term goals can be achieved; however, the lack of a strategic plan means that each year a research program must establish its priorities anew, making it difficult to track program progress, and develop strengths and expertise placing increased reliance on the institutional memory of the long-serving programmatic panel members. Furthermore, the committee does not consider a research program's vision setting booklet or landscape document to be equivalent to a strategic plan.
|
From page 9... ...
participation of the CDMRP program manager on a variety of interagency groups with other governmental and nongovernmental members. During the vision setting and programmatic review processes, the programmatic panel assesses what other organizations are currently funding and the potential for CDMRP to fund duplicative research.
|
From page 10... ...
Additional CDMRP staff and contractor time and expertise may be required to implement these efforts, although the committee notes that some CDMRP research programs already undertake some of these efforts, such as consistently having representatives from NIH or VA on their programmatic panels. However, the development of a long-term strategic plan for each CDMRP program that includes input from other major funding organizations could reduce the risk of funding duplicative research and enhance opportunities for identifying complementary or collaborative research.
|
From page 11... ...
Although the CDMRP program managers stated that they try to obtain input from all relevant stakeholders, the committee does not see the need to limit public access to the stakeholders meeting to only invited attendees, as that may preclude input from interested parties, particularly those with singular or dissenting viewpoints. The committee finds that there are a variety of ways to obtain public input for stakeholders meetings such as announcing meetings on the CDMRP website, allowing Web-based participation, and encouraging electronic input from stakeholders.
|
From page 12... ...
As ad hoc and specialty peer reviewers may provide an overall score for an application and, thus, have a substantial impact on its rating, the committee finds the lack of acknowledgement of these reviewers to be puzzling. Providing the names of the ad hoc reviewers and more information on the CDMRP website on how and when ad hoc and specialty reviewers are used and selected would increase public confidence in the uniformity of reporting reviewers in the peer and programmatic review process.
|
From page 13... ...
The committee suggests that providing applicants with a standardized statement as to why the programmatic panel rejected their pre-application would improve the transparency of the program. Applicants who submit a full application receive the scores and summary statements from peer review, and the committee finds this to be an appropriate level of feedback for that review step.
|
From page 14... ...
Recommendation: CDMRP should consider updating and stan dardizing its scoring system to reflect current review practices and to reduce confusion among reviewers and applicants. Term Limits CDMRP does not impose term limits on members of either peer review or programmatic panels, however, the committee suggests that CDMRP consider standardizing the terms of service for panels.
|
From page 15... ...
Although it is worthwhile to periodically review any research program to ensure that current best practices are being used and to identify any needed updates or gaps, the committee emphasizes that there has been no known concerted effort by an external entity to determine whether the research funded by the various CDMRP research programs have in fact helped them accomplish their individual missions. Thus, perhaps the more fundamental question remains unanswered: Has CDMRP, even with a well-conducted review process, produced the innovative and impactful research it strives to fund?
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.