Evaluation of the Congressionally Directed Medical Research Programs Review Process (2016) / Chapter Skim
Currently Skimming:
8 Findings and Recommendations
8 Findings and Recommendations
Pages 115-134
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 115... ...
Congress, in response to advocacy groups and other interested parties, determines which programs will be funded and at what level each year. Beginning with the congressional appropriation for a Breast Cancer Research Program in 1992, CDMRP has expanded to encompass 29 research programs on diverse health conditions ranging from autism to prostate cancer (see Box 2-2)
|
From page 116... ...
On the basis of the information provided by CDMRP, discussions at the public sessions, and responses to the solicitation of input, the committee finds that, in general, the CDMRP review process has been effective in dispensing research funding across its programs and is not in need of extensive revisions. The solicitation of input comments received by the committee from both peer and programmatic reviewers, whether scientist or consumer, also indicated that the CDMRP review process worked well, and only minor modifications were suggested to enhance it.
|
From page 117... ...
CDMRP may wish to consider those practices as it addresses the following recommendations for developing a strategic plan and improving both the transparency and the standardization of its business processes. CONSUMER ENGAGEMENT The CDMRP process is innovative in that it includes consumer reviewers on both the peer review panels and the programmatic panels.
|
From page 118... ...
The committee disagrees with this reasoning. Two CDMRP programs -- the Breast Cancer Research Program and the Ovarian Cancer Research Program -- have developed landscape documents that provide a snapshot of the epidemiology of these diseases and describe selected research initiatives and outcomes.
|
From page 119... ...
The committee understands that having annual funding makes it difficult to guarantee that long-term plans will necessarily be implemented, but it also finds that this uncertain funding should not preclude the development of a strategic plan for a research program. Furthermore, the lack of a long-term strategic plan means that each year a research program establishes its priorities anew, making it difficult to track program progress and increasing reliance on the institutional memory of the programmatic panel members who serve for more than 1 year.
|
From page 120... ...
The chapter also includes a discussion of the efforts of CDMRP program managers to interact with colleagues at other research institutions for the purpose of identifying research gaps and possibly duplicative research. The committee notes that coordination is best accomplished when all involved parties work together.
|
From page 121... ...
The committee finds that these coordination efforts by CDMRP program managers are helpful but insufficient for providing consistent and current information on what other government and nongovernmental agencies are doing with regard to funding research on a particular health condition. The variability in the mission, funding, and public awareness of each CDMRP research program may preclude a specific, one-sizefits-all approach for coordinating research priorities, but the committee finds that a more formal and systematic approach to the coordination of research priorities with other funding organizations, and particularly NIH and VA, is possible.
|
From page 122... ...
There are four notable areas of the CDMRP review process, however, where transparency is lacking: stakeholders meetings, contractor support activities and policies, the use of ad hoc reviewers, and feedback from programmatic reviewers. On the basis of the information discussed in the following sections, the committee concludes that improving transparency in these areas would increase public knowledge of and engagement with the review process, beginning with the goals of the program, help
|
From page 123... ...
and convenes a meeting with them to explore the current state of research on the health condition and how the new CDMRP program might best address congressional intent and research gaps. The CDMRP process for identifying stakeholders and engaging them in a discussion of the health condition is not publicly available.
|
From page 124... ...
Contractor Support Activities and Policies External contractors provide support to CDMRP program managers for many peer review and programmatic review activities. Many government agencies, such as the U.S.
|
From page 125... ...
The committee finds that the minimum qualifications for scientist reviewers for both peer and programmatic reviewers should be publicly available. Second, by not controlling or owning its training materials (e.g., handbooks and videos)
|
From page 126... ...
Use of Ad Hoc and Specialty Reviewers Given the scope of several of the CDMRP research programs and the number of pre-applications and applications that need to be reviewed in a short period of time, the use of ad hoc and specialty reviewers is necessary to complete the review process in a timely and effective manner. In examining the CDMRP website for lists of peer and programmatic reviewers, the committee found only a few programs that identified these additional reviewers.
|
From page 127... ...
Those peer reviewers commented that they did not understand how the programmatic review part of the process worked, how peer review scores and summaries were used by the programmatic panels, or why peer reviewers were not informed as to which applications were ultimately recommended for funding by the programmatic review panel. Some peer reviewers felt that their input was not given sufficient weight in the programmatic review, particularly if a highly scored application was not funded.
|
From page 128... ...
A better understanding of what programmatic reviewers found lacking in an application would assist applicants in revising their applications for possible resubmission should the proposed research apply to one of the next year's award mechanisms, and it would provide peer reviewers with more insights on the programmatic review process. Given that CDMRP has programmatic review guidelines for each award mechanism that include details on how the programmatic review should be conducted for that award, the committee finds that the programmatic review overall score and panel discussion could be captured and summarized and provided to the applicant along with the peer review scores and summaries that are already provided.
|
From page 129... ...
. In those cases where peer and programmatic reviewers may not have sufficient expertise to comment competently on all aspects of an application CDMRP may use specialty reviewers who review portions or all of an application for such specialized areas as biostatistics and clinical trials.
|
From page 130... ...
For example, peer or programmatic reviewers might serve for a specific term (e.g., 1–3 years) with multiple, but finite, renewals, such as serving for two 3-year
|
From page 131... ...
In general, the committee found that many of the relevant recommendations had been implemented by CDMRP, including the establishment of a two-tiered review process for scientific merit and then program relevance, the provision of summary statements and peer reviews to the applicants, communicating to the scientific community about the role of consumer reviewers, and the inclusion of programmatic evaluation criteria and exclusionary parameters in program announcements. The committee notes that with the inclusion of pre-application submission and screening, the review process is now three-tiered and conducted by two separate panels.
|
From page 132... ...
appropriate tools to measure achievements and progress toward goals of the Cancer Research Program annually and over time" echoes the committee's current recommendation (discussed above) that each CDMRP research program develop a long-term strategic plan, with input from the stakeholders meeting, that is periodically updated.
|
From page 133... ...
• he six fundamental questions that the 1993 committee stated to guide T research remain a useful framework for future research emphasis. • Other recommendations: o Establish measures to ensure the continuation of the current strength of the integration panel; o Spell out in more detail in the broad agency announcements the types of proposals sought, the programmatic evaluation criteria, and exclusion ary parameters; o Lengthen the time between the release of the broad agency announce ment and the deadline for submission of proposals; o ncrease the time between receipt of applications and first-tier peer I review panel meetings; o Communicate detailed information about consumer participation in the Breast Cancer Research Program peer review process to the scientific community; o Move toward establishing standing review panels; o Improve feedback to applicants whose applications were not funded; o Establish a procedure for resubmission of unfunded applications; o Establish a procedure for competitive renewal applications; o Revise the application process to make it less cumbersome; o Reduce the time it takes between funding recommendation by the integration panel and actual awarding of funds to the investigator's institution; o treamline the annual reporting process and allow awardees more flex S ibility in changing experimental design and methodology; and o llow awardees flexibility in use of funds across spending categories.
|
From page 134... ...
In general, the committee found the CDMRP process for reviewing and selecting applications for funding to be effective and not dissimilar to the processes used by NIH, PCORI, and other funding organizations. Nevertheless, improvements could be made in the CDMRP review process, primarily in the areas of transparency and standardization, which would help to align CDMRP more closely with NIH and other organizations.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.