Evolution of Translational Omics Lessons Learned and the Path Forward (2012) / Chapter Skim
Currently Skimming:
Appendix B: Gene ExpressionBased Tests Developed at Duke University and Used in Clinical Trials
Appendix B: Gene ExpressionBased Tests Developed at Duke University and Used in Clinical Trials
Pages 239-280
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 239... ...
DNA microarray analysis became a powerful tool in the CAGP/CAGT for the study of regulatory pathways essential for cancer initiation and tumor growth, and researchers developed several gene expression–based tests to predict patient responses to chemotherapeutic agents and published the results. At a very early stage in the discovery research, such tests were taken into clinical trials.
|
From page 240... ...
Table B-1 outlines some information related to those trials. This appendix provides a concise summary of the research objectives and the approaches taken in developing several of the gene expression– based chemosensitivity tests implemented in the three clinical trials in Table B-1, and presents findings that provide important insights about processes that were in place at Duke University, to enlighten the development of and to provide motivation for many of the IOM committee's recommendations that are intended to enhance the integrity of future omicsrelated research.
|
From page 241... ...
of Early Stage the Use of Non-Small Cell Lung NSCLC in Patients Using Preoperative Cancer (NSCLC) in Genomic Expression Chemotherapy Patients Using a Profiles of Chemo for Early Stage Genomic Predictor of Sensitivity to Guide Breast Cancer Platinum Resistance Therapy to Guide Therapy Disease Breast cancer Lung cancer Lung cancer Start date April 2008 February 2007 October 2007 Trial listed in March 2008 July 2007 October 2007 ClinicalTrials.gov Patient accrual Intended 270 80 117 Actuala 56 47 24 Sponsor DOD Eli Lilly/Duke/NCI Eli Lilly/Duke Principal Paul K
|
From page 242... ...
tests for pemetrexed and vinorelbine sensitivity were used in the trial NCT00545948. For each, a brief explanation of test discovery and validation is provided, including information on the confirmation of the gene expression–based computational models; the availability of the data, metadata, computer code, and fully specified computational procedures used in the discovery and confirmation of the test; and whether the tests were locked down prior to progression to subsequent phases of test development.
|
From page 243... ...
Then, they used statistical methods to develop the gene expression–based signatures that would form the basis of the computational models in the tests. However, conflicting and confusing information in the papers and the cited references regarding the data and the statistical methods contributed to the inability of colleagues in the scientific community to understand and replicate the generation of the computational models (Baggerly, 2011; McShane, 2010a; Review of Genomic Predictors for Clinical Trials from Nevins, Potti, and Barry, 2009)
|
From page 244... ...
(2006a) reported using leaveone-out cross-validation to confirm the docetaxel computational model developed from drug sensitivity data derived from the NCI-60 breast cancer cell lines.
|
From page 245... ...
. However, when statisticians Keith Baggerly and Kevin Coombes attempted to assess the validity of the tests at the request of colleagues at MD Anderson Cancer Center who were interested in using the tests or the same approach to develop new tests, they found insufficient information to reproduce the published results, using the available data and the methods published in the Nature Medicine paper (Baggerly, 2011)
|
From page 246... ...
Used in Lung Cancer Patients in NCT00509366 The gene expression–based chemosensitivity test for cisplatin was published in the Journal of Clinical Oncology (Hsu et al., 2007) , along with a chemosensitivity test for pemetrexed; this paper has now been retracted because of the "inability to reproduce the experiments demonstrating a capacity of a cisplatin response signature to validate in either a collection of ovarian cancer cell lines or ovarian tumor samples" (Hsu et al., 2010, p.
|
From page 247... ...
Pemetrexed (Hsu et al., 2007) and Vinorelbine Chemosensitivity Tests Used in Clinical Trial of Lung Cancer Patients NCT00545948 The gene expression–based chemosensitivity test for pemetrexed was published in the Journal of Clinical Oncology (Hsu et al., 2007)
|
From page 248... ...
In September 2009, NCI was in the process of reviewing a revised clinical trial protocol from the Cancer and Leukemia Group B cooperative group (CALGB-30702) , which was proposing to use six of the Duke chemosensitivity tests in a clinical trial for patients with advanced lung cancer.
|
From page 249... ...
. In contrast to NCI's reviews, oversight committees at Duke did not recognize significant problems with the other Duke chemosensitivity tests, and allowed them to be used to direct therapy selection in clinical trials.
|
From page 250... ...
At Duke, it appeared that in some instances, gene expression–based tests were being used for patient management in clinical trials, while they simultaneously were being tested in other "preliminary" studies for their ability to predict results. This kind of problem arguably should have been apparent to -- and avoided by -- the PIs and their clinician colleagues.
|
From page 251... ...
When the Duke leadership was interviewed by the IOM committee on August 22, 2011, they stated that it is essential to be able to trust the PI because no audit system can totally overcome a fundamental lack of trust. In retrospect, they said that PIs must develop an appropriate culture with an accountability plan that must have (1)
|
From page 252... ...
The CAGT, in which Nevins and Potti worked, was embedded within the IGSP. At the time of the three clinical trials, Duke had an extensive clinical trials infrastructure within the Duke Cancer Center, which normally would have been responsible for oversight and data stewardship in oncology trials conducted at Duke.
|
From page 253... ...
Duke Cancer Center For most NCI-sponsored cancer centers, such as the Duke Cancer Center, there is a requirement for a highly structured process for conducting clinical trials. Components of this system include a Clinical Protocol Review Committee (CPRC)
|
From page 254... ...
In 2010, the University formed a Translational Medicine Quality Framework (TMQF) committee to make recommendations to University leadership on appropriate oversight policies for future omics research being tested in clinical trials (TMQF Committee, 2011a,b)
|
From page 255... ...
. The Duke IRB determined that an IDE was not needed when it did not receive a response from FDA.3 In retrospect, and with FDA guidance in 2010, the Duke IRB chair recognized that an IDE should have been obtained for the omics-based tests used in the trials because the tests were used to direct patient management in the clinical trials (Falletta, 2011)
|
From page 256... ...
. Duke leadership indicated that, whenever IP is filed, the institution, the COI committees, and the IRBs should be informed, but this was not routine procedure during the time of the design and conduct of the three clinical trials.
|
From page 257... ...
. However, the problems with the three clinical trials were not brought to the attention of the appropriate individuals within the university leadership through any of these whistleblowing channels.
|
From page 258... ...
At this time, NCI also had begun the process of reviewing protocol CALGB-30702 that had been submitted to NCI's CTEP. The proposed trial would have used six of the chemosensitivity tests to guide therapy in an advanced lung cancer trial.
|
From page 259... ...
In the 12/22/2009 report of the external statistical reviewers, a reference is made to the pemetrexed test: "In addition, we agree with Nevins and Potti that since the profile is not used in any of the clinical trials patients are not being endangered." This statement contradicts the fact that the pemetrexed test was being used to guide the choice of treatment in the trial NCT00545948 that opened in October 2007 even though, as noted previously, Potti had stated in his submission of R01CA131049-01A1 in March 2008 that the accuracy of the pemetrexed test had not been validated in independent patient samples. This suggests that the independent review process permitted the PIs to be in direct contact with the external independent statistical reviewers, allowing the PIs to provide misleading information to these reviewers.
|
From page 260... ...
Various individuals with biostatistical expertise were involved in the development of the omics-based tests used in the three clinical trials, but there was a lack of continuity in personnel. Numerous errors identified in the statistical methodology and analyses (Baggerly and Coombes, 2009; McShane, 2010a,b)
|
From page 261... ...
. The investigators then provided NCI with the data and computer code for the cisplatin test, but not for the pemetrexed test.
|
From page 262... ...
On October 22, 2010, Duke notified NCI that multiple validation datasets associated with the cisplatin test were corrupted. For example, William Barry explained in his August 22, 2011, testimony to the IOM committee that when he found the original source data for the ovarian cancer cell line drug sensitivity experiments, he was able to determine that the drug sensitivity measurements supplied to NCI for its evaluation of the cisplatin test differed from the true source data.
|
From page 263... ...
But NCI did not pursue this until later when the Duke external review appeared inadequate and after NCI determined that it was supplying partial funding for one of the three Duke clinical trials through an R01 grant to Potti. It was apparently only after discovering the funding tie through the grant that NCI believed it was justified in taking a more active role in the investigation and in requesting data and computer code to evaluate the tests.
|
From page 264... ...
. Meanwhile, the papers were used and cited by hundreds of other investigators.7 Ultimately, the Nature Medicine paper was retracted on January 7, 2011, based on the NCI's recommendation for a full review of all data associated with all of the predictors in the key papers that had been questioned or that had been used in clinical trials.
|
From page 265... ...
• Have NCI and other sponsors conducted appropriately comprehen sive investigations into any suspected scientific integrity issues? Through a search on ClinicalTrials.gov and on the NIH RePORTER, the committee identified a clinical trial at the Moffitt Cancer Center & Research Institute (MCC)
|
From page 266... ...
Jonathan Lancaster, currently an MCC investigator, was identified by Dalton as a coinvestigator on the trial. Lancaster was formerly at Duke University and a coauthor with Nevins and Potti on three of the papers retracted to date: the 2006 Nature Medicine paper and two 2007 Journal of Clinical Oncology papers (Dressmann et al., 2007; Hsu et al., 2007; Potti et al., 2006a)
|
From page 267... ...
Termination of the Moffitt trial was also mentioned by the October 23, 2009, Cancer Letter, where it was reported that a Moffitt spokesperson indicated the closure was unrelated to the controversy concerning the three Duke University clinical trials (Goldberg, 2009b)
|
From page 268... ...
Regarding development of gene expression–based chemosensitivity tests, validation requires steps to lock down important features, including hypotheses, computational models, and analysis plans. In order to protect patients from harm due to use of a faulty predictor, it is essential to follow the kind of scheme presented in Figure S-1 to confirm and then validate omics-based tests before launching clinical trials or offering them commercially for clinical use.
|
From page 269... ...
These themes are reflected in the detailed recommendations presented in Chapters 2-5. While further pursuit of the questions raised about the clinical trials and omics-based tests discussed in this appendix may be undertaken separately from the work of the IOM committee, the need for availability of data and computer code, the need to follow a rigorous test development and evaluation process prior to use of an omics-based test in clinical trials, and the responsibilities of investigators, institutions, journals, and funding agencies are clear lessons.
|
From page 270... ...
. December 2007 Lancet Oncology publishes "Validation of gene signatures that predict the response of breast cancer to neoadjuvant chemotherapy: A substudy of the EORTC 10994/BIG 00-01 clinical trial" (Bonnefoi et al., 2007)
|
From page 271... ...
. June 2008 Nature Medicine requests that Baggerly and Coombes 5/08 letter be sent to Potti and coauthors.b Nature Medicine rejects letter.c Lancet Oncology rejects letter.d July 2008 Genomic Directed Salvage Chemotherapy with Either Liposomal Doxorubicin or Topotecan entered on ClinicalTrials.gov (Identifier NCT00720096)
|
From page 272... ...
send a letter to McShane in response to some of her concerns about the LMS used in CALGB-30506.i McShane and Abrams reply with the conclusions of their analysis of the LMS in the CALGB-30506 clinical trial: The test should not remain as a stratification factor, and the coprimary aim to evaluate its performance should be removed from the study.j April 2010 CTEP requests data and computer code from Potti regarding R01 grant CA131049-01A1 titled "Prospective validation of genomic signatures of chemosensitivity in NSCLC" (cisplatin and pemetrexed tests) .k Potti responds to CTEP.l The Cancer Letter obtains a copy of Duke University's external review report from NCI via a Freedom of Information Act request and publishes the document (Goldberg, 2010a)
|
From page 273... ...
. 7/23 -- Lancet Oncology issues an expression of concern for "Validation of gene signatures that predict the response of breast cancer to neoadjuvant chemotherapy: A substudy of the EORTC 10994/BIG 00-01 clinical trial" (Bonnefoi et al., 2007)
|
From page 274... ...
Receipt of NMED-LE40837, May 30, 2008. bCommunication from Alison Farrell, Nature Medicine, to Keith Baggerly, MD Anderson Cancer Center.
|
From page 275... ...
Consensus review of revised protocol CALGB 30702: Genome-guided chemotherapy for untreated and treated advanced stage non-small cell lung cancer: A limited institution, randomized phase II study, November 9, 2009. gCommunication from Claudio Dansky Ullmann, National Cancer Institute, to Richard Schilsky, CALGB.
|
From page 276... ...
2007. Validation of gene signatures that predict the response of breast cancer to neoadjuvant chemotherapy: A substudy of the EORTC 10994/BIG 00-01 clinical trial.
|
From page 277... ...
2007b. Study Using a Genomic Predictor of Platinum Resistance to Guide Therapy in Stage IIIB/IV Non-Small Cell Lung Cancer (TOP0602)
|
From page 278... ...
2008. A Randomized Phase II Trial Evaluating the Performance of Genomic Expression Profiles to Direct the Use of Preoperative Chemotherapy for Early Stage Breast Cancer.
|
From page 279... ...
2010. Phase II Prospective Study Evaluating the Role of Directed Cisplatin Based Chemotherapy with Either Vinorelbine or Pemetrexed for the Adjuvant Treatment of Early Stage Non-Small Cell Lung Cancer (NSCLC)
|
From page 280... ...
2010. Phase II Prospective Study Evaluating the Role of Personalized Chemo therapy Regimens for Chemo-Naive Select Stage IIIB and IV Non-Small Cell Lung Cancer (NSCLC)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.