Validation of Toxicogenomic Technologies A Workshop Summary (2007) / Chapter Skim
Currently Skimming:
Summary of the Workshop
Summary of the Workshop
Pages 1-38
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
1 The microarray technologies referred to in this report measure mRNA levels in biologic samples. DNA from tens of thousands of known genes (for example, genes that code for toxicologically important enzymes such as cytochrome P450)
|
From page 2... ...
had two related sections. Part 1 of the workshop, on current validation strategies and associated issues, provided background presentations on several components essential to the technical validation of toxicogenomic experiments including experimental design, reproducibility, and statistical analysis.
|
From page 3... ...
John Quackenbush, of the Dana-Farber Cancer Institute and cochair of the workshop, followed up with a discussion of the workshop concept and goals. The workshop concept was generated in response to the standing committee's and other groups' recognition that the promises of toxicogenomic technologies can only be realized if these technologies are validated.
|
From page 4... ...
Are conclusions dependent on the particular technology, platform, or method being used? Part 1: Current Validation Strategies and Associated Issues The first session of the workshop was designed to provide background information on the various experimental, statistical, and bioinformatics issues that accompany the technical validation of microarray analyses.
|
From page 5... ...
Experimental Design of Microarray Studies Kevin Dobbin, of the National Cancer Institute, provided an overview of experimental design issues encountered in conducting microarray assays. Dobbin began by discussing experimental objectives and explaining that there is no one best design for every case because the design must reflect the objective a researcher is trying to achieve and the practical constraints of the experiments being done.
|
From page 6... ...
He discussed the level of biologic and technical replication7 necessary for making statistically supported comparisons between groups. He also discussed issues related to the study design that arise when using dual-label microarrays,8 7 Biologic replicates are mRNA samples from separate individual subjects that were experimentally treated in an identical manner (for example, five mRNA isolates from each identically exposed animal)
|
From page 7... ...
. The ensuing workshop discussion on Dobbin's presentation focused on the interplay between using technical replicates and using biologic replicates.
|
From page 8... ...
, including the "lab effect,"10 practitioner experience, and use of different statistical-assessment and dataprocessing techniques to determine gene-expression levels. Irizarry's presentation focused on understanding the magnitude of the lab effect, and he described a study where a number of laboratories analyzed the same RNA samples to assess the variability in results (Irizarry et al.
|
From page 9... ...
Further questions addressed measurement error in microarray analyses and whether, because of the magnitude of this error, it was possible to detect small or subtle changes in mRNA expression. In response, Irizarry emphasized the importance of using multiple biologic replicates so that consistent patterns of change could be discerned.
|
From page 10... ...
, it is essential to accurately define the degrees of freedom. Hoffman pointed out that the degree of freedom is determined by the number of animal subjects and not the number of chips (when the chips are technical replicates that represent application of the same biologic sample to two or more microarrays)
|
From page 11... ...
are driven by mathematical algorithms and models that "learn" features in a training set (known members of a class) to develop diagnostic classifiers and then classify unknown samples based on those features.
|
From page 12... ...
(2004) , was discussed with an emphasis on prediction confidence and chance correlation.15 The decision forest approach is a consensus modeling method; that is, it uses several classifiers instead of a single classifier (hence, the decision forest instead of a decision tree)
|
From page 13... ...
Since combining several identical trees produces no gain, the rationale behind decision forests is to use individual trees that are different (that is, heterogeneous) in representing the association between independent variables (gene expression in DNA microarray, m/z peaks in SELDI-TOF data, and structural descriptors in SAR modeling)
|
From page 14... ...
The presentation concluded with the overall comment that validation in the regulatory arena is, for the most part, a prerequisite for regulatory acceptance of a new method. In response to the presentation, it was questioned whether regulatory agencies were required to go through the ICCVAM process before they could use or accept information from a new test.
|
From page 15... ...
It was suggested by a participant that one aspect of the testvalidation process (distribution of chemicals for testing) , as outlined in the presentation, would not work well in the field of toxicogenomics but that the distribution of biologic samples (for mRNA quantification)
|
From page 16... ...
Part 2: Case Studies: Classification Studies and the Validation Approaches The second session of the workshop featured case studies where mRNA expression microarray assays were used to classify compounds according to their toxicological mode of action. Authors of the original papers presented salient details of their studies, emphasizing validation techniques and concepts.
|
From page 17... ...
Overall, the conclusions of this study are that it was possible to 16 Hierarchical clustering groups similar objects into a sequence of nested partitions, where the similarity metric is predefined. In DNA microarray applications, the technique is used to identify genes with similar expression patterns.
|
From page 18... ...
The discussion emphasized that mRNA expression results have several layers of intertwined information that can complicate the analysis of factors eliciting gene-expression changes. Beyond the molecular targets that are specifically affected by a compound, there are expression changes associated with the pathology resulting from exposure (for example, necrosis or hypertrophy)
|
From page 19... ...
Rats from the control group and three groups that received different dose levels of each compound were sacrificed after 5 days of dosing, and liver extracts were tested in microarray assays. The purpose of the analysis was to correlate the short-term changes in gene expression with the longterm incidence of carcinogenicity (known from previous studies of these model compounds)
|
From page 20... ...
He also noted that recent mathematical algorithms and models have become increasingly better at class separation. Study Design and Validation Strategies in a Predictive Toxicogenomics Study Guido Steiner, of Roche Pharmaceuticals, presented a study that used microarray analyses to classify compounds by mode of toxicologic action (Steiner et al.
|
From page 21... ...
The gene-expression data from tests of the model toxicants became the training set for the supervised learning methods (in this study, support vector machines [SVMs]
|
From page 22... ...
BOX 5 Classification of Microarray Data Using Algorithms and Learning Methods Various methods are used to analyze large-scale gene-expression data. Unsupervised methods widely reported in the literature include agglomerative clustering (Eisen et al.
|
From page 23... ...
Steiner also stated that a compound classification model should not confuse gene-expression changes associated with a desired pharmacological effect with those from an unwanted toxic outcome. The SVM model addresses this concern based on the assumption that pharmacological action is compound specific and the toxic mechanism is typical for a whole class; if this is true, then the SVM will downgrade features associated with a compound-specific effect and find features for classification that work for all compounds within a class.
|
From page 24... ...
Although technical issues and validation techniques were discussed, many of the comments focused on biologic validation, including the extent to which microarray results indicated biologic pathways, the linking of gene-expression changes to biologic events, the different requirements of biologic and technical validation, the impact of individual, species and environmental variability on microarray results, and the use of microarray assays to evaluate the low-dose effects of chemicals. The primary themes of this discussion are presented here.
|
From page 25... ...
In this context, he noted the level of difficultly in drawing statistically sound conclusions from self-contained data sets and asked whether analyzing a fully populated database would create even greater complexities and whether it was possible to achieve sufficient statistical power from data mining. John Quackenbush suggested that a level of data standardization would be necessary to analyze a compiled database and that the quality of experiments in the database would exert a major influence.
|
From page 26... ...
However, when these gene sets were mapped biologically, there was complete overlap of the pathways in which those genes were involved, thus, there was good agreement in terms of the biologic pathways. John Quackenbush also commented on the results of recent studies presented in Nature Methods,21 where a variety of platforms were tested using the same biologic samples.
|
From page 27... ...
Joseph DeGeorge, of Merck Pharmaceuticals, also commented on the importance of understanding the biologic context of gene-expression changes in validation efforts. For instance, if a gene-expression signal is obtained from animal tests with no associated pathology, it is important to ask if the change in mRNA expression is the first step to pathology or just an adaptive response.
|
From page 28... ...
to demonstrate the validity of mechanistic or mode-of-action biomarkers. Thus, biologic validation will not be solely achieved with microarray technologies.
|
From page 29... ...
In fact, the wealth of information provided in microarray assays can permit researchers to find errors in study implementation, and he mentioned an experience where perturbations in expression profiles indicated improper animal feeding or watering. Sarah Gerould, of the U.S.
|
From page 30... ...
Ramos moderated a summary discussion where, to initiate the discussion, he asked whether participants were comfortable with technical validation of the microarray technologies and whether it is appropriate for the field to progress to focusing on biologic validation. Indeed, similar to the roundtable session, the ensuing discussion focused on issues surrounding biologic validation, and some participants brought up themes mentioned earlier, such as the need to define mRNA expression changes that do and do not constitute a negative effect and that genetic diversity will confound extrapolation between species and among humans.
|
From page 31... ...
Other participants also asserted that it was necessary to understand the context in which the term validation was being used. Yvonne Dragan, of FDA, said that it has been shown with microarray technologies that technical reproducibility can be achieved in the 23 May 2005, Volume 2, No.
|
From page 32... ...
Kerry Dearfield, of USDA, commented that the question of whether a technology was ready for prime time really meant it was ready to be accepted by the regulatory agencies. In this regard, Dearfield commented that the microarray technologies were not quite there yet.
|
From page 33... ...
Goodsaid stated that efforts were under way at FDA to develop an efficient and standard process to receive genomic information and to minimize the confusion regarding potential regulatory applications of the technologies. Wrap-Up Discussion To finish the workshop, John Quackenbush assembled several summary statements of themes he heard emerge from the workshop discussions and projected these for the audience (see Box 6)
|
From page 34... ...
2000. Knowledge-based analysis of microar ray gene expression data by using support vector machines.
|
From page 35... ...
2000. Support vector machine classification and validation of cancer tissue samples using microarray expression data.
|
From page 36... ...
2004. Acute molecular markers of rodent hepatic carcinogenesis identified by transcription profiling.
|
From page 37... ...
2004. Using decision forest to classify prostate cancer samples on the basis of SELDI-TOF MS data: Assessing chance correlation and pre diction confidence.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.