Enhancing Human Performance Issues, Theories, and Techniques, Background Papers (Complete Set) (1988) / Chapter Skim
Currently Skimming:
Pages 1-152
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
PART I Issues of Theory and Methodology
|
From page 2... ...
Human Performance Research: An Overview Monica J Harris and Robert Rosenthal Harvard University
|
From page 3... ...
Meta-Analysis of Expectancy Mediation eeee..eeeee Human Performance Technologies and Expectancy Effects ee e. Research on Accelerated Learning...
|
From page 4... ...
Many of the human performance technologies of interest in the present paper are relatively recent innovations, and thus may be especially susceptible to expectancy effects. In principle, expectancy effects could be investigated by introducing expectations as a manipulation in addition to the independent variable of theoretical interest.
|
From page 5... ...
We will focus specifically on the possible influence of interpersonal expectancy effects on several human performance technologies. The paper advances in three steps: First, we describe the methodological, theoretical, ant empirical issues relevant to the study of expectancy effects, including how expectancy effects are mediated.
|
From page 6... ...
Evidence for Interpersonal Expectancy Effects Although originally fraught with controversy, the existence of interpersonal expectancy effects is no longer in serious doubt. In 197B, Rosenthal ant Rubin reported the results of a meta-analysis of 345 studies of expectancy effects.
|
From page 7... ...
There were eight categories of expectancy studies: reaction time experiments, inkblot tests, animal learning, laboratory interviews, psychophysical judgments, learning and ability, person perception, and everyday situations or field studies. Although effect sizes varied across categories, the importance of expectancy effects within each category was firmly established.
|
From page 8... ...
It is not until later that the technique may be investigated by more impartial or skeptical researchers, who may be less prone to expectancy effects operating to favor the technique. Many of the human performance technologies of interest in the present paper are relatively recent innovations, and thus may be especially susceptible to expectancy effects.
|
From page 9... ...
Mediation of Interpersonal Expectancy Effects Basic Issues A primary question of interest with respect to expectancy effects is the question of mediation: How are one person's expectations communicated to another person so as to create a self-fulfilling prophecy? This question in turn can be broken down into two components.
|
From page 10... ...
Luckily, many studies address the mediation of expectancy effects, and we have conducted a meta-analysis of this literature (Harris & Rosenthal, 1985)
|
From page 11... ...
An intuitive way of understanding these effect sizes is given by the Binomial Effect Size Display (BESD; Rosenthal & Rubin, 1982)
|
From page 12... ...
With respect to the behavior-outcome link, again all four factors were statistically significant , but in terms of effect size, feedback did not seem to be very important: climate, r=.36; feedback, r-.07; input, r=.33; and output, r=.20. Human Performance Technologies and Expectancy Effects We now turn to a more focused discussion of the possible influence of expectancy effects on research on techniques for the enhancement of human performance.
|
From page 13... ...
method, and offer our assessment of the extent to which expectancy effects could be responsible for the observed learning gains. The SALT technique, an Americanized version of Lozanov's (1978)
|
From page 14... ...
In this phase of the SALT technique, then, expectancy effects are not an experimental artifact but rather are an explicit part of the experimental manipulation. Note, however, that these expectations are intrapersonal rather than interpersonal; they are the students' self-expectancies for their performance.
|
From page 15... ...
wanted to test SALT on a large class of adults learning English as a second language. Rather than randomly assigning students to the experimental and control conditions, though, she instead described the procedures for the two conditions to the 80 subjects and asked them to choose which class they preferred: the traditional teaching control class, or the experimental SALT class!
|
From page 16... ...
(This is a serious prob lem in teems of expectancy effects that is true of all the studies on SALT, and which we will discuss in more detail later.) The next serious error committed by this author was in the analysis of the results.
|
From page 17... ...
Again, there was no randomization of students to condition, the author of the study delivered the manipulation, and was not blind to the experimental condition of the students. These are characteristics that leave open the possibility of alternative hypotheses including expectancy effects.
|
From page 18... ...
There are two issues to address: the first is the general methodological adequacy of the studies, and the second (of more relevance to the goals of this paper) is the extent to which effects of SALT may actually be due to expectancy effects.
|
From page 19... ...
On the basis of the preceding discussion, we conclude that the empirical evidence on SALT is so methodologically weak that it remains an open question as to whether SALT is effective, a conclusion that makes asking about interpersonal expectancy effects as a possible rival hypothesis less urgent. Suppose, however, that we pretend that the results of these studies can be trusted.
|
From page 20... ...
For endogenous effects, on the other hand, we would want to acknowledge the role of expectancies ant see if we court apply the literature on expectancy effects to the SALT technique to make it even more effective. There is a very real possibility of exogenous expectancy effects in the SALT research.
|
From page 21... ...
None of these approaches solves the problem completely, but they would help. Clearly, endogenous expectancy effects play a prominent role in SALT in the guise of the positive self-expectancies elicited in the students.
|
From page 22... ...
This condition court also use tape-recortet relaxation exercises and class ma serial to minimize expectancies communicated during the presentation phase. We court then compare the results found in this condition against those found for the regular SALT technique.
|
From page 23... ...
19 representational systems corresponding to the five senses of sight, sound, touch, taste, and smell. The latter two systems are rarely uses, so most research focuses on differences among the visual, auditory, ant kinesthetic systems.
|
From page 24... ...
We are more interested in the latter category of studies for the purposes of this paper, as we want to determine whether expectancy effects might be operating when NLP principles are used in applies settings. However, it is also useful to review briefly the first category of studies and to evaluate the PRS concept on more general methodological grounds .
|
From page 25... ...
studies in which there is no prior assessment of PRS and counselors are instructed to either match or mismatch the subject's use of perceptual predicates as it occurs during the ongoing interaction. This distinction has important implications for the potential expression of interpersonal expectancy effects, so we will discuss prototypical research examples from each subcategory.
|
From page 26... ...
However, there are features of this experimental design that leave open the possibility that expectancy effects may be responsible for some of the positive results. The most striking characteristic is that both the interviewers and the experimenter were not blind either to the hypotheses under study or to the experimental condition of the subjects.
|
From page 27... ...
We do not know, of course, whether such differential behavior actually occurred in this study, but it is likely as nonverbal behavior is very difficult to monitor ant control in a standardized manner. Indeed, there is indirect evidence in this study to suggest that something else other than PRS matching, such as expectancy effects, was operating.
|
From page 28... ...
=4.96, p<.05, r=.29. Again ~ though ~ we are left with the question of to what extent these results may be due to expectancy effects.
|
From page 29... ...
The possibility that interviewers may unintentionally behave differently in the matching vs mismatching conditions, and thus the possibility of interviewer expectancy effects, is therefore eliminated. However, it must be noted that the kind of standardization suggested here involves an inevitable trade-off with experimental realism; watching a prepared videotape is not as natural and realistic as taking part in a one-on-one interview.
|
From page 30... ...
. This lack of support leaves open the very real possibility that interpersonal expectancy effects are responsible in part for the positive results found in the predicate tracking studies .
|
From page 31... ...
We now turn to a description of that research, providing a summary of the findings on mental practice and our assessment of how much the findings could be due to interpersonal expectancy effects. Fortunately for our purposes, Feltz and Landers (1983)
|
From page 32... ...
A third possibility, more relevant to the goals of this paper, is that if interpersonal expectancy effects are operating in this area of research, performance on cognitive tasks would most likely be more susceptible to influence from expectations than would strength tasks. Now that we have outlined some of the major issues ant findings of this research area, we turn to a more Be tailed description of some examples of typical research.
|
From page 33... ...
Subjects in the mental plus motor practice group were given the same instructions but also stood up and pretended to throw darts, moving their arms appropriately. Analyses of the posttest minus pretest gain scores showed that the mental practice group scored signif icantly higher than the control group, F(1,28)
|
From page 34... ...
If experimenters are expecting better performance in the mental practice condition, and they know which sub jects are using mental
|
From page 35... ...
they are receiving considerably greater amounts of experimenter time and attention, relevant to the possible mediation of expectancy effects. A better way of designing these studies would be to have a control group that spends the exact same amount of time in the lab and is treated as much as possible like the mental practice group so that the only difference between the two groups would be that one of them uses mental practice and the other does not.
|
From page 36... ...
As noted above, sub jects in the preparatory arousal condition performed equally as well if not better than subjects in the mental practice condition. The instructions given to subjects in the preparatory arousal condition were "...I would like you to 'emotionally charge-up.' In essence, psych yourself up for maximum performances by getting mad, aroused, pumped-up or charged-upt' (Gould ~ Weinberg , & Jackson ~ 1980 ; p.
|
From page 37... ...
Despite the attention lavishes on it there is considerable doubt and controversy about the ef ficacy of biofeedback. We will discuss some of the issues involved in that controversy, particularly those relevant to expectancy effects, and describe some examples of typical research.
|
From page 38... ...
For example, Jessup, Neufield, & Mersky ( 1979 ~ reviewed 28 studies and found no support for the specific benefit of biofeedback in reducing migraine headaches. Furthermore, this review found that the most promising results were obtained in uncontrolled studies, where the potential for expectancy effects or other experimental artifacts is greater.
|
From page 39... ...
They found that biofeedback was the most effective in reducing frontalis EMG levels, with a mean of .426, followed by other forms of relaxation, with a mean of .332, and control conditions, with a mean of .210. An analysis of variance accompanied by post-hoc comparisons was conducted on the 60 means, and it was concluded that biofeedback was significantly better than control conditions, but there was no significant difference between biofeedback and other forms of relaxation.
|
From page 40... ...
However, the F's for treatment condition in all these analyses were always less than 1.0, so it is unlikely that the biofeedback conditions differed significantly from the control conditions in efficacy (maximum possible F<5)
|
From page 42... ...
The former explanation is highly plausible, however, and has not been ruled out. What does the all the information covered so far indicate about the possible influence of interpersonal expectancy effects?
|
From page 43... ...
Even if positive experimenter expectations were an essential aspect of biofeedback therapy, however, we would still be interested in learning whether or not the actual machinery of biofeedback training contributed an independent benefit. Equally clear is the importance of endogenous expectancy effects in biofeedback , taking the form of the placebo ef feet or sub ject self-expectations.
|
From page 44... ...
There are masses of clinical and lab studies showing that biofeedback is effective, but many of these studies suffer from methodological and design flaws, and there are also many failures to replicate. We have also seen that experimenter- and subject self-expectancy effects are pervasive in this area.
|
From page 45... ...
The first point was in 1882 when the Society for Psychical Research was founded in London by a group primarily from Cambridge University. Among the distinguished presidents of this Society were William James, Henri Bergson, Lord Rayleigh, ant C.D.
|
From page 46... ...
42 of parapsychological investigations, we have confined our discussion to a focused domain of parapsychological inquiry: the ganzfeld experiments. Ganzfe Id Experiments In these experiments subjects typically are asked to guess which of four stimuli had been "transmitted" by an agent or sender with these guesses made under conditions of sensory restriction (usually white noise ant an unpatterned visual field)
|
From page 47... ...
13~. Stem-and-Leaf Display Table 3 shows a stem-and-leaf display of the 28 effect size estimates based on the direct hits studies summarized by Honorton (1985, p.
|
From page 48... ...
Relative to what we would expect from a normal distribution, we have studies that show larger positive and larger negative effect sizes than would be reasonable. Indeed, the two largest positive effect sizes are significant outliers at p<.O5, and the largest negative effect size approaches significance with a Dixon index of .37 compared to one of .40 for the largest positive effect size (Snetecor & Cochran, 1980, pp .
|
From page 49... ...
That many studies unretrieved seems unlikely for this specialized area of parapsychology (Hymen, 1985; Honorton, 1985~. Based on experience with meta-analyses in other domains of research (e.g., interpersonal expectancy effects)
|
From page 50... ...
. In that analysis, eight areas of expectancy effects were summarized; effect sizes (Cohen's d, roughly equivalent to Cohen's h)
|
From page 51... ...
. The subtle kinds of cues described by these early workers were just the kind we have come to look for in searching for cues given off by experimenters that might serve to mediate the experimenter expectancy effects found in laboratory settings (Rosenthal, 1966, 1985)
|
From page 52... ...
were judged significant overall by the investigators. This proportion of significant results was not significantly (or appreciably)
|
From page 53... ...
49 fount their way into the file drawers. In any case, it is nearly impossible to have an accurate estimate of the number of unretrieved studies or pilot studies actually conducted.
|
From page 54... ...
. However, the lack of independence of the studies court have implications for the estimation of effect sizes if a small proportion of the investigators were responsible for all the nonzero effects.
|
From page 55... ...
There was little evidence to suggest, however, that those investigators tending to conduct more studies obtained higher mean effect sizes; the F(1, 18) testing that contrast was 0.38, EM 54' r=.14.2 Conclusion On the basis of our summary ant the very valuable meta-analytic evaluations of Honorton (1985)
|
From page 56... ...
of Ganzfeld studies to the set of 28 summarized in this section. Accordingly we constructed Table 4a to investigate the effect on the mean and median effect sizes of omitting all the studies conducted by this investigator.
|
From page 57... ...
What is our best estimate of the "adjusted" or "true" effect size for each of these areas after taking into account the possibility of interpersonal expectancy effects and other methodological weaknesses? In attempting to answer these questions, we developed a situational taxonomy of the five areas of SALT, NLP, mental practice, biofeedback, and ESP.
|
From page 58... ...
, and the length of experimenter-subject interaction. Of these factors, random assignment and experimenter blindness in particular are the most important in determining the possibility that exogenous expectancy effects could have occurred.
|
From page 59... ...
Interestingly, there is a strong inverse relationship between the rated quality of an area and its mean effect size; the correlation coefficient is r(3~=-.85, p=.03, one-tailed. The last line of Table 5 gives our estimate of the "residual" effect sizes for each of the five areas, that is, our judgment of what the "true" effect size for an area would be after adjusting it for any possible bias due to expectancy effects or methodological weaknesses.
|
From page 60... ...
We feel there are several important implications of the realization that these areas are characterized by small effect sizes. The first is that "small" toes not mean "unimportant." Even the smallest (unadjusted)
|
From page 61... ...
But this is information worth gathering, because the selection ant training of subjects in these human performance technologies might be very different if we thought a given technology more or less affected all people in a normally distributed manner than if it affected only a portion of the population in a skewed manner. The shirt important implication concerns the nature of replication.
|
From page 62... ...
. Suggestions for Future Research Expectancy Control Designs Throughout this paper, we have offered our opinion on the extent to which interpersonal expectancy effects may be responsible for the results of atuties on various human performance technologies.
|
From page 63... ...
Analogous expectancy control designs could easily be used in research on the human performance technologies described in this paper. For example, experiments in the area of neurolinguistic programming on predicate matching could easily adopt an expectancy control design.
|
From page 64... ...
This second teacher could then be given false labels about which classes had received SALT training. This design, however, would not be able to address expectancy effects taking place during the actual SALT training, yet that is when expectancy effects are probably most prevalent.
|
From page 65... ...
Lastly, even beyond the issue of expectancy effects, increasing the number of experimenters increases the generality of the results. As mentioned in the SALT section, we can be more confident of a result if it was obtained by a larger number of people
|
From page 66... ...
Another strategy is to have the principal investigator observe the experimenters as they conduct their sessions. This will not by itself eliminate expectancy effects, but it would help in identifying unprogrammed, differential experimenter behaviors.
|
From page 67... ...
It is our hope that, in the future, studies in these areas can incorporate some of these suggestions ant thus produce results of which we can be more confident. Expectancies and the Enhancement of Human Performance If expectancy effects may be responsible for some of the results reported in human technologies research, then why not use positive expectations themselves as a means of enhancing human performance?
|
From page 68... ...
It is certainly feasible to identify behaviors associated with positive communications and to train teachers, supervisors, ant other people in leadership positions to use those behaviors. The meta-analysis of the mediation of interpersonal expectancy effects (Harris & Rosenthal, 1985)
|
From page 69... ...
The workshops focus first on educating teachers about interpersonal expectancy effects and then on training them in each of the 15 skills. A recent evaluation of the TESA program (Penman, 1982)
|
From page 70... ...
The decision of whether to pursue these programs depends in part on the Cost of the program compared to the cost of using a program specifically designed to enhance expectations. It also depends on how well the expectancy effects generalize from the laboratory to applied contexts ~ a ques tion that needs to be addressed empirically.
|
From page 71... ...
of these areas are not likely to be mate by proponents of an area who are so convinced of the utility of a technique that they do not entertain the possibility of negative results, or by critics who are so convinced that a 66 phenomenon toes not exist that they to not accept positive results. We support the spirit o' the meta-analytic approach, an approach that says "show me the data" and does not make premature judgments either about "what is surely true" or about "what cannot possibly be."
|
From page 72... ...
67 References Banner, C N., and Meadows, W
|
From page 73... ...
M 1983 The effects of mental practice on motor skill learning and performance: A meta-analysis.
|
From page 74... ...
1985 Mediation of interpersonal expectancy effects: 31 meta-analyses. Psvcholo~ical Bulletin 97: 363-386.
|
From page 75... ...
L 1986 Interpretation of significance levels and effect sizes by psycho logical researchers.
|
From page 76... ...
1985 Nonverbal cues in the mediation of interpersonal expectancy effects.
|
From page 77... ...
Journal of the American Society for Psychical Research 46:111-117. Snedecor, G
|
From page 78... ...
Truzzi, M
|
From page 79... ...
~ At. Author Notes Preparation of this paper was supported in part by the National Science Foundation.
|
From page 80... ...
From X To ~ Size (r) From ~ To Accepts ideas .28 36% 64X - _ Asks questions .17 41: 59: .24 38% 62: Correc tive feedback .24 3BX 62X -- - -- a Dis Lance .20 40: 60Z .45 28: 72% Encourages .09 46X 54% .41 30X 701 Eye contact .11 44X 56Z .32 34Z 66Z Gestures -.04 52% 48Z .31 351 66Z Input .26 37: 63% .33 33Z 671 Interaction duration .17 41% 59X .46 27X 73X Interac tion frequency .21 40: 60Z .21 39Z 61 Z Negat ive c lima te .29 36Z 64Z .36 32Z 68Z Nods .20 40X 60Z ~~ Off-task behavior .19 40: 60X .43 29: 71X Positive climate .21 39: 61X .40 30X 70Z Praise .12 44: 56Z .12 44Z 561 Smiles .12 44X 56X .29 35Z 651 Speech rate .04 4BX 52: .48 26Z 741 Wait time ~ -- - .
|
From page 82... ...
TABLE 3 Stem-and-Leaf Plot and Statistical Sugary of Ganzfeld Studies Employing Criterion of Direct Hits Cohen' ~ h _ Stem Leaf Sugary Statistics 1.4 4 Maximum 1.44 1 .3 3 Quartiles (Q ~ .42 1 .
|
From page 83... ...
78 TABLE 4 Proportion of Studies Reaching Critical Levels of Significance for Two Research Areas Expected Expectancy Gansfeldb Interval for Z Proportion Research Research Difference Unpredicted Direction -1.65 and below .05 .03 .07 +.04 Not Significant —1 .64 to +1.64 .90 .60 .50 -.10 Predicted Direction l +1.65 and above .05 .36 .43 +.07 +2.33 and above .01 .19 .25 +.06 +3 .09 and above .001 .
|
From page 84... ...
Table 4a . Ef feet on Effect Size (_)
|
From page 85... ...
79 TABLE 5 Situational Taxonomy of Human Performance Technologies Human Performance Techno logy Mental Biofeed SALT NAP Prac t ice back ESP Mean ef feet size (r) .29 .27 .23 .13 .14 N of studies on which our analysis is based 6 8 60 20 28 Approxima te number of studies in population 30 15 100 2300 28 Exogenous Fac tars Random ass ignment no yes yes yes yes Es blind to condition no rarely no rarely yes Adequate comparison groups or values no yes sometimes sometimes yes Lean length of inter- 32 1 hr 9.4 ses- 26 30 action per study hours signs days trials Endogenous Fac tars Se 1 f -expe c Lane ie s important?
|
From page 87... ...
Postscript We have been asked to respond to a letter from the committee raising questions about the presence and consequences of methodological flaws in the ganzfeld studies discussed by Honorton (1985) , Hyman (1985)
|
From page 88... ...
The out come variables were the significance level Z and the effect size Cohen Is h. Ihe adjusted canonical correlation was only .46, a magnitude that for two-predicted-from-six could have arisen under the null hypothesis 54 times out of 100 (F(12,40)
|
From page 89... ...
. For the original 28 studies plus the 10 new ones from Honorton's lab, the combined Z is now 7.10 and the mean effect size is now an h of .27.
|
From page 90... ...
. Enhancing human performance.
|
From page 92... ...
Fly- i- i '-a.
|
From page 94... ...
F.eply t.~~. F.~;~l-~-iFi: ~,,b~`s;tne3-i=.~ ctat:iC;t.i., ianc.; ,'7ave devised a nc~ml:ler ,~,f incii' c~t,-'r~ 5~-,L-' as inflc~en,~~: citatisti'.s t '-' assess the r,,bL~=itnesis '-' f a given d=`ta tJaEie.
|
From page 95... ...
F~ply C,-, 7,-,~;~.~.~.. ripe: ~ I Fit ~~ c: i .r3 en.
|
From page 96... ...
Feplv t:,:, F:,-IC I~.=, I i nc!
|
From page 97... ...
I;.ep:' 1 ~ 4~; ~ f :'i-, i.
|
From page 98... ...
Reply to Postscript 11 recommended that our sponsors establish a committee of experts to help them keep track and evaluate the work of Honorton, Jahn, Schmidt and other of the more promising parapsychologists.
|
From page 102... ...
Reply to Pos recommended that our sang .
|
From page 103... ...
Intuitive Judgment and th Dale Griffin Stanford University Evaluation of Evidence
|
From page 105... ...
(Bertrand Russe11,1969, p.l6) Intuitive judgment is often misleading.
|
From page 106... ...
Specific examples win focus on the difficulty of assessing claims for techniques designed to enhance human performance, especially those related to parapsychological phenomena. More generally, the themes will include why personal experience is not a trustworthy source of evidence, why people disagree about beliefs despite access to the same evidence, and why evidence so rarely leads to belief change.
|
From page 107... ...
The key difference between intuitive and scientific methods is that He measurement and analysis of the scientific invesi~gai~on are publicly Available, while ir~tuinve hypothesis-test~ng takes place inside one person's mind. Recent psychological research has examined ways in which intuitive judgment departs from formal models of analysis -- and in focusing on such "errors" and "biases", this research has pinpointed some natural mechanisms of human judgment.
|
From page 108... ...
Cog~ ve judgments have the same feeling of "truth" -- it iS difficult to believe that our personal experience does not perfectly capture the objective world. The systematic biases ~ win be discussing throughout this section operate at a basic and automatic level.
|
From page 109... ...
The p~ictiorm from the human judges and the statistical models were then compared win the actual outcomes. The cI=cal judges involved In these studies were exceedingly confident that statistical models based on obvious relationships could Dot capture the subtle strategies mat they had developed over yean of personal expenence.
|
From page 110... ...
This comb~nabon of demonstrably sum optimal judgments and continued confidence of the judges set the stage for Me two themes of me judgment literature: What is wrong with human judgment? and Why don't people naturally realize the limitanor~s of human intuitive judgment?
|
From page 111... ...
] - Intuition versus formal models: selecting the Connation One reason for the superiorly of statistical judgment is Mat it utilizes information based on me observed quantifiable relationship between the predictors and the outcome.
|
From page 112... ...
entitled Pygmalion in the Classroom. Teacher were given false information on the expected achievement of some of their students.
|
From page 113... ...
9 blind to our expectations or theories about how people will behave. Snyder (1981)
|
From page 114... ...
. Marks and Kamman ran 35 remote viewing studies of their own and discovered that despite me fact that both Me subjects and the judges were confident Mat they coed make the correct matchups, Me results were never statishc~y s~ficant.
|
From page 115... ...
Personal experience is Me only source of evidence In many areas of life, but it can never be decisive when pitted against objective quanutai~ve evidence.3 3 This is not to say mat objective quantitative evidence should necessanly overthrow singly held theones. When a theory parsimoniously explains a whole range of empirical phenomena, it is appropriate to be conservative about accepting new objective evidence that challenges it.
|
From page 116... ...
. The effect of salience -- the tendency to notice some things more than others because they are brighter, louder, unique, or noticeable in some other way -- underlies many of the failures of human judgment of probability and improbability.
|
From page 117... ...
13 much less certain about buying a Volvo, though your thousand case data-base has been incremented by only one. Further, when you lie In bed thinking about the pros and cons of buying a Volvo, the vivid stones of the dentist's brother-~n-law's car steaming, screeching and moaning come to mind wad greater impact than do the blue and white dots from Consumer Reports.
|
From page 118... ...
ThiS problem of He "diagnosticity" of individuating infomlanon is e~cplicidy taken into account in a statistical prediction model. Some piece of evidence that is present In almost an cases Of one category, and absent in almost an cases of the other, is highly diagnostic -- it differentiates one category from another.
|
From page 119... ...
15 The results showed that Me students p~icuons were based entirely on the s~milanty of me personality description to the stereotype of the profession. The predictions were the same in both conditions: the lack of diagnostic info~madon did not prevem We students from using me invalid infonnadon Hey had.
|
From page 120... ...
Yet each of us has a powerful illusion Hat our personal experience is valid and our conclusions about a demonstration am diagnostic of their truth. Many biases of personal experience work together to make us overly mpressed by exciting novel events.
|
From page 121... ...
An important part of judgment is moving beyond the immediate case or example, and generalizing our conclusions to some larger domain. 3- Intuition versus formal models: the generalizability of sample conclusions hnfonnal inference is prone to systematic bias in data collection -- either from simple cognitive attentional biases or from biases in retrieving information.
|
From page 122... ...
But note that the problems with this evaluation are public knowledge -- we know enough about the threats to the validity of such studies to qualify our conclusions, and ~ncorporate uncertainty into future policy. But He personal experience of any one person involved in this program would likely lead to strong beliefs about its efficacy without He appropriate caution -- and without publicly observable methods of evaluation.
|
From page 123... ...
than the more likely patterned altemadve. If people believe that random processes cannot produce sequences that look systematic, how do they respond lo patterns in random data?
|
From page 124... ...
If we have a bag with 99 amen balls and 1 red ball, the total sample space is made up of 100 balls. The probability is small, just 1 in 100, that we can pun the red ball out of the bag on our fist try.
|
From page 125... ...
"One's uniqueness in one's own eyes makes all the components of an event that happened to oneself seem like a singular combination. It is difficult to perceive one's own adventure as just one element in a sample space of people,
|
From page 126... ...
Intuitive judgments based on personae experience do not have these structures for learning from expenence. hnstead, human judgment seems designed as much for protecting the ego of the decision-maker as generating accurate predictions and assessments.
|
From page 127... ...
The explanations were created on the basis of false information, but humans are so good at composing causal scenarios that the 6 Built into this paradigm is the not-so-sub~e hint mat people are usually overconfident, but this is one of the most unponant messages mat decision analysts can give decision-makers.
|
From page 128... ...
72~. Human judgment allows lime room for uncertainty; it is set up to explain the world -- and to prevent the anxiety that comes with
|
From page 129... ...
The process of testing hypotheses through personal experience leads to certain common violations of the basic tenets of research design. A simple but non-obvious nde of correct research design is mat relationships can only be sum ported by examining aD four possible outcomes of a successJfai!
|
From page 130... ...
Another clear lesson for decision-makers: look at the summary of the whole dataset, don't get caught up in the excitement of getting a personal feel for the results -- and if you're testing yourself, keep an objective tally of all 4 possible outcomes. Another essential facet of research design that is neglected in He search for evidence Cough personal experience is He need for experiment control.
|
From page 131... ...
the experimental condition. Of course, this never occurs in personal experience -- as subject or researcher, we are always informed.
|
From page 132... ...
Without careful consideration of research design, people cannot help but bias the sample of data Hey collect. The risks of gathering evidence through personal experience: an ex~nple Part of me responsibility of He Committee on Techniques for He Enhancement of Human Performance has been to make site visits to venous research establishments.
|
From page 133... ...
. Further, Here is no a pnon definition of emotional response, so the sample space of successful outcomes is defined after the fact, malting probability estimates meaningless.
|
From page 134... ...
Depressives seems to be more willing to accept that Hey do not have control over random events, while normals show an "illusion of control". The illusion of corded is the belief that a person has control over chance events that have personal relevance ~anger, 1982~.
|
From page 135... ...
. People win wager more before they have tossed the dice than after the toss but before the result is disclosed.
|
From page 136... ...
Probably the most powerful force motivating our desire to protect our beliefs -- from others' attacks, from our own questioning, and from the challenge of new evidence -- is commitment. Commimnent as motivation Modem social psychology came lo public consciousness with the development of Leon Festinger's (1957)
|
From page 137... ...
People feel unpleasantly aroused when two cognitions are dissonant -- when they contradict one another -- or when behavior is dissonant with a stated belief. To avoid this unpleasant arousal, people win often react to disconfim~'ng evidence by strengthening Weir beliefs and creating more consonant explanations.
|
From page 138... ...
34 When we are committed to a belief, it is unpleasant to even consider that contradictory evidence may be true. ~ this sense, it is generally easier to be a skeptic in the face of novel evidence; skeptics may be overly conservative, but they are rarely held up to ridicule.
|
From page 139... ...
During the process of selling a product, a person observes his own claims for that product -- and unless the salesperson is content to conclude that he or she is motivated only by the money -- will likely conclude that he or she has very good reason tO believe in the quality of the produce Beyond perceptual and judgmental biases, misunderstandings of chance phenomena and motivated distortions, lies the essential reason why personal experience cannot be decisive: We can never determine the true cause of our behavior or our experience. When an experimenter manipulates variables, he or she is briefly omniscient.
|
From page 140... ...
This is appropriate to the extent Hat people are able to evaluate second-hand information rationally. But aD the atteni~onad biases that are active in our personal experience are doubly pernicious when we evaluate processed evidence because the media further emphasizes the trivia, emotionally gripping aspects of infomlation while ignoring or downplaying cautions and unexciting statistical summanes.
|
From page 141... ...
The only students who showed immediate attitude change were those who read the statements attributed to the scientist, the credible trusted source. A delayed measurement, however, showed that Pravda had as much eject on attitude change once the source was forgotten.
|
From page 142... ...
. Both stones developed Croup extremely unreliable personal experience (Rand)
|
From page 143... ...
They reasoned that common sense must dictate that mixed evidence should lead to a decrease in certainty In the beliefs of both partisan groups. But if partisans accept
|
From page 144... ...
Giving the same mixed evidence to two opposing groups may drive the partisans farther apart. How is intellectual and emotional rapprochement possible?
|
From page 145... ...
41 Another reaction to processed evidence is the perception of hostile media bias. Why should politicians from both ends of the spectrum believe that the media is particularly hostile to their side?
|
From page 146... ...
But when we rely on casual observation, personal experience and entenain~ng narratives as sources of evidence, we have too much room to create our own persuasive consnual of the evidence. Problems in Evaluanng Evidence [V: The Elect of Formal Research FonTIal research structure and quantitative analysis may not be me only, or best, route to "understanding" problems.
|
From page 147... ...
Kurt Back (1972) titled his personal history of the human potential movement "Beyond Words" but it could have been just as accurately called "Beyond Measurement".
|
From page 148... ...
This example is often used in introductory statistics' classes because it demonstrates that good research really makers In the world. It shows how opt bask on personal experience or even unconsoled research can cause the adoption or condnuanon of dangerous policies.
|
From page 149... ...
Because He Amoral studies were publicly available, and because the quality of the studies could be evaluated on the basis of their experunental method, the overall conclusions were decisive. Until the human potential movement agrees on the importance of quantitative evaluation, it win remain spUt into factions based on ideologies maintained by personal experience.
|
From page 150... ...
135~. Intuitive Judgment and the evaluation of evidence: A summary Personal experience seems a compeding source of evidence because it involves He most basic processing of information: perception, attention, and memory storage and retrieval.
|
From page 151... ...
But with the complex technological nature of our society, most researchers believe that some instruction on how ~ be a better consumer of infonnadon should start In public schools. The immediate response should be a renewed commitment to formal structures In deciding important policy, and a new realization that personal experience cannot be decisive in forming such policy.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.