Keeping Score for All The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessments (2004) / Chapter Skim
Currently Skimming:
5 Available Research on the Effects of Accommodations on Validity
5 Available Research on the Effects of Accommodations on Validity
Pages 85-102
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 85... ...
(1999a) , for example, called for a research agenda that includes studies of "the need for particular types of accommodations and the adequacy and appropriateness of accommodations applied to various categories of students with disabilities and English-language learners" and "the validity of different types of accommodations" (National Research Council, 1999a, pp.
|
From page 86... ...
In Chapter 6 we present the committee's view of the way the validity of inferences based on accommodated assessments can best be evaluated. EFFECTS OF ACCOMMODATIONS AND THE INTERACTION HYPOTHESIS The committee commissioned a review and critique of the available research on the effects of test accommodations on the performance of students with disabilities and English language learners in order to gauge both any discernible trends in this research and the thoroughness with which the issues have been studied.
|
From page 87... ...
. For example, if both native English speakers and English language learners benefit from a plain-language accommodation, does that mean that the scores are not valid for English language learners who received this accommodation?
|
From page 88... ...
Sireci and his colleagues were given criteria for including a study in their review, specifically that the study should examine the effects of test accommodations on test performance and should involve empirical analyses. The authors found that while the literature on test accommodations is "vast and passionate," with some authors arguing against accommodations on the grounds that they are unfair and others arguing in favor of them, only a subset of the literature explicitly addressed the effects of accommodations on performance using empirical analyses.
|
From page 89... ...
65) point out that in this study, "the gap was narrowed because native English speakers scored worse on the linguistically modified test, not because the English language learners performed substantially better." In addition, in a study by Abedi (2001a)
|
From page 90... ...
90 a Interaction Detected Yes Yes Yes No Yes Yes Yes others students students students for students not for for for disabilities learning for disabilities large benefited but gains to both gains gains with without with gains with for disabilities disabilities and aloud disabilities Results Greater with Moderate improvement with Greater students Gains with disabilities Read students disabilities Greater with Significant students only between- between treatment treatment and design and design different Design Single-subject alternating design Single-subject alternating design Between-groups design Five studies Between-groups design Within- groups Within- groups Disabilities to with various mark time, encouragement, extra print, Students manipulatives assistance, aloud on oral, large administration, place, reads time, spelling time time time, Studies Accommodations Extra "packages" Encouragement, individual oral, maintain Extra Extra Extra student Oral Oral with with with with with with with Experimental Sample of Characteristics of Students disabilities Students disabilities Students learning disabilities Students learning disabilities Students learning disabilities Students disabilities Students disabilities Summary Elliott, (2000)
|
From page 91... ...
91 Partial No No No No Partial No on students students for for students benefit response without for gains gains disabilities and disabilities gains gains gains gains Greater with No Similar with disabilities No No Differential constructed items No with between- between- between- between and design and design and design and design Between-groups design Within- groups Within- groups Within- groups Unclear Between-groups design Within- groups time, transcription, extra English reading aloud, selected Oral Oral Oral Screen Simplified Calculators, reading teacher Multiday/session with with with with with with with Students disabilities Students disabilities Students disabilities Students disabilities Students disabilities Students learning disabilities Students disabilities 5-1. and and Thurlow Figure Eaton, Karns in and (2000)
|
From page 92... ...
92 not and students, but on disabled extended retesters extended (medium retesters those formats disabilities level. disabled with with than item between learning standard disabilities standard grade conditions differences accommodated test more without for lower for for learning retesters retesters No with across for than learning than those lower between 0.40)
|
From page 93... ...
93 to without. would for those advantage who constructed than on between conditions unfair an more lower in determining gap in differences scored result size not accurate unaccommodated narrowed did students not effect and disabilities gains accommodation items were with from education group significant significant Accommodations response Teachers benefit No accommodated either Students Accommodations special No facto facto facto facto post post post post Ex Ex Ex Ex Quasi- experimental visual, large Braille, native specific Various Oral Oral English, and language dictionaries, scribes, print, oral Not with with with with with Students disabilities Students disabilities Students disabilities Students disabilities Students disabilities Elliot Hamilton Tindal Bryant and Kimball, (2001)
|
From page 94... ...
94 a Detected eighth for sample Interaction Only grade No No Yes No No No on gain helped learners Learners grade narrowed assoc. all language language others on Small eighth glossary gains for performance significant in English gains language groups helped no Language grade.
|
From page 95... ...
95 factor between to confirmatory correlation Yes Partial No N/A No Unable determine a The cluded under in kers. learners spea language learners publication English language English 2000 language native English both The and for others English conditions both and (2000)
|
From page 96... ...
In fact, logic suggests in this case that the accommodated version of the test measures a skill that is quite different from the intended one. The fact that the accommodation affects English language learners and native English speakers differently may have any number of explanations.
|
From page 97... ...
The researchers concluded that in most cases the scores of test-takers who received accommodations were roughly comparable to scores obtained by nondisabled test-takers under standard conditions. The one major exception to this conclusion involved test-takers who were granted extended time.
|
From page 98... ...
One of the planned studies involves examining the cognitive processes required for NAEP items through the use of a "think aloud" approach. NAEP-like items would be administered to small groups of students with disabilities and English language learners in a cognitive lab setting.
|
From page 99... ...
and the criterion validity studies proposed for NAEP, the determination of whether the test-criterion relationships are the same for accommodated and unaccommodated administrations is complicated by the confounding of disability or English language learner status and accommodation status. That is, in a natural setting, those who use accommodations are likely to be students with disabilities or English language learners, and they are likely to differ from other students on many dimensions.
|
From page 100... ...
point out that students with and without disabilities are likely to differ in course-taking patterns, and that this disparity should be taken into account when comparing the degree to which accommodated and unaccommodated SAT scores predict college grade-point average. Moreover, the differences among students with disabilities and English language learners in course-taking, general courses of study, teacher assignments, the instructional methods they are likely to experience, and the like, all compound the difficulty of obtaining usable criterion variables.
|
From page 101... ...
Based on these findings, the committee believes that a program of research is needed that would systematically investigate the extent to which scores obtained from accommodated and unaccommodated test administrations are comparable and support similar kinds of inferences about the performance of students with disabilities and English language learners on NAEP and other large-scale assessments. RECOMMENDATION 5-1: Research should be conducted that focuses on the validation of inferences based on accommodated assessments of students with disabilities and English language learners.
|
From page 102... ...
Without a well articulated validation argument that explicitly specifies the claims and intended inferences that underlie NAEP and other assessments, and that also explicitly specifies possible counterclaims and competing inferences, research into the effects of accommodations on assessments of students with disabilities and English language learners is likely to consist largely of a series of loosely connected studies that investigate various accommodations, more or less at random. An approach and a framework for articulating a logical validation argument for an assessment is discussed in the next chapter.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.